VideoPipe中集成多模态大模型做视频(图片)分析

切沽邮哨前言

1.1 开发该框架的动机

? OpenGL ES 是一个渲染指令接口集合,每渲染一帧图像都是一系列渲染指令的排列组合。常用的渲染指令约有 70 个,记住这些渲染指令及其排列组合方式,是一件痛苦的事情。另外,在图形开发中,经常因为功耗、丢帧等问题需要性能优化,如何从框架层面进行性能优化是一件有挑战的问题。

? 基于上述原因,笔者手撕了一个 nimi 版的渲染框架,将这些常用的渲染指令有条理地封装、组织、归类,方便愉快并高效地进行 OpenGL ES 渲染开发。笔者在 OpenGL ES 领域从业也有些时日,对现有碎片化的知识进行归纳凝练,形成系统的认知,是件势在必行的事。

1.2 一个 mini 版的渲染框架应该具备哪些能力

? 一个 mini 版的渲染框架需要对 OpenGL ES 的常用指令进行归类(如下图),封装 EGL、error check、Shader Program、Mesh、VAO、VBO、IBO、Texture、FBO 等类,方便开发者快速开发渲染程序,将更多的注意力聚焦在业务上,而不是如何去组织 OpenGL ES 指令上。

img

1.3 为什么强调 mini 版渲染框架

? 从渲染指令的角度来看,OpenGL ES 3.0 约有 300 个渲染指令,本文框架只封装其中最常用的 70 个,指令覆盖程度仍有较大提升空间。

? 从功能的角度来看,笔者深知一个成熟完备的渲染框架应该具备相机、光源、光照模型(Lambert、Phong、PBR 等)、阴影、射线拾取、重力、碰撞检测、粒子系统等功能。

? 鉴于上述原因,笔者审慎地保留了 "mini" 前缀。

1.4 本框架的优势

? 本框架具有以下优势。

封装友好:对常用的 EGL 和 GL 指令(约 70 个)进行封装,提供了 EGL 环境搭建、着色器程序生成、网格构建、纹理贴图、离屏渲染、异常检测等基础能力,方便开发者快速开发渲染程序,将精力从繁杂的渲染指令中解放出来,将更多的注意力聚焦到业务上。

代码规整:框架中多处设计了 bind 和 unbind 接口,用于绑定和解绑 OpenGL ES 状态机相关 "插槽",如:VBO、IBO、VAO 中都设计了 bind 和 unbind 接口,ShaderProgram、Texture、FBO、TextureAction 中都设计了 bind 接口;另外,在 FBO 中设计了 begin 和 end 接口,很直观地告诉用户夹在这中间的内容将渲染到 FBO。接口规整简洁,方便用户记忆。

易于扩展:定义了 TextureAction 接口,并提供 bind 函数,GLTexture、FBO 都继承了 TextureAction,用户自定义的渲染器或特效类也可以继承 TextureAction,将它们统一视为纹理活动(可绑定),这在特效叠加(或后处理)中非常有用,方便管理多渲染目标图层,易于扩展。

性能高效:封装了 VBO、IBO、VAO,用于缓存顶点数据、索引、格式等信息到显存,减少 CPU 到 GPU 的数据传输,提高渲染效率;缓存了 attribute 和 uniform 变量的 location,避免 CPU 频繁向 GPU 查询 location,进一步提高渲染效率;基于 C++ 语言实现渲染框架,代码执行效率较高。

跨平台:基于 C++ 语言实现,具有更好的跨平台特性;封装了 core_lib,使得平台相关头文件可以轻松替换;封装了 Application,使得平台相关 api 可以轻松替换。

方便调试:设计了 EGL_CALL 和 GL_CALL 两个宏,对每个 EGL 和 GL 指令进行异常检测,方便调试渲染指令,并且通过预编译宏 DEBUG 开关动态控制是否生成异常检测的代码,Release 版本会自动屏蔽异常检测代码,避免带来额外功耗。

2 渲染框架

? 经过深思熟虑,笔者给该渲染框架命名为 glcore,命名空间也是 glcore。Windows 上 OpenGL 环境搭建主要有 GLFW / freeglut + Glad / GLEW 方案,详见 → Windows上OpenGL环境搭建,本文采用目前广泛使用的 GLFW + Glad 方案。Android 版本的 glcore 实现详见 → 在Android上手撕一个mini版的渲染框架。

? 本文完整资源(包含 glcore 框架和第 4 节的应用)详见 →【OpenGL ES】一个mini版的Windows渲染框架。

2.1 框架结构

img

2.2 CMakeLists

? CMakeLists.txt

设置库名

set(LIB_GL_CORE_NAME "glcore")

递归添加源文件列表

file(GLOB_RECURSE GL_CORE_SOURCES ./src/ *.cpp)

添加预构建库

add_library({LIB_GL_CORE_NAME} {GL_CORE_SOURCES})

将当前目录设为公共头文件目录 (任何链接glcore库的目标都能自动获得这个头文件路径)

target_include_directories(${LIB_GL_CORE_NAME} PUBLIC ./)

2.3 核心头文件

? 核心头文件分为对内和对外的,即内部依赖 core_lib,外部开放 core。

? core_lib.h

#pragma once

/**

* glcore 依赖的核心 GL 库, 便于将 glcore 移植到其他平台

* Android: EGL + GLESv3

* Windows: glfw / freeglut + glad / glew

*

* @author little fat sheep

*/

#include

#include

? 之所以要单独拎出 core_lib.h,是为了方便将该框架迁移到其他平台,如 Android 上依赖的三方渲染库是 EGL + GLESv3,如果不抽出 core_lib.h,就需要将很多地方的 glfw3.h + glad.h 改为 egl.h + gl3.h ,工作量大,也容易漏改。另外,还可以很方便地替换渲染环境,如将 glfw3.h + glad.h 替换为 freeglut.h + glew.h。

? core.h

#pragma once

/**

* glcore核心头文件

* 该头文件是留给外部使用的, glcore内部不能使用, 避免自己包含自己

* @author little fat sheep

*/

// OpenGL ES API

#include "core_lib.h"

// glcore 核心头文件

#include "application.h"

#include "elg_surface_view.h"

#include "format.h"

#include "frame_buffer_object.h"

#include "gl_inspector.h"

#include "gl_texture.h"

#include "mesh.h"

#include "mesh_utils.h"

#include "shader_program.h"

#include "texture_action.h"

#include "vertex_attribute.h"

? core.h 只提供给外部使用,方便外部只需要包含一个头文件,就能获取 glcore 的基础能力。

2.4 Application

? Application 主要用于管理全局环境,使用单例模式,方便获取一些全局的变量。它也是 glcore 中唯一一个依赖平台相关的接口(除日志 log 接口外),如:m_window 是 Windows 特有的,如果将 glcore 迁移到 Android 中,就需要将该变量替换为 ANativeWindow* 类型,将这些平台相关变量都集中在 Application 中,迁移平台时修改起来也比较容易,避免太分散容易漏掉。另外,还可以很方便地替换渲染环境,如渲染平台替换为 freeglut 时,需要将 GLFWwindow 替换为 void*,因为 freeglut 未提供类似 window 的数据结构。

? application.h

#pragma once

#include "core_lib.h"

#define app Application::getInstance()

namespace glcore

{

/**

* 应用程序, 存储全局的参数, 便于访问

* @author little fat sheep

*/

class Application {

private:

static Application* sInstance;

public:

int width = 0;

int height = 0;

float aspect = 1.0f;

private:

GLFWwindow* m_window = nullptr;

public:

static Application* getInstance();

~Application();

void resize(int width, int height);

GLFWwindow* getWindow() { return m_window; }

void setWindow(GLFWwindow* window);

void releaseWindow();

private:

Application() {};

};

} // namespace glcore

? application.cpp

#include "glcore/application.h"

namespace glcore

{

Application* Application::sInstance = nullptr;

Application *Application::getInstance()

{

if (sInstance == nullptr)

{

sInstance = new Application();

}

return sInstance;

}

Application::~Application()

{

}

void Application::resize(int width, int height)

{

this->width = width;

this->height = height;

this->aspect = (float) width / (float) height;

}

void Application::setWindow(GLFWwindow* window)

{

m_window = window;

int width, height;

glfwGetFramebufferSize(window, &width, &height);

resize(width, height);

}

void Application::releaseWindow()

{

if (m_window)

{

m_window = nullptr;

}

}

} // namespace glcore

2.5 GLInspector

? GLInspector 主要用于异常信息检测,另外设计了 EGL_CALL 和 GL_CALL 两个宏,分别对 EGL 和 GL 指令进行装饰。如果定义了 DEBUG 宏,就会对每个 EGL 和 GL 指令进行异常检测,方便调试代码;如果未定义了 DEBUG 宏,就不会进行异常检测。

? 用户可以在 CMakeLists.txt 中添加预编译宏 DEBUG,这样就可以根据 Release 和 Debug 版本自动构建不同的版本。

if (CMAKE_BUILD_TYPE STREQUAL "Debug")

添加预编译宏

add_definitions(-DDEBUG)

endif ()

? gl_inspector.h

#pragma once

#include "core_lib.h"

#ifdef DEBUG

#define EGL_CALL(func) func;GLInspector::checkEGLError();

#define GL_CALL(func) func;GLInspector::checkGLError();

#else

#define EGL_CALL(func) func;

#define GL_CALL(func) func;

#endif

namespace glcore

{

/**

* OpenGL ES命令报错监视器

* @author little fat sheep

*/

class GLInspector

{

public:

static void checkEGLError(const char* tag); // 检查EGL配置

static void checkEGLError(); // 通用检查EGL错误

static void printShaderInfoLog(GLuint shader, const char* tag); // 打印Shader错误日志

static void printProgramInfoLog(GLuint program, const char* tag); // 打印Program错误日志

static void checkGLError(const char* tag); // 检查GL报错信息

static void checkGLError(); // 通用检查GL错误

};

} // namespace glcore

? gl_inspector.cpp

#include

#include

#include

#include "glcore/core_lib.h"

#include "glcore/gl_inspector.h"

// 以下内容为了不报错临时添加的 (glfw/freeglut已经创建了egl环境)

#define EGL_SUCCESS 0x3000

#define EGL_NOT_INITIALIZED 0x3001

#define EGL_BAD_ACCESS 0x3002

#define EGL_BAD_ALLOC 0x3003

#define EGL_BAD_ATTRIBUTE 0x3004

#define EGL_BAD_CONFIG 0x3005

#define EGL_BAD_CONTEXT 0x3006

#define EGL_BAD_SURFACE 0x3007

#define EGL_BAD_DISPLAY 0x3008

#define EGL_BAD_CURRENT_SURFACE 0x3009

#define EGL_BAD_NATIVE_PIXMAP 0x300A

#define EGL_BAD_NATIVE_WINDOW 0x300D

#define EGL_BAD_PARAMETER 0x300C

#define EGL_BAD_MATCH 0x3010

#define EGL_CONTEXT_LOST 0x300E

// 以上内容为了不报错临时添加的

#define TAG "GLInspector"

int eglGetError() { return 0; }

using namespace std;

namespace glcore

{

void GLInspector::checkEGLError(const char *tag)

{

int error = eglGetError();

if (error != EGL_SUCCESS) {

printf("%s: %s failed: 0x%x\n", TAG, tag, error);

}

}

void GLInspector::checkEGLError()

{

GLenum errorCode = eglGetError();

if (errorCode != EGL_SUCCESS) {

string error;

switch (errorCode)

{

case EGL_BAD_DISPLAY:

error = "EGL_BAD_DISPLAY";

break;

case EGL_NOT_INITIALIZED:

error = "EGL_NOT_INITIALIZED";

break;

case EGL_BAD_CONFIG:

error = "EGL_BAD_CONFIG";

break;

case EGL_BAD_CONTEXT:

error = "EGL_BAD_CONTEXT";

break;

case EGL_BAD_NATIVE_WINDOW:

error = "EGL_BAD_NATIVE_WINDOW";

break;

case EGL_BAD_SURFACE:

error = "EGL_BAD_SURFACE";

break;

case EGL_BAD_CURRENT_SURFACE:

error = "EGL_BAD_CURRENT_SURFACE";

break;

case EGL_BAD_ACCESS:

error = "EGL_BAD_ACCESS";

break;

case EGL_BAD_ALLOC:

error = "EGL_BAD_ALLOC";

break;

case EGL_BAD_ATTRIBUTE:

error = "EGL_BAD_ATTRIBUTE";

break;

case EGL_BAD_PARAMETER:

error = "EGL_BAD_PARAMETER";

break;

case EGL_BAD_NATIVE_PIXMAP:

error = "EGL_BAD_NATIVE_PIXMAP";

break;

case EGL_BAD_MATCH:

error = "EGL_BAD_MATCH";

break;

case EGL_CONTEXT_LOST:

error = "EGL_CONTEXT_LOST";

break;

default:

error = "UNKNOW";

break;

}

printf("checkEGLError failed: %s, 0x%x", error.c_str(), errorCode);

assert(false);

}

}

void GLInspector::printShaderInfoLog(GLuint shader, const char* tag)

{

char infoLog[512];

glGetShaderInfoLog(shader, 512, nullptr, infoLog);

printf("%s: %s failed: %s\n", TAG, tag, infoLog);

}

void GLInspector::printProgramInfoLog(GLuint program, const char* tag)

{

char infoLog[512];

glGetProgramInfoLog(program, 512, nullptr, infoLog);

printf("%s: %s failed: %s\n", TAG, tag, infoLog);

}

void GLInspector::checkGLError(const char *tag) {

GLenum error = glGetError();

if(error != GL_NO_ERROR) {

printf("%s: %s failed: 0x%x\n", TAG, tag, error);

}

}

void GLInspector::checkGLError()

{

GLenum errorCode = glGetError();

if (errorCode != GL_NO_ERROR)

{

string error;

switch (errorCode)

{

case GL_INVALID_ENUM:

error = "GL_INVALID_ENUM";

break;

case GL_INVALID_VALUE:

error = "GL_INVALID_VALUE";

break;

case GL_INVALID_OPERATION:

error = "GL_INVALID_OPERATION";

break;

case GL_INVALID_INDEX:

error = "GL_INVALID_INDEX";

break;

case GL_INVALID_FRAMEBUFFER_OPERATION:

error = "GL_INVALID_FRAMEBUFFER_OPERATION";

break;

case GL_OUT_OF_MEMORY:

error = "GL_OUT_OF_MEMORY";

break;

default:

error = "UNKNOW";

break;

}

printf("checkError failed: %s, 0x%x\n", error.c_str(), errorCode);

assert(false);

}

}

} // namespace glcore

2.6 EGLSurfaceView

? EGLSurfaceView 主要承载了 EGL 环境搭建。EGL 详细介绍见 → 【OpenGL ES】EGL+FBO离屏渲染。

? 由于 GLFW 中已经创建了 EGL 环境,EGLSurfaceView 实际上是多余的,为了保持 glcore 在各个平台上的代码具有高度一致性,并且方便随时自主创建 EGL 环境,这里仍然保留了 EGLSurfaceView。本文中主要使用到 EGLSurfaceView 的内部类 Renderer,起到定制渲染流程规范的作用。

? elg_surface_view.h

#pragma once

#include "core_lib.h"

// 以下内容为了不报错临时添加的 (glfw/freeglut创建了egl环境)

typedef void* EGLDisplay;

typedef void* EGLConfig;

typedef void* EGLContext;

typedef void* EGLSurface;

// 以上内容为了不报错临时添加的

namespace glcore

{

/**

* EGL环境封装类

* glfw和freeglut中创建了egl环境, 因此该类可以去掉,

* 但是为了方便将glcore迁移到需要用户搭建egl环境的平台中,

* 暂时保留该类.

*

* @author little fat sheep

*/

class EGLSurfaceView

{

public:

class Renderer;

private:

Renderer* m_renderer = nullptr;

EGLDisplay m_eglDisplay;

EGLConfig m_eglConfig;

EGLContext m_eglContext;

EGLSurface m_eglSurface;

bool m_firstCreateSurface = true;

public:

EGLSurfaceView();

~EGLSurfaceView();

void setRenderer(Renderer* renderer);

bool surfaceCreated();

void surfaceChanged(int width, int height);

void drawFrame();

void surfaceDestroy();

private:

void createDisplay();

void createConfig();

void createContext();

void createSurface();

void makeCurrent();

public:

/**

* 渲染器接口, 类比GLSurfaceView.Renderer

* @author little fat sheep

*/

class Renderer {

public:

virtual ~Renderer() {};

virtual void onSurfaceCreated() = 0;

virtual void onSurfaceChanged(int width, int height) = 0;

virtual void onDrawFrame() = 0;

};

};

} // namespace glcore

? elg_surface_view.cpp

#include

#include "glcore/application.h"

#include "glcore/elg_surface_view.h"

#include "glcore/gl_inspector.h"

#define TAG "EGLSurfaceView"

// 以下内容为了不报错临时添加的 (glfw/freeglut已经创建了egl环境)

#define EGL_RED_SIZE 0x3024

#define EGL_GREEN_SIZE 0x3023

#define EGL_BLUE_SIZE 0x3022

#define EGL_ALPHA_SIZE 0x3021

#define EGL_DEPTH_SIZE 0x3025

#define EGL_RENDERABLE_TYPE 0x3040

#define EGL_OPENGL_ES3_BIT 0x0040

#define EGL_SURFACE_TYPE 0x3033

#define EGL_WINDOW_BIT 0x0004

#define EGL_NONE 0x3038

#define EGL_CONTEXT_CLIENT_VERSION 0x3098

#define EGL_DEFAULT_DISPLAY ((EGLDisplay)0)

#define EGL_NO_DISPLAY ((EGLDisplay)0)

#define EGL_NO_CONTEXT ((EGLContext)0)

#define EGL_NO_SURFACE ((EGLSurface)0)

typedef int32_t EGLint;

EGLDisplay eglGetDisplay(EGLDisplay a1) { return nullptr; }

void eglInitialize(EGLDisplay a1, int* a3, int* a4) {}

void eglChooseConfig(EGLDisplay a1, const EGLint* a2, EGLConfig* a3, EGLint a4, EGLint* a5) {}

EGLContext eglCreateContext(EGLDisplay a1, EGLConfig a2, EGLContext a3, const EGLint* a4) { return nullptr; }

EGLSurface eglCreateWindowSurface(EGLDisplay a1, EGLConfig a2, void* a3, const EGLint* a4) { return nullptr; }

void eglMakeCurrent(EGLDisplay a1, EGLSurface a2, EGLSurface a3, EGLContext a4) {}

void eglSwapBuffers(EGLDisplay a1, EGLSurface a2) {}

void eglDestroySurface(EGLDisplay a1, EGLSurface a2) {}

void eglDestroyContext(EGLDisplay a1, EGLContext a2) {}

void eglTerminate(EGLDisplay a1) {}

// 以上内容为了不报错临时添加的

namespace glcore

{

EGLSurfaceView::EGLSurfaceView()

{

printf("%s: init\n", TAG);

createDisplay();

createConfig();

createContext();

}

EGLSurfaceView::~EGLSurfaceView()

{

printf("%s: destroy\n", TAG);

if (m_renderer)

{

delete m_renderer;

m_renderer = nullptr;

}

if (m_eglDisplay && m_eglDisplay != EGL_NO_DISPLAY)

{

// 与显示设备解绑

EGL_CALL(eglMakeCurrent(m_eglDisplay, EGL_NO_SURFACE, EGL_NO_SURFACE, EGL_NO_CONTEXT));

// 销毁 EGLSurface

if (m_eglSurface && m_eglSurface != EGL_NO_SURFACE)

{

EGL_CALL(eglDestroySurface(m_eglDisplay, m_eglSurface));

delete &m_eglSurface;

}

// 销毁 EGLContext

if (m_eglContext && m_eglContext != EGL_NO_CONTEXT)

{

EGL_CALL(eglDestroyContext(m_eglDisplay, m_eglContext));

delete &m_eglContext;

}

// 销毁 EGLDisplay (显示设备)

EGL_CALL(eglTerminate(m_eglDisplay));

delete &m_eglDisplay;

}

app->releaseWindow();

delete app;

}

void EGLSurfaceView::setRenderer(Renderer *renderer)

{

printf("%s: setRenderer\n", TAG);

m_renderer = renderer;

}

bool EGLSurfaceView::surfaceCreated()

{

printf("%s: surfaceCreated\n", TAG);

app->resize(app->width, app->height);

createSurface();

makeCurrent();

if (m_renderer && m_firstCreateSurface)

{

m_renderer->onSurfaceCreated();

m_firstCreateSurface = false;

}

return true;

}

void EGLSurfaceView::surfaceChanged(int width, int height)

{

printf("%s: surfaceChanged, width: %d, height: %d\n", TAG, width, height);

app->resize(width, height);

if (m_renderer)

{

m_renderer->onSurfaceChanged(width, height);

}

}

void EGLSurfaceView::drawFrame()

{

if (!m_eglSurface || m_eglSurface == EGL_NO_SURFACE || !m_renderer)

{

return;

}

m_renderer->onDrawFrame();

EGL_CALL(eglSwapBuffers(m_eglDisplay, m_eglSurface));

}

void EGLSurfaceView::surfaceDestroy()

{

printf("%s: surfaceDestroy\n", TAG);

if (m_eglDisplay && m_eglDisplay != EGL_NO_DISPLAY)

{

// 与显示设备解绑

EGL_CALL(eglMakeCurrent(m_eglDisplay, EGL_NO_SURFACE, EGL_NO_SURFACE, EGL_NO_CONTEXT));

// 销毁 EGLSurface

if (m_eglSurface && m_eglSurface != EGL_NO_SURFACE)

{

EGL_CALL(eglDestroySurface(m_eglDisplay, m_eglSurface));

m_eglSurface = nullptr;

}

}

app->releaseWindow();

}

// 1.创建EGLDisplay

void EGLSurfaceView::createDisplay()

{

EGL_CALL(m_eglDisplay = eglGetDisplay(EGL_DEFAULT_DISPLAY));

EGL_CALL(eglInitialize(m_eglDisplay, nullptr, nullptr));

}

// 2.创建EGLConfig

void EGLSurfaceView::createConfig()

{

if (m_eglDisplay && m_eglDisplay != EGL_NO_DISPLAY)

{

const EGLint configAttrs[] = {

EGL_RED_SIZE, 8,

EGL_GREEN_SIZE, 8,

EGL_BLUE_SIZE, 8,

EGL_ALPHA_SIZE, 8,

EGL_DEPTH_SIZE, 8,

EGL_RENDERABLE_TYPE, EGL_OPENGL_ES3_BIT,

EGL_SURFACE_TYPE, EGL_WINDOW_BIT,

EGL_NONE

};

EGLint numConfigs;

EGL_CALL(eglChooseConfig(m_eglDisplay, configAttrs, &m_eglConfig, 1, &numConfigs));

}

}

// 3.创建EGLContext

void EGLSurfaceView::createContext()

{

if (m_eglConfig)

{

const EGLint contextAttrs[] = {

EGL_CONTEXT_CLIENT_VERSION, 3,

EGL_NONE

};

EGL_CALL(m_eglContext = eglCreateContext(m_eglDisplay, m_eglConfig, EGL_NO_CONTEXT, contextAttrs));

}

}

// 4.创建EGLSurface

void EGLSurfaceView::createSurface()

{

if (m_eglContext && m_eglContext != EGL_NO_CONTEXT) {

EGL_CALL(m_eglSurface = eglCreateWindowSurface(m_eglDisplay, m_eglConfig, app->getWindow(), nullptr));

}

}

// 5.绑定EGLSurface和EGLContext到显示设备(EGLDisplay)

void EGLSurfaceView::makeCurrent()

{

if (m_eglSurface && m_eglSurface != EGL_NO_SURFACE)

{

EGL_CALL(eglMakeCurrent(m_eglDisplay, m_eglSurface, m_eglSurface, m_eglContext));

}

}

} // namespace glcore

2.7 ShaderProgram

? ShaderProgram 主要用于编译 Shader、链接 Program、设置 attribute 属性、更新 uniform 属性。

? glGetAttribLocation、glGetUniformLocation 两个接口需要 CPU 向 GPU 查询 location 信息,并且会频繁调用,为提高性能,笔者设计了 m_attributes 和 m_uniforms 两个 map 存储 name 到 location 的映射,方便快速获取 location,避免 CPU 频繁与 GPU 交互,以提高渲染性能。

? shader_program.h

#pragma once

#include

#include "core_lib.h"

using namespace std;

namespace glcore

{

/**

* 着色器程序

* @author little fat sheep

*/

class ShaderProgram

{

public:

static constexpr char* ATTRIBUTE_POSITION = "a_position"; // 着色器中位置属性名

static constexpr char* ATTRIBUTE_NORMAL = "a_normal"; // 着色器中位法线性名

static constexpr char* ATTRIBUTE_COLOR = "a_color"; // 着色器中颜色属性名

static constexpr char* ATTRIBUTE_TEXCOORD = "a_texCoord"; // 着色器中纹理坐标属性名

static constexpr char* ATTRIBUTE_TANGENT = "a_tangent"; // 着色器中切线属性名

static constexpr char* ATTRIBUTE_BINORMAL = "a_binormal"; // 着色器中副切线属性名

static constexpr char* UNIFORM_TEXTURE = "u_texture"; // 着色器中纹理名

static constexpr char* UNIFORM_VP = "u_projectionViewMatrix"; // 着色器中VP名

private:

GLuint m_program;

map m_attributes;

map m_uniforms;

public:

ShaderProgram(const char* vertexCode, const char* fragmentCode);

~ShaderProgram();

void bind();

GLuint getHandle() { return m_program; }

// 操作attribute属性

void enableVertexAttribArray(const char* name);

void enableVertexAttribArray(int location);

void setVertexAttribPointer(const char* name, int size, int type, bool normalize, int stride, int offset);

void setVertexAttribPointer(int location, int size, int type, bool normalize, int stride, int offset);

void disableVertexAttribArray(const char* name);

void disableVertexAttribArray(int location);

// 操作uniform属性

void setUniformi(const char* name, int value);

void setUniformi(int location, int value);

void setUniformi(const char* name, int value1, int value2);

void setUniformi(int location, int value1, int value2);

void setUniformi(const char* name, int value1, int value2, int value3);

void setUniformi(int location, int value1, int value2, int value3);

void setUniformi(const char* name, int value1, int value2, int value3, int value4);

void setUniformi(int location, int value1, int value2, int value3, int value4);

void setUniformf(const char* name, float value);

void setUniformf(int location, float value);

void setUniformf(const char* name, float value1, float value2);

void setUniformf(int location, float value1, float value2);

void setUniformf(const char* name, float value1, float value2, int value3);

void setUniformf(int location, float value1, float value2, int value3);

void setUniformf(const char* name, float value1, float value2, int value3, int value4);

void setUniformf(int location, float value1, float value2, int value3, int value4);

void setUniform1fv(const char* name, int length, const float values[]);

void setUniform1fv(int location, int count, float const values[]);

void setUniform2fv(const char* name, int count, const float values[]);

void setUniform2fv(int location, int count, const float values[]);

void setUniform3fv(const char* name, int count, const float values[]);

void setUniform3fv(int location, int count, const float values[]);

void setUniform4fv(const char* name, int count, const float values[]);

void setUniform4fv(int location, int count, const float values[]);

void setUniformMatrix2fv(const char* name, int count, bool transpose, const float *value);

void setUniformMatrix2fv(int location, int count, bool transpose, const float *value);

void setUniformMatrix3fv(const char* name, int count, bool transpose, const float *value);

void setUniformMatrix3fv(int location, int count, bool transpose, const float *value);

void setUniformMatrix4fv(const char* name, int count, bool transpose, const float *value);

void setUniformMatrix4fv(int location, int count, bool transpose, const float *value);

int fetchAttributeLocation(const char* name);

int fetchUniformLocation(const char* name);

private:

void compileShaders(const char* vertexCode, const char* fragmentCode);

GLuint loadShader(GLenum type, const char* source);

GLuint linkProgram(GLuint vertexShader, GLuint fragmentShader);

};

} // namespace glcore

? shader_program.cpp

#include

#include "glcore/gl_inspector.h"

#include "glcore/shader_program.h"

#define TAG "ShaderProgram"

namespace glcore

{

ShaderProgram::ShaderProgram(const char* vertexCode, const char* fragmentCode)

{

compileShaders(vertexCode, fragmentCode);

}

ShaderProgram::~ShaderProgram()

{

if (m_program)

{

GL_CALL(glUseProgram(0));

GL_CALL(glDeleteProgram(m_program));

m_program = 0;

}

m_attributes.clear();

m_uniforms.clear();

}

void ShaderProgram::bind()

{

GL_CALL(glUseProgram(m_program));

}

void ShaderProgram::enableVertexAttribArray(const char* name)

{

int location = fetchAttributeLocation(name);

enableVertexAttribArray(location);

}

void ShaderProgram::enableVertexAttribArray(int location)

{

GL_CALL(glEnableVertexAttribArray(location));

}

void ShaderProgram::setVertexAttribPointer(const char *name, int size, int type, bool normalize, int stride, int offset)

{

int location = fetchAttributeLocation(name);

setVertexAttribPointer(location, size, type, normalize, stride, offset);

}

void ShaderProgram::setVertexAttribPointer(int location, int size, int type, bool normalize, int stride, int offset)

{

GL_CALL(glVertexAttribPointer(location, size, type, normalize, stride, (void*) offset));

}

void ShaderProgram::disableVertexAttribArray(const char* name)

{

int location = fetchAttributeLocation(name);

disableVertexAttribArray(location);

}

void ShaderProgram::disableVertexAttribArray(int location)

{

GL_CALL(glDisableVertexAttribArray(location));

}

void ShaderProgram::setUniformi(const char* name, int value)

{

int location = fetchUniformLocation(name);

GL_CALL(glUniform1i(location, value));

}

void ShaderProgram::setUniformi(int location, int value)

{

GL_CALL(glUniform1i(location, value));

}

void ShaderProgram::setUniformi(const char* name, int value1, int value2)

{

int location = fetchUniformLocation(name);

GL_CALL(glUniform2i(location, value1, value2));

}

void ShaderProgram::setUniformi(int location, int value1, int value2)

{

GL_CALL(glUniform2i(location, value1, value2));

}

void ShaderProgram::setUniformi(const char* name, int value1, int value2, int value3)

{

int location = fetchUniformLocation(name);

GL_CALL(glUniform3i(location, value1, value2, value3));

}

void ShaderProgram::setUniformi(int location, int value1, int value2, int value3)

{

GL_CALL(glUniform3i(location, value1, value2, value3));

}

void ShaderProgram::setUniformi(const char* name, int value1, int value2, int value3, int value4)

{

int location = fetchUniformLocation(name);

GL_CALL(glUniform4i(location, value1, value2, value3, value4));

}

void ShaderProgram::setUniformi(int location, int value1, int value2, int value3, int value4)

{

GL_CALL(glUniform4i(location, value1, value2, value3, value4));

}

void ShaderProgram::setUniformf(const char* name, float value)

{

int location = fetchUniformLocation(name);

GL_CALL(glUniform1f(location, value));

}

void ShaderProgram::setUniformf(int location, float value)

{

GL_CALL(glUniform1f(location, value));

}

void ShaderProgram::setUniformf(const char* name, float value1, float value2)

{

int location = fetchUniformLocation(name);

GL_CALL(glUniform2f(location, value1, value2));

}

void ShaderProgram::setUniformf(int location, float value1, float value2)

{

GL_CALL(glUniform2f(location, value1, value2));

}

void ShaderProgram::setUniformf(const char* name, float value1, float value2, int value3)

{

int location = fetchUniformLocation(name);

GL_CALL(glUniform3f(location, value1, value2, value3));

}

void ShaderProgram::setUniformf(int location, float value1, float value2, int value3)

{

GL_CALL(glUniform3f(location, value1, value2, value3));

}

void ShaderProgram::setUniformf(const char* name, float value1, float value2, int value3, int value4)

{

int location = fetchUniformLocation(name);

GL_CALL(glUniform4f(location, value1, value2, value3, value4));

}

void ShaderProgram::setUniformf(int location, float value1, float value2, int value3, int value4)

{

GL_CALL(glUniform4f(location, value1, value2, value3, value4));

}

void ShaderProgram::setUniform1fv(const char* name, int count, const float values[])

{

int location = fetchUniformLocation(name);

GL_CALL(glUniform1fv(location, count, values));

}

void ShaderProgram::setUniform1fv(int location, int count, const float values[])

{

GL_CALL(glUniform1fv(location, count, values));

}

void ShaderProgram::setUniform2fv(const char* name, int count, const float values[])

{

int location = fetchUniformLocation(name);

GL_CALL(glUniform2fv(location, count / 2, values));

}

void ShaderProgram::setUniform2fv(int location, int count, const float values[])

{

GL_CALL(glUniform2fv(location, count / 2, values));

}

void ShaderProgram::setUniform3fv(const char* name, int count, const float values[])

{

int location = fetchUniformLocation(name);

GL_CALL(glUniform3fv(location, count / 3, values));

}

void ShaderProgram::setUniform3fv(int location, int count, const float values[])

{

GL_CALL(glUniform3fv(location, count / 3, values));

}

void ShaderProgram::setUniform4fv(const char* name, int count, const float values[])

{

int location = fetchUniformLocation(name);

GL_CALL(glUniform4fv(location, count / 4, values));

}

void ShaderProgram::setUniform4fv(int location, int count, const float values[])

{

GL_CALL(glUniform4fv(location, count / 4, values));

}

void ShaderProgram::setUniformMatrix2fv(const char* name, int count, bool transpose, const float *value)

{

int location = fetchUniformLocation(name);

GL_CALL(glUniformMatrix2fv(location, count, transpose, value));

}

void ShaderProgram::setUniformMatrix2fv(int location, int count, bool transpose, const float *value)

{

GL_CALL(glUniformMatrix2fv(location, count, transpose, value));

}

void ShaderProgram::setUniformMatrix3fv(const char* name, int count, bool transpose, const float *value)

{

int location = fetchUniformLocation(name);

GL_CALL(glUniformMatrix3fv(location, count, transpose, value));

}

void ShaderProgram::setUniformMatrix3fv(int location, int count, bool transpose, const float *value)

{

GL_CALL(glUniformMatrix3fv(location, count, transpose, value));

}

void ShaderProgram::setUniformMatrix4fv(const char* name, int count, bool transpose, const float *value)

{

int location = fetchUniformLocation(name);

GL_CALL(glUniformMatrix4fv(location, count, transpose, value));

}

void ShaderProgram::setUniformMatrix4fv(int location, int count, bool transpose, const float *value)

{

GL_CALL(glUniformMatrix4fv(location, count, transpose, value));

}

int ShaderProgram::fetchAttributeLocation(const char* name)

{

int location;

auto it = m_attributes.find(name);

if (it == m_attributes.end())

{

GL_CALL(location = glGetAttribLocation(m_program, name));

if (location == -1) {

printf("%s: no attribute: %s\n", TAG, name);

//GLInspector::printProgramInfoLog(m_program, "fetchAttributeLocation");

return -1;

}

m_attributes[name] = location;

}

else

{

location = it->second;

}

return location;

}

int ShaderProgram::fetchUniformLocation(const char* name)

{

int location;

auto it = m_uniforms.find(name);

if (it == m_uniforms.end())

{

GL_CALL(location = glGetUniformLocation(m_program, name));

if (location == -1) {

printf("%s: no uniform: %s\n", TAG, name);

//GLInspector::printProgramInfoLog(m_program, "fetchUniformLocation");

return -1;

}

m_uniforms[name] = location;

}

else

{

location = it->second;

}

return location;

}

void ShaderProgram::compileShaders(const char* vertexCode, const char* fragmentCode)

{

GLuint vertexShader = loadShader(GL_VERTEX_SHADER, vertexCode);

GLuint fragmentShader = loadShader(GL_FRAGMENT_SHADER, fragmentCode);

m_program = linkProgram(vertexShader, fragmentShader);

}

GLuint ShaderProgram::loadShader(GLenum type, const char* source)

{

GL_CALL(GLuint shader = glCreateShader(type));

GL_CALL(glShaderSource(shader, 1, &source, nullptr));

GL_CALL(glCompileShader(shader));

GLint success;

glGetShaderiv(shader, GL_COMPILE_STATUS, &success);

if (!success) {

GLInspector::printShaderInfoLog(shader, "loadShader");

return 0;

}

return shader;

}

GLuint ShaderProgram::linkProgram(GLuint vertexShader, GLuint fragmentShader)

{

GL_CALL(GLuint program = glCreateProgram());

GL_CALL(glAttachShader(program, vertexShader));

GL_CALL(glAttachShader(program, fragmentShader));

GL_CALL(glLinkProgram(program));

GLint success;

glGetProgramiv(program, GL_LINK_STATUS, &success);

if (!success) {

GLInspector::printProgramInfoLog(m_program, "linkProgram");

}

GL_CALL(glDeleteShader(vertexShader));

GL_CALL(glDeleteShader(fragmentShader));

return program;

}

} // namespace glcore

2.8 VBO

? VBO 是 Vertex Buffer Object 的简称,即顶点缓冲对象,作用是缓存顶点数据到显存中,避免频繁调用 glVertexAttribPointer 传输顶点数据,减少 CPU 到 GPU 的数据传输,提高渲染效率。

? 顶点属性主要有位置、颜色、纹理坐标、法线、切线、副切线等,每个属性又有属性标识、维数、是否已标准化、数据类型、偏移、别名、纹理单元等。

? 由于 VBO 中有多个属性数据,每个属性有多个字段,笔者除了封装 VertexBufferObject 类,还封装了 VertexAttributes 和 VertexAttribute 两个类。VertexAttribute 是属性描述类,VertexAttributes 是属性描述集合。

? vertex_buffer_object.h