之前学习的 iOS OpenGL ES 编程的资料和知识比较零散,现在希望通过观看 Ray Wenderlich - Beginning OpenGL for iOS 的同时,把目前掌握的 OpenGL ES 知识点整理一下,第四篇文章会讲 Texture 这些基础内容。

OpenGL

简介

如下的左图是在 3D 建模软件中制作的模型,如下的右图就是将模型导出的时候,分离出来的纹理,当然导出文件中还包含 OBJ 模型;纹理映射就是将网格模型中每一个顶点坐标和纹理坐标结合起来,类似于一张 2D 包装纸裹到 3D 物体表面,顶点坐标之间的坐标位置就在纹理坐标之间做 interpolation,就像颜色值一样。

Metal By Example Texture Mapping

加载纹理

加载纹理的流程

示例代码

  • Get Core Graphics image reference. Since we’re going to use Core Graphics to write out the raw pixel data, we need a reference to the image! This is quite simple – UIImage has a CGImageRef property we can use.

  • Create Core Graphics bitmap context. The next step is to create a Core Graphics bitmap context, which is a fancy way of saying a buffer in memory to store the raw pixel data.

  • Draw the image into the context. We can do this with a simple Core Graphics function call – and then the buffer will contain raw pixel data!

  • Send the pixel data to OpenGL. To do this, we need to create an OpenGL texture object and get its unique ID (called it’s “name”), and then we use a function call to pass the pixel data to OpenGL.

—- Quote from OpenGL ES 2.0 for iPhone Tutorial Part 2: Textures

- (GLuint)setupTexture:(NSString *)fileName {
    // 从 Bundle 中加载纹理图片
    CGImageRef spriteImage = [UIImage imageNamed:fileName].CGImage;
    if (!spriteImage) {
        NSLog(@"Failed to load image %@", fileName);
        exit(1);
    }
    
    size_t width = CGImageGetWidth(spriteImage);
    size_t height = CGImageGetHeight(spriteImage);
    
    // 每个像素由 4 个像素组成,代表 r, g, b, a
    GLubyte *spriteData = (GLubyte *) calloc(width * height * 4, sizeof(GLubyte));
    CGContextRef spriteContext = CGBitmapContextCreate(spriteData, width, height, 8, width * 4, CGImageGetColorSpace(spriteImage), kCGImageAlphaPremultipliedLast);
    // 在 bitmap context 中绘制纹理图片
    CGContextDrawImage(spriteContext, CGRectMake(0, 0, width, height), spriteImage);
    CGContextRelease(spriteContext);
    
    GLuint texName;
    // OpenGL 中创建纹理
    glGenTextures(1, &texName);
    // 绑定纹理,GL_TEXTURE_2D 当前指向 texName 的纹理存储区域
    glBindTexture(GL_TEXTURE_2D, texName);
    // 给纹理设置参数
    glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MIN_FILTER, GL_NEAREST);
    // 传递像素数据给纹理
    glTexImage2D(GL_TEXTURE_2D, 0, GL_RGBA, width, height, 0, GL_RGBA, GL_UNSIGNED_BYTE, spriteData);
    
    free(spriteData);
    
    return texName;
}

Texture Filtering

其中下面这段代码是在设置 texture filtering:

glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MIN_FILTER, GL_NEAREST);

We’ll also need to specify what should happen when the texture is expanded or reduced in size, using texture filtering. When we draw a texture onto the rendering surface, the texture’s texels may not map exactly onto the fragments generated by OpenGL. There are two cases: “minification” and magnification. Minification happens when we try to cram several texels onto the same fragment, and magnification happens when we spread one texel across many fragments. We can configure OpenGL to use a texture filter for each case.

—- Quote from OpenGL ES 2 for Android

Mipmap

glGenerateMipmap(GL_TEXTURE_2D);

Opengl Mipmap

While bilinear filtering works well for magnification, it doesn’t work as well for minification beyond a certain size. The more we reduce the size of a texture on the rendering surface, the more texels will get crammed onto each fragment. Since OpenGL’s bilinear filtering will only use four texels for each fragment, we still lose a lot of detail. This can cause noise and shimmering artifacts with moving objects as different texels get selected with each frame.

To combat these artifacts, we can use mipmapping, a technique that generates an optimized set of textures at different sizes. When generating the set of textures, OpenGL can use all of the texels to generate each level, ensuring that all of the texels will also be used when filtering the texture. At render time, OpenGL will select the most appropriate level for each fragment based on the number of texels per fragment.

—- Quote from OpenGL ES 2 for Android

Opengl Texture Parameters

GLKTextureLoader

GLKit 提供了更方便的方式来加载纹理:

func loadTexture(_ filename: String) {
    let path = Bundle.main.path(forResource: filename, ofType: nil)!
    let option = [ GLKTextureLoaderOriginBottomLeft: true]
    do {
        let info = try GLKTextureLoader.texture(withContentsOfFile: path, options: option as [String : NSNumber]?)
        self.texture = info.name
    } catch {
        
    }
}

可以替代之前如下代码:

glGenTextures(1, &texName);
glBindTexture(GL_TEXTURE_2D, texName);
glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MIN_FILTER, GL_NEAREST);
glTexImage2D(GL_TEXTURE_2D, 0, GL_RGBA, width, height, 0, GL_RGBA, GL_UNSIGNED_BYTE, spriteData);

纹理坐标

OpenGL 的 texture coordinates 如下,原点在左下脚:

OpenGL Texture Coordinates

iOS 的 UIKit 坐标和 Core Graphics 坐标分别如下:

iOS Coordinates

当我们通过 UIImage 从磁盘加载纹理图像到内存,需要注意图像的方向,可以通过 CGContext Draw 进行相应的方向调整,就可以保证纹理数据方向的正确性;GLTextureLoader 中设置了 GLKTextureLoaderOriginBottomLeft 属性,也可以保证纹理数据方向的正确性。

之前绘制立方体的时候,相接的两个平面可以共享两个 Vertex 的位置坐标,但是纹理坐标却不能共享,因为在两个平面上要保证正确的绘制顺序,相接处的纹理坐标并不相同,所以需要每个平面单独定义两个三角形:

OpenGL Cube UV

纹理坐标是 normalized coordinates,大小范围在 [0, 1]:

class Cube : Model {
    let vertexList : [Vertex] = [
        
        // Front
        Vertex( 1, -1, 1,  1, 0, 0, 1,  1, 0), // 0
        Vertex( 1,  1, 1,  0, 1, 0, 1,  1, 1), // 1
        Vertex(-1,  1, 1,  0, 0, 1, 1,  0, 1), // 2
        Vertex(-1, -1, 1,  0, 0, 0, 1,  0, 0), // 3
        
        // Back
        Vertex(-1, -1, -1, 0, 0, 1, 1,  1, 0), // 4
        Vertex(-1,  1, -1, 0, 1, 0, 1,  1, 1), // 5
        Vertex( 1,  1, -1, 1, 0, 0, 1,  0, 1), // 6
        Vertex( 1, -1, -1, 0, 0, 0, 1,  0, 0), // 7
        
        // Left
        Vertex(-1, -1,  1, 1, 0, 0, 1,  1, 0), // 8
        Vertex(-1,  1,  1, 0, 1, 0, 1,  1, 1), // 9
        Vertex(-1,  1, -1, 0, 0, 1, 1,  0, 1), // 10
        Vertex(-1, -1, -1, 0, 0, 0, 1,  0, 0), // 11
        
        // Right
        Vertex( 1, -1, -1, 1, 0, 0, 1,  1, 0), // 12
        Vertex( 1,  1, -1, 0, 1, 0, 1,  1, 1), // 13
        Vertex( 1,  1,  1, 0, 0, 1, 1,  0, 1), // 14
        Vertex( 1, -1,  1, 0, 0, 0, 1,  0, 0), // 15
        
        // Top
        Vertex( 1,  1,  1, 1, 0, 0, 1,  1, 0), // 16
        Vertex( 1,  1, -1, 0, 1, 0, 1,  1, 1), // 17
        Vertex(-1,  1, -1, 0, 0, 1, 1,  0, 1), // 18
        Vertex(-1,  1,  1, 0, 0, 0, 1,  0, 0), // 19
        
        // Bottom
        Vertex( 1, -1, -1, 1, 0, 0, 1,  1, 0), // 20
        Vertex( 1, -1,  1, 0, 1, 0, 1,  1, 1), // 21
        Vertex(-1, -1,  1, 0, 0, 1, 1,  0, 1), // 22
        Vertex(-1, -1, -1, 0, 0, 0, 1,  0, 0), // 23
        
    ]
    
    let indexList : [GLubyte] = [
        
        // Front
        0, 1, 2,
        2, 3, 0,
        
        // Back
        4, 5, 6,
        6, 7, 4,
        
        // Left
        8, 9, 10,
        10, 11, 8,
        
        // Right
        12, 13, 14,
        14, 15, 12,
        
        // Top
        16, 17, 18,
        18, 19, 16,
        
        // Bottom
        20, 21, 22,
        22, 23, 20
    ]
}

修改 Shader

vertex shader 传入和输出纹理坐标:

attribute vec2 a_TexCoord;

varying lowp vec2 frag_TexCoord;

void main(void) {
    frag_TexCoord = a_TexCoord;
}

fragment shader 中 u_Texture 就是我们传入的纹理,texture2D 函数根据纹理坐标 frag_TexCoord 到纹理 u_Texture 中获取对应的颜色值:

uniform sampler2D u_Texture;

varying lowp vec4 frag_Color;
varying lowp vec2 frag_TexCoord;

void main(void) {
    gl_FragColor = frag_Color * texture2D(u_Texture, frag_TexCoord);
}

绑定数据

纹理坐标:

glBindAttribLocation(self.programHandle, VertexAttributes.texCoord.rawValue, "a_TexCoord")

glEnableVertexAttribArray(VertexAttributes.texCoord.rawValue)
glVertexAttribPointer(
    VertexAttributes.texCoord.rawValue,
    2,
    GLenum(GL_FLOAT),
    GLboolean(GL_FALSE),
    GLsizei(MemoryLayout<Vertex>.size), BUFFER_OFFSET((3+4) * MemoryLayout<GLfloat>.size)) // x, y, z | r, g, b, a | u, v :: offset is (3+4)*sizeof(GLfloat)

纹理:

self.textureUniform = glGetUniformLocation(self.programHandle, "u_Texture")

glActiveTexture(GLenum(GL_TEXTURE1)) // 激活位置编号 position slot
glBindTexture(GLenum(GL_TEXTURE_2D), self.texture) // 绑定 GL_TEXTURE_2D 当前指向的纹理数据到位置编号
glUniform1i(self.textureUniform, 1) // 设置位置编号到指定标识

开启 Blending

才能使纹理中透明生效:

glEnable(GLenum(GL_BLEND))
glBlendFunc(GLenum(GL_SRC_ALPHA), GLenum(GL_ONE_MINUS_SRC_ALPHA))

解释下示例代码

立方体的每一面贴相同的 Texture

OpenGL Texture

示例代码

上面主要讲解就是这里面的代码。

立方体的每一面贴 Texture 的不同部分

OpenGL Texture Dice

示例代码

主要就是 texture coordinates 的值正确设置:

class Cube : Model {
    let vertexList : [Vertex] = [
        
        // Front
        Vertex( 1, -1, 1,  1, 0, 0, 1,  0.25, 0), // 0
        Vertex( 1,  1, 1,  0, 1, 0, 1,  0.25, 0.25), // 1
        Vertex(-1,  1, 1,  0, 0, 1, 1,  0, 0.25), // 2
        Vertex(-1, -1, 1,  0, 0, 0, 1,  0, 0), // 3
        
        // Back
        Vertex(-1, -1, -1, 0, 0, 1, 1,  0.5, 0), // 4
        Vertex(-1,  1, -1, 0, 1, 0, 1,  0.5, 0.25), // 5
        Vertex( 1,  1, -1, 1, 0, 0, 1,  0.25, 0.25), // 6
        Vertex( 1, -1, -1, 0, 0, 0, 1,  0.25, 0), // 7
        
        // Left
        Vertex(-1, -1,  1, 1, 0, 0, 1,  0.75, 0), // 8
        Vertex(-1,  1,  1, 0, 1, 0, 1,  0.75, 0.25), // 9
        Vertex(-1,  1, -1, 0, 0, 1, 1,  0.5, 0.25), // 10
        Vertex(-1, -1, -1, 0, 0, 0, 1,  0.5, 0), // 11
        
        // Right
        Vertex( 1, -1, -1, 1, 0, 0, 1,  1, 0), // 12
        Vertex( 1,  1, -1, 0, 1, 0, 1,  1, 0.25), // 13
        Vertex( 1,  1,  1, 0, 0, 1, 1,  0.75, 0.25), // 14
        Vertex( 1, -1,  1, 0, 0, 0, 1,  0.75, 0), // 15
        
        // Top
        Vertex( 1,  1,  1, 1, 0, 0, 1,  0.25, 0.25), // 16
        Vertex( 1,  1, -1, 0, 1, 0, 1,  0.25, 0.5), // 17
        Vertex(-1,  1, -1, 0, 0, 1, 1,  0, 0.5), // 18
        Vertex(-1,  1,  1, 0, 0, 0, 1,  0, 0.25), // 19
        
        // Bottom
        Vertex( 1, -1, -1, 1, 0, 0, 1,  0.5, 0.25), // 20
        Vertex( 1, -1,  1, 0, 1, 0, 1,  0.5, 0.5), // 21
        Vertex(-1, -1,  1, 0, 0, 1, 1,  0.25, 0.5), // 22
        Vertex(-1, -1, -1, 0, 0, 0, 1,  0.25, 0.25), // 23
        
    ]
}

Texture with Mask

OpenGL Texture Mask

示例代码

上图中前面是 u_Texture 纹理,中间是 u_Mask 纹理,注意非白色部分是透明色,最后是合成显示的图像,原理就是通过 u_Mask 的 alpha 值做颜色过滤:

uniform sampler2D u_Texture;
uniform sampler2D u_Mask;

varying lowp vec4 frag_Color;
varying lowp vec2 frag_TexCoord;

void main(void) {
    lowp vec4 texColor = texture2D(u_Texture, frag_TexCoord);
    lowp vec4 maskColor = texture2D(u_Mask, frag_TexCoord);
    gl_FragColor = vec4(texColor.r, texColor.g, texColor.b, maskColor.a * texColor.a);
}