6.1 Re-arrangement of Textures
A problem of current implementation of PBR is that black seam occurs when the roughness of object is high. The “mip-map” arrangement in pre-filtered environment map in interim report uses a pyramid-like scaling. Given a 256*256 resolution of a single face in LOD 0, the size of a single face in LOD 7 will be 4*4, in which UV components may fall outside of the region due to float precision limit in OpenGL texture2D function. In addition, compressed texture sizes also cause inaccurate color, which is more apparent when size of object gets larger. Therefore, a re-arrangement of the “mip-maps” is necessary.
Noticing that there are blank spaces unused in left upper corner and right lower corner, why don’t we put maps for higher roughness there? For LOD levels beyond 2, the sizes are maintained as same as that of level 2. This method solves the aforementioned problems, while it utilises image canvas more evenly and efficiently. The modification of shader is also simple – for LOD greater than 2, an array is declared to store their relative coordinates; in texture fetch, a condition is used to branch the UV coordinates to the contents of the array. Below is a sample picture of our new pre-filtered environment map.
Another adjustment of game textures is combining several single monochrome textures into one texture. The maps for metalness, roughness, ambient occlusion and transmittance are assigned to R, G, B and A channel respectively. This is helpful for conserving texture slots since our system has a mobile version using OpenGL ES rasterizer that limit at most 8 game textures. The total number of textures for a single material will be limited to 6 (diffuse map, mixed map (metalness, roughness, ambient occlusion, ambient occlusion and transmittance), normal map, irradiance map, pre-integrated BRDF map, pre-filtered environment map), regardless of additional environment maps required for environment switch.
6.2 SSS Based on Normal Map
A problem for current local subsurface scattering is that it depends on direct light source, which is a problem if there is only image-based lighting. However, indirect lighting often has more evenly distributed irradiance intensity, which means the effect of local scale scattering will not be affected too much by the object orientation. Therefore, I approximate the case by assuming amount of environment light coming equally from all directions. Under this assumption, it is equivalent to scatter the normal map, as explained in the interim report of a similar case. The scattering is only determined by the skin’s diffusion profile. Also, scattering the normal map turns real-time calculation into pre-processing calculation, which can help to boost performance.
The calculation uses convolution of 6 Gaussian functions as introduced in interim report. However, since the texture space is distorted in geometry, we need to “stretch” it back to the magnitude in the original 3D mesh. As introduced in the texture space diffusion method (introduced in interim report), baking a “stretch map” aids the problem efficiently. In my implementation, this is done by calling the “ddx/ddy” function which returns local curvature in GLSL, using the fragment color the store the stretch magnitude in X and Y direction, and capturing the rendering result which has been projected to texture space. This was then used as an input of the normal map scattering program as a LUT.
This method has a good rendering result. The grainy and dry look of skin without scattering is replaced by a pinkish and smooth appearance, while the details are also preserved. The lack of local subsurface scattering in case of strong contrast in environment light can be partly compensated by the translucency effect.
6.3 Environment Switch
Up to now, our PBR only supports a single environment or a single room if using box projected cube environment mapping. But the fact is our game system has complex scenes, game characters need to walk around rooms in a building, for example. Therefore, we must use different environment maps for different places and even different parts of a single large place. It is obviously impossible to store all textures in the texture slots of an object. Actually, we only need two sets of environment maps, each of which is for one of the two adjacent scenes. When entering a place from another place, color interpolation can be used between the two sets of environment maps, which only needs to use the position of the adjacent border and the scale and position of the shaded object. Assuming we are walking from environment map 1 to environment map 2. We can derive the formula below:
.
where the positions and lengths are all projected to 1D – the line perpendicular to the environment border.
We can always keep 4 slots for environment maps – 2 for older one, 2 for newer one (each environment has an irradiance map and a pre-filtered environment map), keeping the total number of textures 8. Upon the event of touching a new environment, we can put the corresponding environment maps into slots for newer one. After fully entering that environment, we can move the maps to the slots for older one. The assignment of texture slots can be done by calling appropriate APIs in the game engine. Thus, we can guarantee a smooth environment switch for PBR.
6.4 Conclusion
Physically-based rendering is the main methodology to achieve photorealistic rendering quality. In common polygon-rasterization-based game engines with limited computational resources, this methodology is implemented by a collaboration of precomputation (irradiance map, pre-filtered environment map) and real-time shading using HDR image-based lighting. Meanwhile, many specified methods are researched and implemented to tackle particular problems including subsurface skin scattering and translucency. If you pack all these into a PBR Tool with user-friendly api and detailed documentation, other people in your mobile or pc game projects like artists can implement realistic PBR effects without extra efforts.
All the research and implementation indicate that realistic rendering is a complex task. There is not a “put things right once and for all” formula for it. The nature, governed by physics laws, has infinite structures that lead to phenomenon. However, the computational power is always finite. It is impossible to simulate all phenomena by one formula. Instead, particular problem should be tackled particularly and approximation is always used. The goal should focus on giving customers a satisfying result, providing the level of realism that meets their need. However, experimental methods should always be encouraged as computer graphics should develop infinitely until perfection.
List of Figures
Figure 16: New arrangement of Pre-filtered Environment map. Source: My original picture generated by OpenCV.