Normal Texture UV Channel Re-Bake Instructions
Aug 15, 2019 3:41:01 GMT
Warhammer_81 and cptrex like this
Post by BadHorse on Aug 15, 2019 3:41:01 GMT
I use the software XNormal (free) to do the re-baking for me. It's more often used to create a normal texture for a lower density mesh from a higher density mesh, but it can also be used with identical meshes with different UV channel data.
The pre-requisite for using this method is that you'll need two different version of your mesh -- one with each of the two UV channels of interest in the primary channel slot. There's a number of softwares that can be used for this purpose. I personally use UltimateUnwrap3D (not free), which has the option to swap UV channels, but I've also done it with 3DS Max. Keep in mind that you'll have to do this process separately for every sub-mesh in your model.
Once that's done, I usually take a few extra steps using an image editor (I use GIMP). The first thing that you'll notice about your new normal texture is that the program fills all the unused areas with alpha transparency, which depending on how the map transfers can be a problem. If for example your new normal texture is 2048x2048, I'll create a 2048x2048 image in GIMP and fill it with RGB 128,128,255, and then drag the new texture onto it and merge layers.
One other thing I've noticed is that the process of re-baking the normals tends to cause some loss of contrast. If you look at the histogram for the R or G color channels, there's almost no data below 35 or above 220. To correct this, in GIMP I usually use the "Levels" function under the "Colors" menu, and set the minimum and maximum input levels to 35 and 220 respectively.
Despite all the steps above, once you get used to it the whole process goes fairly quickly. I can usually knock one out in ~10 minutes. There can be other complications however. Sometimes, meshes with large flat areas (e.g. shuttle tydirium's wings) aren't intended to have any normal data because it would be a waste of image space. So they put the flat areas of the mesh outside of the UV 1x1 square, and set up their shaders so that the normal textures don't tile. Unfortunately, XNormal doesn't have an option to disable texture tiling, so you'll end up with weird normals all over your flat areas. To combat this, you'll have to manually remove the non-textured areas from your meshes before re-baking.
I'm sure I've forgotten some details, but this should be enough to get you started.
The pre-requisite for using this method is that you'll need two different version of your mesh -- one with each of the two UV channels of interest in the primary channel slot. There's a number of softwares that can be used for this purpose. I personally use UltimateUnwrap3D (not free), which has the option to swap UV channels, but I've also done it with 3DS Max. Keep in mind that you'll have to do this process separately for every sub-mesh in your model.
- When you start XNormal, on the right-hand side, you'll see the various workflow tabs. Starting at the top on the "High Definition Meshes" tab ...
- Add the mesh with the original normal mapping in UV Channel 1 as your high definition mesh.
- Select the original normal texture as the "Base texture to bake"
- Check the box for "Base texture is a tangent-space normal map"
- Under the "Smooth Normals" drop-down, select "Harden normals"
- Leave all other options unchanged and move on to the "Low Definition Meshes" tab ...
- Add the identical mesh with the diffuse color mapped to UV Channel 1 as your low definition mesh.
- Change the "Maximum frontal ray distance" and "Maximum rear ray distance" to 0.001. The meshes are identical, so you don't need a wide tolerance, and leaving these at their default values will detect other nearby geometry and create artifacts.
If you're trying to bake from a decal onto the main mesh, you may need to open up the tolerance so that the decal mesh is detected, but that's more of an advanced method. - Again, under the "Smooth Normals" drop-down, select "Harden normals"
- Un-check the box that says "Highpoly normals override tangent space" <-- this is important. If you miss this step, some of your new normal texture will bump in the wrong direction.
- Leave all other options unchanged and move on to the "Baking Options" tab ...
- Pick a filename (duh)
- When selecting a size, it's usually a good idea to go at least one size larger than the original normal texture, to make up for any potential loss of image quality. For example, if the original normal texture was 1024x1024, you should probably use 2048x2048.
- "Closest hit if ray fails" --> uncheck
- "Discard back face hits" --> check
- "Edge Padding" --> Use 0 pixels. There's a reason for this that I'll expound upon later as I add more detail to the tutorial.
- Under "Maps to render" select "Bake Base Texture" and leave all others unchecked.
- At the lower right-hand corner, select "Generate Maps" and watch it go.
Once that's done, I usually take a few extra steps using an image editor (I use GIMP). The first thing that you'll notice about your new normal texture is that the program fills all the unused areas with alpha transparency, which depending on how the map transfers can be a problem. If for example your new normal texture is 2048x2048, I'll create a 2048x2048 image in GIMP and fill it with RGB 128,128,255, and then drag the new texture onto it and merge layers.
One other thing I've noticed is that the process of re-baking the normals tends to cause some loss of contrast. If you look at the histogram for the R or G color channels, there's almost no data below 35 or above 220. To correct this, in GIMP I usually use the "Levels" function under the "Colors" menu, and set the minimum and maximum input levels to 35 and 220 respectively.
Despite all the steps above, once you get used to it the whole process goes fairly quickly. I can usually knock one out in ~10 minutes. There can be other complications however. Sometimes, meshes with large flat areas (e.g. shuttle tydirium's wings) aren't intended to have any normal data because it would be a waste of image space. So they put the flat areas of the mesh outside of the UV 1x1 square, and set up their shaders so that the normal textures don't tile. Unfortunately, XNormal doesn't have an option to disable texture tiling, so you'll end up with weird normals all over your flat areas. To combat this, you'll have to manually remove the non-textured areas from your meshes before re-baking.
I'm sure I've forgotten some details, but this should be enough to get you started.