Using Mayatomr: Normal Displacement

Normal displacement is displacement relative to the surface normal.  Heres a quick tutorial for using normal displacement with Mayatomr.

Reference mesh, divisions 5

Start with

• Base mesh (created by down-res’ing a high-res reference mesh)
• Normal displacement map (exported with values that match the base mesh to the reference mesh)

Step-by-step

Open Maya; import base mesh; apply material shader.  Typically your base mesh will have a low poly count:

Base mesh, divisions 1

From the material’s shadingEngine, attach a displacement shader.

shadingEngine > Shading Group Attributes > Dispacement mat. [map button]

Select the “File” node from prompt and load the correct displacement map. Note: Maya automatically creates a displacementShader between the shading group and the file node. Because normal displacement only requires a scalar (greyscale) value, the file node connects to the displacementShader via it’s alpha channel. “Alpha Is Luminance” has also been enabled so that the alpha is the average of the RGB channels. Set the file node’s Filter Type to Off.

file > File Attributes > Filter Type = Off

This is important because you do not want Maya to try and interpolated displacement samples. If you render now, you’ll get an interesting looking result.

cool, but not useful

The reason the image looks like this is because the displacement map is only 8-bit.  Mental ray expects negative values to be inward displacement, positive values to be outward displacement, and zero to be neutral displacement.  Because 8-bit images only map to positive values (0.0 to 1.0), the sculpting application has exported a displacement map where 0.5 corresponds to neutral.  Offsetting the alpha by that amount (subtracting) will adjust for difference.

file > Color Balance > Alpha Offset = -0.5

You will most likely want to leave Alpha Offset at 0.0 for 32-bit/floating-point displacement maps (depending on how it was exported).

Better

The mesh still does not match.  Here’s why:

1. The base mesh still has hard normals causing the displacement to open up seams as it pushes the surface outward.
2. As soon as you attach a displacement shader, Mayatomr “helps” users by applying a length-distance-angle approximation to the polygon surface.  This is not ideal.

We can fix both these problems by turning the mesh into a Catmull-Clark subdivision surface (ccmesh).  Using the approximation editor, apply a parametric subdivision to the surface.  By default, applying this approximation triggers Mayatomr to export the object as a ccmesh.

Window > Rendering Editors > mental ray > Approximation Editor
mental ray Approximation Editor > Subdivisions (Polygons and Subd. Surfaces) > Create
mentalraySubdivApprox > Subdivision Surface Quality > Approx Method = Parametric

N Subdivisions = 2

The seams have closed up, however the mesh has lost all detail.  The shadows look particularly bad because of the low triangle count.   Increasing the number of subdivisions will bring back detail.  To match the reference mesh, set N Subdivisions = N(reference) – N(base), in this case 5 – 1 = 4.

N Subdivisions = 4

Lower N Subdivisions result in a loss of detail.

N Subdivisions 3

 Larger N Subdivisions may cause visual artifacts as you exceeded the resolution of the displacement map (displacement map pixels become visible).

N Subdivisions 6

Unfortunately, normal displacement is inherently limited in the geometry it can describe. For modern workflows (and the my next blog post!) use vector displacement.