GPU Ambient Occlusion in Maya 2014

One of the new features in mental ray 3.11 is the ability to render Ambient Occlusion (AO) on the GPU while your CPU renders the other framebuffers.

Shown some time ago at Siggraph in 2011, mental ray makes use of the CUDA and Optix technologies from nVidia. Using Standalone you also have the option to generate only the AO pass and skip other framebuffers. This is useful if you have a few machines with really nice graphics cards that allow you to render those frames quickly and send your other frames to the renderfarm that might not have adequate GPUs.

We don’t typically render AO passes for everything like was done years ago for many reasons. It’s a multiplicative pass in compositing which can cause issues and higher quality indirect lighting like Final Gather or the Environment Light will create this effect naturally by default. However, there are many times there is an artistic look or requirement to use AO.

(If you prefer different instructions, you can find another way to do this on the nVidia ARC forum: Maya 2014 and mental ray AO on the GPU )

How do I set this up in 2014?

Maya 2014 ships with the required Optix Libraries to use GPU AO. Keep in mind you should be using an nVidia graphics processor (mental ray is, after all, an nVidia product.) I have a caveat here: I don’t have examples from my machine because I have an ancient Quadro that is not supported. And yes, I’m aware of the irony that I don’t have a new nVidia card. But this should work fine for most users using a modern Quadro or GTX card with the latest drivers. If  you do NOT have the latest drivers, it’s likely you will crash. Current Quadro drivers are 320 as of this writing.

Special note: for OSX and Linux users you need to install the CUDA driver separately

There are three main steps:

1. Move the necessary files to a better location (only has to be done once for the machine)
2. Setup the String Options (once per scene using .mel)
3. Setup the framebuffer to write the result to (once per scene/camera)

Moving the files

You can find the libraries here on Windows x64: C:\Program Files\Autodesk\mentalrayForMaya2014\bin

The files you want are:

• cudart64_42_9.dll
• gpu_ao_plugin.dll
• optix.1.dll
• optixu.1.dll

Usually mental ray searches this folder for the libraries. However I have not found that to be successful so there is another way that is more consistent for me.

1. Copy the files to a new directory you create in your user Maya folder. For example, create a folder called: mental ray  The new path would be something like this: C:\Users\UserName\Documents\maya\2014-x64\mentalray

2. Copy the above .dll files to that directory.

3. Go up one directory and find the Maya.env file. Open it with a text editor and put this text in it and save:

MI_LIBRARY_PATH = $MAYA_APP_DIR/2014-x64/mentalray
PATH = $MI_LIBRARY_PATH;$PATH

Ok you’re done with that part and that should work from here on out regardless of scene. Now, open Maya and create a very simple scene.

Creating the String Options

Now you need the correct string options to generate and trigger the GPU AO pass.

From the documentation:

Added a plugin for fast computation of ambient occlusion pass if NVIDIA GPU is present. If enabled, ambient occlusion pass is computed on the GPU while the main and other framebuffers rendered on the CPU simultaneously. As CPU is not involved significantly into AO pass rendering, the AO pass is rendered almost for free.

GPU AO rendering can be enabled with the string option "ambient occlusion gpu" on, or with the command line options "-ao gpu".

If enabled, the framebuffer name used to store AO data has to be specified with the "ambient occlusion framebuffer"<string> string option.

The "ambient occlusion max distance" <float> string option and –ao_max_distance <f> command line options can be used to specify the maximal length of rays used to compute AO. A negative value describes a factor relative to the scene extent. The default value is -1.

The "ambient occlusion falloff" <float> string option and -ao_falloff <float> command line options can be used to specify the exponent of AO fading to the max distance. Default value is 1 (linear falloff).

The "ambient occlusion falloff min distance" <float> string option and -ao_falloff_min_distance <float> command line options can be used to specify the distance after which the falloff takes place. The default value is 0.

The "ambient occlusion gpu passes" <int> sting option and -ao_gpu_passes <int> command line option specify the number of times the accumulating AO framebuffer is rendered before averaging. It this option is not set, it is derived from mental ray max samples (per pixel) rate.

The "ambient occlusion rays" <int> string option and -ao_rays <int> specify the number of rays per gpu pass. The default is computed based on mental ray max samples (per pixel) rate. The actual value used is rounded down to a Fibonacci number (1,2,3,5,8,13,…).

The total number of ambient occlusion rays used per pixel is approximately the product of "ambient occlusion gpu passes" times "ambient occlusion rays".

GPU AO respects visible, shadow and transparency flags in order to produce intuitive AO results.

If GPU is not present, a fallback functionality is provided. The AO framebuffer is rendered according to mental ray’s raster sampling pattern and is not pixel identical with the GPU one.

Below is an example of some options I added to a scene:

AO on the GPU String Options

Here’s a quick(er) .mel way to add some strings. You can edit this to create your own settings:

select miDefaultOptions;
int $idx = `getAttr -size miDefaultOptions.stringOptions`;
setAttr -type "string" miDefaultOptions.stringOptions[$idx].name "ambient occlusion framebuffer";
setAttr -type "string" miDefaultOptions.stringOptions[$idx].value "ao_buffer";
setAttr -type "string" miDefaultOptions.stringOptions[$idx].type "string";
$idx = `getAttr -size miDefaultOptions.stringOptions`;
setAttr -type "string" miDefaultOptions.stringOptions[$idx].name "ambient occlusion gpu";
setAttr -type "string" miDefaultOptions.stringOptions[$idx].value "on";
setAttr -type "string" miDefaultOptions.stringOptions[$idx].type "boolean";
$idx = `getAttr -size miDefaultOptions.stringOptions`;
setAttr -type "string" miDefaultOptions.stringOptions[$idx].name "ambient occlusion max distance";
setAttr -type "string" miDefaultOptions.stringOptions[$idx].value "5.0";
setAttr -type "string" miDefaultOptions.stringOptions[$idx].type "scalar";
$idx = `getAttr -size miDefaultOptions.stringOptions`;
setAttr -type "string" miDefaultOptions.stringOptions[$idx].name "ambient occlusion falloff";
setAttr -type "string" miDefaultOptions.stringOptions[$idx].value "1";
setAttr -type "string" miDefaultOptions.stringOptions[$idx].type "integer";

You can use the above .mel template to add more to the settings, for example, you can add the occlusion rays setting like this:

select miDefaultOptions;
int $idx = `getAttr -size miDefaultOptions.stringOptions`;
setAttr -type "string" miDefaultOptions.stringOptions[$idx].name "ambient occlusion rays";
setAttr -type "string" miDefaultOptions.stringOptions[$idx].value "8";
setAttr -type "string" miDefaultOptions.stringOptions[$idx].type "integer";

Now what’s left is to add the framebuffer you just specified with “ambient occlusion framebuffer” to the user framebuffers.

You can do this in a similar way to using framebuffers with the MILA shaders.

Create the Framebuffer for writing the output.

1. Select the miDefaultOptions node

select miDefaultOptions;

2. Create a framebuffer:

AEmrUserBuffersAppend miDefaultOptions.frameBufferList;

The above command creates a user framebuffer. Be sure and match the name with your chosen string option value, in this case: ao_buffer

The created AO pass

3. In the camera, go to the Attribute Editor and then mental ray > Output Shaders > Create

Camera Output Pass added

Now render your scene. Things to look for:

1. That it finds and loads the plug-ins:

LIB 0.12 13 MB info : loaded library "C:/Users/UserName/Documents/maya/2014-x64/mentalray\gpu_ao_plugin.dll".
LIB 0.12 13 MB info : library "C:/Users/UserName/Documents/maya/2014-x64/mentalray\gpu_ao_plugin.dll": loaded mr_gpu plugin "mr_gpu_ao"

2. The GPU begins to process the scene and renders:

GPU 0.3 13 MB info : rcgpu: regions:2 indices:2286 vertices:481 = sharing:4
GPU 0.3 13 MB info : gpuao: time to create gpu ao data: 0.01 seconds
GPU 0.3 13 MB info : gpuao: transfer memory consumption: 0.0204716 MiBytes
GPU 0.3 13 MB info : gpuao: time to initialize on gpu: 0.47 seconds.
GPU 0.3 13 MB info : gpuao thread: prepare render entry
GPU 0.3 13 MB info : gpuao thread: [62.2981 ms] initialized optix
GPU 0.3 13 MB info : gpuao thread: Optix devices: [Dev0 'Quadro FX 1700' (compute 1.1) mem 512MB] Dev0:307/512MB
GPU 0.3 13 MB info : gpuao thread: Restricting to one CUDA device for Lbvh.
GPU 0.3 13 MB info : gpuao thread: [99.3315 ms] start load scene
GPU 0.3 13 MB info : gpuao thread: [300.254 ms] start load geometry data
GPU 0.3 13 MB info : gpuao thread: finish load scene
GPU 0.3 13 MB info : gpuao thread: Gpu buffer total size:0.0167866 MiB 762 tris
GPU 0.3 13 MB info : gpuao thread: 960x540 AO:8 AA:36 (288/3) ao distance:5 falloff:1 AS:Lbvh/Bvh leaf:8
GPU 0.3 13 MB info : gpuao thread: [0.462541 s total]

Then the GPU will output progress while the beauty frame renders. . .in simple scenes it might output after the beauty render ends.

GPU  0.3     16 MB info : gpuao:  Waiting for gpu...

You can then check the images/tmp folder of your project directory for the resulting image.

You might also notice visual stuttering of your computer while it processes the frame. Your GPU resources are being used to render the Ambient Occlusion and therefore your Youtube experience will suffer while rendering.

Notes:

• The amount of RAM used for the scene on the GPU needs to be similar for the CPU. That’s usually the case (typically your GPU has less)
• On OSX and Linux the environment variable(s) DYLD_LIBRARY_PATH (Mac  OSX), or LD_LIBRARY_PATH (Linux) should point to the folder of the OptiX libraries before starting Maya
• The AO pass derives its settings from global sampling settings unless you specify with the correct string options mentioned above. However, there are some cases in OEM software where this isn’t respected and may render with less quality than you specified/desire
• Each AO render might be slightly different each time you render. This has since been fixed in mental ray but hasn’t made it to OEM libraries yet