FAQ/Combining 3D Passes

From VFXPedia

< FAQ
Jump to: navigation, search

Contents

Multi-Pass Compositing

Introduction

V-Ray rendering engine is capable of separating the rendered image into so-called elements. Every element represents a certain property of the rendered scene (similar to Red, Green and Blue channels of a color image), so rendering into elements gives you a possibility to tweak those properties separately in compositing for quicker integration into the background plate or to achieve a certain artistic result without re-rendering of the 3D scene over again.
The elements are often called passes. This term comes from the times when each separated property of the rendering had to be rendered in a separate step (pass). Nowadays the render engines allow you to output multiple elements in one pass. However, some additional elements may still require dedicated passes.
Sometimes, the elements are also called layers or channels, but in fact, most elements contain three RGB-channels each, with different values in the color channels. The term layer, in its turn, is more suitable for a part of the final image which comes in front of another part (like in Photoshop).

Meaning of Render Elements

The render elements of V-Ray are:

Diffuse - the flat colors and textures of the surfaces, not affected by any lighting, shadowing, reflections etc. (same as Color in the p_MegaTK shader for Maya).
Raw Light - the amount of direct light the surfaces get.
Lighting - the colors of the surfaces with direct lighting but without reflections/refractions. The Lighting element actually presents the same image as the Diffuse multiplied by the Raw Light.
Raw Shadow - the areas of the Raw Lighting pass which were blocked from light by scene objects. This pass doesn't make it into the basic pass compositing formula, but if for some stupid reason you want to adjust the intensity of the cast shadows or remove the cast shadows completely, you can add it to the Raw Lighting pass.
Shadow - similar to the Raw Shadow pass, but should be added to Lighting to adjust the cast shadows intensity.
Matte Shadow - essentially, a mask for cast shadows which allows you to color-correct the shadows for artistic purposes or to introduce shadows from 3D objects to the real world objects of the background plate.
Raw Global Illumination - the amount of indirect (reflected) light the surfaces get.
Global Illumination - the color of the surfaces with indirect lighting (the reflected diffuse light), but without direct light and reflections. The Global Illumination element actually presents the same image as the Diffuse multiplied by the Raw Global Illumination. It's name is usually contracted to GI.
Ambient Occlusion - shadows from uniform ambient light illuminating the objects from outside of the scene. This is an optional pass generated separately. Useful to soften the lighting of the scene, increase/introduce contact shadows or to exaggerate the surface details. It's name is usually contracted to AO. Many people call it just Occlusion, but strictly speaking, it's not quite precise. This pass represents occlusion of the ambient light, not direct light, not speculars.
Raw Reflection - pure reflection without consideration of surface reflectivity and Fresnel effect. The image you see in Reflection isn't separated into elements, i.e. in Reflection you always see the complete Beauty.
Reflection Filter - combination of surface reflectivity and Fresnel which represents the resulting amount of reflection.
Reflection - complete reflection, affected by the surface reflectivity and Fresnel effect. The Reflection element is the same as the Raw Reflection multiplied by the Reflection Filter.
Specular - reflection of the 3D scene lights in the surfaces.
Raw Refraction - the details we see through transparent objects. Raw Refraction isn't affected by surface transparency and Fresnel effect. The image you see in Refraction isn't separated into elements, i.e. in Refraction you always see the complete Beauty.
Refraction Filter - combination of surface transparency and Fresnel effect which represents how much we see through the surfaces.
Refraction - the details we see through transparent objects. This is the resulting refraction which involves transparency and Fresnel effect. In other words, the Refraction element presents the same image as the Raw Refraction multiplied by the Refraction Filter.
Sub-Surface Scattering - represents the light which is diffused inside the object volume and then gets somehow out. It is a very important element for such materials as paraffin, human flesh, fine objects from stone or plastic, stems of plants etc.
Self-Illumination - the color the surfaces get by emitting light themselves.

Short Multi-Pass Formula

  • To combine the render elements into the final image (Beauty), you can use such a simplified basic formula:
Beauty = Lighting + Global Illumination + Reflection + Specular + Refraction + Sub-Surface Scattering + Self Illumination

This math can be implemented in Eyeon Fusion, using the Merge or ChannelBoolean tools.
To add two images together using Merge, feed them to the Background and Foreground inputs of the Merge node, set Alpha Gain = 0.
ChannelBoolean gives you the same result. Feed the images into the corresponding ChannelBoolean inputs, set Operation = Add, To Alpha = Do Nothing.

File:MultipassCompositing_Mrg_Controls_Add.gif‎

Although the ChannelBoolean approach seams to be more straight forward, the Merge is quicker to setup and tweak.,!-- Besides, the Merge is more optimized on the code level, so it works faster, especially if you connect masks to it. (this was official information but tests show that it's not relevant to the current Fusion versions anymore, the BOL is in fact faster)-->


File:MultipassCompositing Short Flow_v01A.gif

You may struggle to find the Sub-Surface Scattering (SSS) on the example screenshot. Sorry for that, but there were no parts made of paraffin or human skin in this car, so it didn't make sense to render and merge a black image. Anyway, SSS is just added like any of other listed elements.

Full Multi-Pass Formula

  • The full formula which involves raw passes is:
Beauty = Diffuse * (Raw Light + Raw Global Illumination) + Raw Reflection * Reflection Filter + Specular + Raw Refraction * Refraction Filter + Sub-Surface Scattering + Self Illumination

As you probably remember from school, in any kinds of mathematical expressions with more than two numbers, multiplication is performed before addition. The operations inside the brackets also have the priority.
This formula is rebuilt in Eyeon Fusion's Flow, using also the Merge or ChannelBoolean tools.
For multiplication, set the Apply Mode = Multiply in Merge, or Operation = Multiply and To Alpha = Do Nothing in the ChannelBoolean tool.

File:MultipassCompositing_Mrg_Controls_Multiply.gif‎

Although the Merge is generally easier to use, it applies the foreground Alpha as a mask while performing the multiplication, which results in lighter semi-transparent areas and edges, where the Alpha is not 100% white. In other words, the Merge mixes the result of the multiplication with the original background where the foreground Alpha is <1.
The easiest way to avoid that is to divide the Alpha by itself in the branch connected to the Foreground input of the multiplying Merge.


File:MultipassCompositing_Full_Flow_v02.gif

Involving raw passes gives you even more control over various surface properties by separating the image to the elements further.

You can also combine different parts of the full and short formulas, depending on which elements you would like to tune more.
For instance, if you need to change the colors of certain objects in the reflections significantly, it makes sense to work with the Raw Reflection to have the direct access to the initial objects' color and preserve the reflectivity, multiplying the adjusted Raw Reflection by the Reflection Filter. But if you want to merely adjust the amount of reflection, the Reflect element is enough.
Similarly, if you feel like playing with relighting or texturing in comp, you definitely need the RawLight and RawGI, multiplied by the Diffuse. But if you don't, then better reduce your electricity consumption by using the Lighting and Global Illumination elements.

There is, however, one issue with using raw passes instead of the result passes.
If any two adjacent areas change their color and brightness correlation to the opposite after multiplication of the raw pass by its filter, you will see a nasty edge between them.
This happens most of the time due to anti-aliasing which mixes colors of intersecting or overlapping surfaces on the image, but gets even worse in case of motion blur and defocus.
For instance, if you have a cube with one face painted much lighter than another, but the main light illuminates the darker face much stronger, you will experience this for sure.

To fix this situation, you can actually derive a kind of raw passes from the result passes using the filters.
For instance, if Raw Light * Diffuse ≈ Lighting, then Lighting / Diffuse ≈ Raw Light
Quite logical, isn't it? And in this case, the Raw Light you derive in compositing, gives you an ideal Lighting element, being multiplied with the Diffuse back.
I borrowed this technique from www.vfxvault.com.

Ambient Occlusion Hustle

As it follows from the name of the pass, Ambient Occlusion (AO) displays occlusion of the ambient light by scene objects. This means that if you multiply it with a solid color plate representing Ambient Light Color, you'll get an additional super-soft ambient light source surrounding the whole scene. Obviously, to use it correctly, you need to add this extra light pass directly to your Raw Light:

Result = Diffuse * (Raw Light + Ambient Light Color * Ambient Occlusion + Raw Global Illumination) + Raw Reflection * Reflection Filter + Specular + Raw Refraction * Refraction Filter + Sub-Surface Scattering + Self Illumination

Instead of multiplication with a solid color plate, you can also just tint the color of the Ambient Occlusion pass itself with a ColorCorrector set to the Fast mode. Mathematically it's the same operation.
If the Ambient Occlusion was rendered as an option and wasn't taken into consideration while setting the light in 3D scene, you may need to darken the basic lighting to avoid overexposure.

File:MultipassCompositing_Full_AO_Flow_v02.gif


You can also apply AO in other ways, which are technically less correct, but may let you quickly achieve a desired artistic result like exaggerating shape details or adding locally a soft contact shadow without affecting main lighting balance too much:

  • As a multiplier for the Raw Light
Result = Diffuse * (Raw Light * Ambient Occlusion + Raw Global Illumination) + Raw Reflection * Reflection Filter + Specular + Raw Refraction * Refraction Filter + Self Illumination

You should be careful applying AO in such way, because you create a strange situation when the areas which have already caught some direct light are being suddenly darkened by some mysterious black forces. On the final image it looks like dirt (which makes you recall where this algorithm originally came from).

  • As a multiplier for the Raw Global Illumination
Result = Diffuse * (Raw Light + Raw Global Illumination * Ambient Occlusion) + Raw Reflection * Reflection Filter + Specular + Raw Refraction * Refraction Filter + Self Illumination

Multiplying AO with GI is also not quite correct, especially in case of self-illuminating objects, because again, as it follows from the name, it's supposed to occlude ambient light, not the light diffused or emitted by particular surfaces.

Mysterious Pre-Divide / Post-Multiply

By default, all the passes are rendered as multiplied by alpha, against black background. This means that semi-transparent areas as well as blurred or anti-aliased edges become darker, being mixed with black.
This is the ideal situation to apply the rendered Beauty directly over an arbitrary background plate in compositing. But when you combine the Beauty from separate elements, it brings a trouble.
The problem is that such properties as the amount of the direct light, reflection color, self illumination, velocity or depth, which have actually nothing to do with the background color, display darker values in semi-transparent areas. For multi-pass compositing it means dark edges and darker transparency on the result image, compared to the Beauty.

To avoid this, you need to divide all the render elements by Alpha to restore the original color before putting them together. The tools which can perform this operation are the ChannelBoolean and dedicated AlphaDivide.
After the elements are put together, you can re-introduce the mixing with the background by multiplying the result by Alpha. Use the MatteControl or ChannelBoolean or dedicated AlphaMultiply tool for this.

File:MultipassCompositing_Short_Divide_Flow_v01A.gif

File:MultipassCompositing_Full_Divide_Flow_v02.gif

The Beauty Is Re-Assembled, What's Next?

Once you've assembled the basic setup, recreating the Beauty, you can start adding various color correction tools and masks to separate parts of it.

The corrections for particular elements are added between the AlphaDivide node and the node which combines the element with the others. Of course, you can apply corrections to groups of elements too.

All the formulas can be changed depending on particular needs and available passes.

However

Sometimes, if the Beauty pass isn't too ugly, it's easier to use some render elements just as masks for color correction of the Beauty or as source of glowing, applied on top of the final image. But in practice, it's almost never the case ;)



Gregory Chalenko
www.compositing.ru

Personal tools