Env light background alpha through glass / transmission

   4051   12   3
User Avatar
Member
39 posts
Joined: Feb. 2017
Offline
Is it possible to hide a Dome Light from the camera when seen through a transmissive material, such as glass, so that all transmission rays that terminate at the Dome light evaluate to an alpha value of 0?

I have a clunky little test scene here in which I am viewing a cone through a thick piece of glass, rendering in Karma with MaterialX:


The scene is lit by a single Dome Light with an environment texture. I can hide the Dome Light from direct camera rays by unchecking "Render Light Geometry" on the Dome Light, but the area seen through the glass is still opaque:


If I switch to Arnold, with the same scene setup, I can enable "Transmit AOVs" on the Standard Surface glass shader + set the Dome Light Transmission Contribution to 0 and get the intuitive result seen here:


Is it possible to recreate this result in Karma? I have been bumping my head against this for a year now and I can't tell if the engine isn't capable of it or if I'm not smart enough to understand the documentation...

Attachments:
pcoip_client_Py0QZBe1EE.png (1.3 MB)
pcoip_client_g8dcdhNupg.png (190.2 KB)
pcoip_client_NEGDitoAKA.png (158.1 KB)

User Avatar
Member
7741 posts
Joined: Sept. 2011
Online
Why would you want that though? it's wrong and if you were to comp it like that, the dome light passing through the glass would be plussed onto the background.

jedmitchell
If I switch to Arnold, with the same scene setup, I can enable "Transmit AOVs" on the Standard Surface glass shader + set the Dome Light Transmission Contribution to 0 and get the intuitive result seen here:

That result isn't intuitive to me. Why does the transparent part have fresnel gradations all over it? Where is the subject alpha in the internally reflected transmission? This result isn't useful for anything.
User Avatar
Member
39 posts
Joined: Feb. 2017
Offline
I disagree that this is undesirable -- it is a hack but an extremely practical one. We use this for comping cars onto plates regularly, where we can see through multiple windows to a background. In fast turnarounds we may not get a high quality plate to put back there in the render itself -- especially if we're turning over renders for the client to comp.

Removing the transmission contribution of the light in Arnold is only making it invisible to rays that pass directly from a transmission hit to the light itself, so it is still illuminating the interior areas but not being represented in the RGB or A channels:


How would you accomplish the goal of matting this Karma render, with the accompanying alpha, over a plate?



Edit: oh, I missed part of your comment -- regarding the fresnel response in the Arnold render. What is in the alpha channel there is from the reflect contribution, and there is no fresnel effect on the cone seen through the glass because thin-walled is enabled. That feature does not function in Karma CPU so the results are not identical in that regard. That's my fault for not matching the settings.
Edited by jedmitchell - Feb. 4, 2023 00:40:49

Attachments:
pcoip_client_KGqf3Gkw5j.png (752.7 KB)
pcoip_client_ZquUyHCIyN.png (671.9 KB)
pcoip_client_5i8JBBFwxp.png (194.3 KB)

User Avatar
Member
8532 posts
Joined: July 2007
Online
it's indeed very wrong to expect refractions to be just see through unless IOR=1 so such renders are usually not very useful

what would be more useful would be the ability to propagate AOVs through reflections/refractions
so then you could get masks of different objects or probably even cryptomatte with a proper contribution seen through reflections/refractions

however since Mantra doesn't support it I wouldn't necessarily expect Karma CPU to have it until it catches up with Mantra and I don't expect XPU to have that feature until proper way of doing AOVs is figured out, which may be hindered by very incomplete MaterialX
Tomas Slancik
FX Supervisor
Method Studios, NY
User Avatar
Member
8532 posts
Joined: July 2007
Online
jedmitchell
How would you accomplish the goal of matting this Karma render, with the accompanying alpha, over a plate?

render your beauty with the dome light hidden from camera and refractions (Contributions set to : -refract)
and then render pure refraction pass using UV map instead of plate or env map (whatever you want to replace)

then in comp use such refracted UV map to map any image you want and add beauty on top you will get much closer representation of properly refracted new plate in comp
of course to be done properly one would have to also consider refraction contribution per pixel etc, but it will be definitely closer than pure see through shader
Edited by tamte - Feb. 4, 2023 00:55:31
Tomas Slancik
FX Supervisor
Method Studios, NY
User Avatar
Member
39 posts
Joined: Feb. 2017
Offline
tamte
render your beauty with the dome light hidden from camera and refractions (Contributions set to : -refract)
and then render pure refraction pass using UV map instead of plate or env map (whatever you want to replace)

Thanks, I will give that a try. And it sounds like the answer to my original question is simply "no", this is not a feature offered by the engine.

To be clear about the source of the question: I'm not suggesting that the Arnold setup I'm comparing it with is an accurate simulation of light. It's just a very practical solution to get mid-to-background elements rendered quickly for shots that will be solved mostly in comp.

Generally we like Karma quite a bit, but the absence of "hacks" like these has made it difficult for our small team to incorporate it on many jobs. It's fine if that's not the direction SideFX is taking development, but it's not always clear from the Karma docs what is a feature, bug, or simply not yet implemented.
User Avatar
Member
39 posts
Joined: Feb. 2017
Offline
Experimenting with this a little further, I feel like I am missing a step somewhere.

I have set up two Karma outputs, the first produces a beauty pass with the HDRI set to be hidden from camera & refraction rays:

Edit: I had the wrong image here when I initially posted -- it has been updated.

The second output disables the Dome Light & places a background plane behind the scene with a ST map projected from the camera to a surface shader. All but the refract component is disabled on the rendersettings prim, and this is the result:


In Nuke I'm using the same image HDRI in the background to make comparison easier, and there are two obvious problems:


First: compared to the in-engine render of the Dome Light through the glass, the refraction is significantly deflecting the angle of the light. This is probably caused by what appears to be a drop in values for areas seen through the glass. I'm not sure why this is, as the glass shader does not have absorption or roughness.

Second: because black is a valid coordinate on an st map, we get whatever color happens to be in the bottom-left corner of the source image mapped to any dark areas in the refraction pass render, which leads to the plus operation adding value to areas on the torus that should not be affected by refraction.

Any thoughts on this? I have only employed this trick of using an st map to distort an environment into single-layer things like raindrops or water on a lens, etc. so I feel like I'm missing something doing it in a scene that has multiple layers of objects.

Thanks for your help, it is an educational exercise at this point, as much as anything else...
Edited by jedmitchell - Feb. 4, 2023 19:25:17

Attachments:
pcoip_client_aNw2Ua0mCx.png (591.3 KB)
pcoip_client_H2sPTArten.png (414.1 KB)
ApplicationFrameHost_xqODzlBtGQ.png (353.0 KB)

User Avatar
Member
8532 posts
Joined: July 2007
Online
this is where the mask AOV propagation through refractions would have helped as then you could have both mask and uv refracted in the single pass so that you'd have contribution

now you mau need to do another render of just refracted white dome to get the contribution that will act like Alpha for stmap, so that you can unpremultiply it to cleaner values for mapping
and then scale back down before adding the rest
Tomas Slancik
FX Supervisor
Method Studios, NY
User Avatar
Member
7741 posts
Joined: Sept. 2011
Online
tamte
now you mau need to do another render of just refracted white dome to get the contribution that will act like Alpha for stmap, so that you can unpremultiply it to cleaner values for mapping
and then scale back down before adding the rest

if you put solid color in the blue channel of the st map, you can skip having to do a separate render.
User Avatar
Member
39 posts
Joined: Feb. 2017
Offline
jsmack
if you put solid color in the blue channel of the st map, you can skip having to do a separate render.

Aha, I was wondering if there was a way to use the blue channel to do masking somehow. That worked for me, and the resulting comp looked similar to the in-engine render with the Dome Light as a background:


I noticed that when switching the comp background to something with more pronounced HDR values that I was getting a *lot* of noise though, even when I cranked the samples way up on the refraction pass render:


It looked like there was something up with the refraction pass itself -- bumping up the viewer exposure I could see there was a lot of noisy information in the R & G channels on the torus that (as far as I understand) should not have been included in the render with the components set only to "refract":


Am I misunderstanding the effect of setting the export components like that? It feels like it's somehow still including a reflect component.

As a test I made my own "refract" AOV with a simple LPE and a custom tag on the st map material: C.*T'stmap', based on the idea that I want to record *only* the rays exiting a transmission event and striking the background.

Using my custom LPE AOV for the refraction pass I get a much cleaner result that doesn't affect the torus at all, it only distorts the background plate as expected:


I'm not that knowledgeable about LPEs or the Karma AOV workflow though -- are there conditions I'm not considering here? Was there a better way to set up this refract only AOV?

Thanks!

Attachments:
pcoip_client_YTNuVTfWeV.png (391.0 KB)
pcoip_client_Acccc3RI2R.png (989.6 KB)
pcoip_client_f1N45zH2jv.png (162.3 KB)
pcoip_client_dFHlUaqCi2.png (21.7 KB)
pcoip_client_7LOwLY9h1F.png (491.1 KB)

User Avatar
Member
8532 posts
Joined: July 2007
Online
jedmitchell
Was there a better way to set up this refract only AOV?
custom LPE is the way to go if you want specific paths, because in just refraction AOV (Karma's Glossy Transmission) you will get lots of stuff refracted, but not everything makes sense as uv map doesn't work well with average values from diffuse or rough surfaces

you can have 1 for refraction and 1 for reflection if you want to replace those also
Tomas Slancik
FX Supervisor
Method Studios, NY
User Avatar
Member
39 posts
Joined: Feb. 2017
Offline
tamte
custom LPE is the way to go if you want specific paths, because in just refraction AOV (Karma's Glossy Transmission) you will get lots of stuff refracted, but not everything makes sense as uv map doesn't work well with average values from diffuse or rough surfaces

Gotcha. I went back and enabled the default Glossy Transmission AOV in Karma and compared it to my own LPE-based approach and in this scenario they appear to be identical, which I guess makes sense since there is nothing else to refract in this scene.

Taking the difficulty up a notch, I'm trying to work this now with multiple transmissive planes -- like a car with an interior. This is a Karma reference render with all Dome Light contributions enabled:


It looks like the -refract setting on the Dome Light for Karma results in preventing illumination from entering the interior of the glass box even for the purpose of other glossy scattering events. This behavior is different from the equivalent settings in Arnold, which only removes the final transmission-->light event (Karma on the left, Arnold on the right):


So now it is necessary to create all that illumination in comp -- and the results are problematic in terms of both noise and the artifacts from internally reflected rays having nothing meaningful to sample from the st map:


This is a place where I feel the Arnold setup is just more practical. I understand it's not optically correct, but having those options is not preventing you from doing it the right way on hero elements, and having to work out a comp solution for this kind of transmission scenario isn't an appealing prospect for day-to-day renders of non-hero elements...

Maybe there's something else I'm missing though? Like is there an alternative to setting the Dome Light contributions to -refract to get it out of the beauty pass?
Edited by jedmitchell - Feb. 5, 2023 14:28:18

Attachments:
ApplicationFrameHost_ZnKLKndecy.png (985.2 KB)
ApplicationFrameHost_xyXmV9uoHb.png (1.2 MB)
pcoip_client_K6SDWFiZCa.png (678.5 KB)

User Avatar
Member
39 posts
Joined: Feb. 2017
Offline
Playing around some more I found a solution that almost works -- instead of disabling the refracted light in the beauty pass, I've left the background in the beauty and then done a subtractive comp operation with a custom LPE that only contains Camera-->Transmission-->Light events with a tag like CT+'env'(the 'env' tag is on the Dome Light):


I still need the mask generated by the blue channel of the refract pass, but I can modify it to correctly represent the transmission-only path and then subtract the Dome Light background from the beauty. Basically it works:


The results are still very noisy though (this is after the minus operation):


I was able to get the noise down to a reasonable level but only by massively oversampling:


Denoising helps in some areas but introduces more artifacts elsewhere, as usual -- and generally we can't denoise sequences anyway. Are there any other tricks I'm not thinking of here? I'm sure I could optimize the sampling settings to some extent but the jump in render time to get a clean comp was so great that it makes this approach basically unworkable for us.

Attachments:
mstsc_15J4d6i6kl.png (273.1 KB)
mstsc_Roy1srkgZw.png (715.0 KB)
mstsc_oKwYXQoKdx.png (960.3 KB)
mstsc_8utsvUJisR.png (642.2 KB)

  • Quick Links