Karma OCIO file rules in XPU and differences between h19 and

   2252   26   7
User Avatar
Member
7785 posts
Joined: 9月 2011
Offline
pawel-fin
Personally, I think that the metadata should not be used when the file rules are in place. With both active it will be very hard to predict the outcome and debug problems. I expect the rule that says "all EXRs are ACEScg" to take over everything else.
Additionally, controlling metadata that is coming from a variety of DCC apps is not easy.
If the file rules take over metadata like you described then I think this is the correct behaviour.

Ideally, metadata would be the main way to control color, however lacking a standard and that the required metadata would only be specific to the ocio config in use makes this not feasible. Metadata is the only way to correctly read an image with multiple image planes that are all in different color spaces. A single file rule or file name token cannot handle such a case. For mplay for example, metadata does take precedence because it must display multipart exrs that can be rendered to different colorspaces with husk. The main problem with how metadata is currently handled stems from the open image IO library creating a default value for the colorspace metadata even when no value exists in the file. This makes it impossible for the image consumer to know if the metadata is bogus or not. Maybe additional metadata could be used, such as a tag added by karma when rendering multi-part images to custom color spaces, something like "karma:ocio:usemeta" to distinguish between the fake "oiio:Linear" tag that's added to every file by oiio and a real "oiio:Linear (Rec709)" tag added by husk/karma.
User Avatar
Member
433 posts
Joined: 4月 2018
Offline
For me Karma XPU ignores all rules in the config in all situations. CPU works fine.
Subscribe to my Patreon for the best CG tips, tricks and tutorials! https://patreon.com/bhgc [patreon.com]

Twitter: https://twitter.com/brianhanke [twitter.com]
Behance: https://www.behance.net/brianhanke/projects [www.behance.net]
User Avatar
Member
7785 posts
Joined: 9月 2011
Offline
BrianHanke
For me Karma XPU ignores all rules in the config in all situations. CPU works fine.

weird, it works fine for me except for certain cases where an image that is Linear (rec709) is correctly loaded as Linear (rec709) even though the file rule says to load it as ACEScg. All my tests with ACEScc and sRGB worked as expected.
User Avatar
Member
253 posts
Joined: 7月 2013
Offline
Maybe it's a good idea to leverage PDG to render out a big set of all sorts of combinations. Assemble them into a mosaic so it's easy to compare outputs from different Houdini versions and flag any errors and such.

In the old days there was a thing called the ACID test which showed any issues in how CSS edge cases where handled by different browsers, was quite interesting. I can envision something similar for OCIO.
Edited by Jonathan de Blok - 2024年2月29日 02:48:09
More code, less clicks.
User Avatar
Member
433 posts
Joined: 4月 2018
Offline
That would be a good thing to have in the docs! This seems to be a very complicated topic and nobody is exactly sure how it works...
Subscribe to my Patreon for the best CG tips, tricks and tutorials! https://patreon.com/bhgc [patreon.com]

Twitter: https://twitter.com/brianhanke [twitter.com]
Behance: https://www.behance.net/brianhanke/projects [www.behance.net]
User Avatar
スタッフ
2592 posts
Joined: 7月 2005
Offline
@pawel-fin, thank you for submitting the test files.

After going through the examples, there were a few things that I thought would be useful to share with the community. Understandings about how OCIO is handled in H20.0.

I hope this helps clarify some of the behaviours in H20.0.

USD/MaterialX

USD stores the color space information for MaterialX nodes as metadata on the parameter. The Hydra interface (used by Karma/husk) does not currently pass this metadata through to the renderer. So, the color space metadata doesn't make it through from VOPs to Karma.
If a renderer loads the USD directly (without Hydra), it's able to access this metadata, so behaviour might be different. Later versions of USD will have this fixed.

OpenEXR Color Space

To determine the source color space, Karma (and VEX) use the following rules:
a) If the color space is explicitly specified, use it (this doesn't work with USD/MaterialX, see above)
b) Match using the OCIO file rules
c) If there are no file rules, use metadata in the image
d) If there's no metadata, fall back to a default

If we get to the last stage, the default for 8-bit images is to assume the color space is sRGB. For OpenEXR, Karma assumes they are Linear Rec.709. This is according to the OpenEXR spec for files missing the chromaticities attribute (https://openexr.com/en/latest/TechnicalIntroduction.html)
Note that OpenEXR suggests the chromaticities metadata should be used to determine the color space. However, neither OpenImageIO, OpenColorIO or the Houdini core libraries are able to determine an OCIO color space from the chromaticities.
Houdini will use the oiio:ColorSpacemetadata. The example .exr files you included both had the oiio:ColorSpaceset to Linear. Since there are multiple linear color spaces, we've found making the assumption that "linear" maps to "scene_linear" can often lead to bad results, so we don't count "linear" as being a meaningful value for oiio:ColorSpace.

H19.5

H19.5 assumed scene_linear was Linear Rec.709 in all cases. It ignored the OCIO file. For the most part, H19.5 also ignored the OCIO config file entirely when interpreting color spaces. For 8 bit files, there was a built-in conversion from sRGB to Linear Rec.709. OpenEXR files were read as "raw", so no color space conversion happened.
As I pointed out, in your example, H19.5 rendered identical images, regardless of the OCIO config file. That's because H19.5 ignored OCIO.

H20.0

As mentioned above, H20.0 obeys the OCIO scene_linear ccolor space and uses the OCIO rules to perform color space computations.
None of the textures in your example were optimized (tiled and mip-mapped). In H20, Karma XPU automatically creates an optimized texture for rendering (see the HOUDINI_TEXTURE_DISK_CACHE variable). There was a bug with this process when scene_linear was not set to Linear Rec.709. This was fixed in H20.0.633 (March 1, 2024).
It's highly recommended that you use optimized textures since they consume less memory, have better filtering and are in general more efficient. Look at HOUDINI_TEXTURE_DISK_CACHE for options to automate this process.

Your Test Cases

For those readers without access, there were three textures provided:
- textureA = JPG image
- textureB = OpenEXR in ACEScg color space
- textureC = OpenEXR in Linear Rec.709 color space
As mentioned above, none of the textures had color space metadata and were not tiled/mip-mapped.

Default OCIO Config
Scene Linear = Linear Rec.709
textureA: source space srgb_tx -> target space lin_rec_709
textureB: lin_rec_709 -> lin_rec_709
textureC: lin_rec_709 -> lin_rec_709

This matches H19.5 exactly. textureB is interpreted as being in Linear Rec.709 color space, so comes through unmodified.

RedShift OCIO Config
Scene Linear = ACES - ACEScg
textureA: Input - Generic - sRGB - Texture -> ACES - ACEScg
textureB: Linear Rec.709 -> ACES - ACEScg
textureC: Linear Rec.709 -> ACES - ACEScg

The reason textureB and textureC are assumed to be in Linear Rec.709 is:
- There are no file rules in the OCIO config
- There's no meaningful color space metadata in the .exr file

Since the output image stores its colors in ACEScg space, textureC gets converted from Linear Rec.709 to ACEScg and appears correct. textureB also gets converted and has the ACEScg colors transformed again, so it appears incorrect.
There seems to be a bug in OpenColorIO handling the FileTransform rule for the "Input - Generic - SRGB Texture" to "ACEScg". It seems the gamma is not removed. This may be because I was missing a LUT in the test.

Houdini Config - ACEScg
Scene Linear = ACEScg
textureA: srgb_tx -> ACEScg
textureB: ACEScg -> ACEScg
textureC: ACEScg -> ACEScg

Before the fix for HOUDINI_TEXTURE_DISK_CACHE, XPU would generate incorrect images. This has been fixed.
Since scene_linear matches the file rules for .exr files, the .exr texture data is unmodified and the output image is written in ACEScg color space. The oiio:ColorSpacemetadata reflects this.

Houdini Config - ACEScg - No file rules
Scene Linear = ACEScg
textureA: srgb_tx -> ACEScg
textureB: Linear Rec.709 -> ACEScg
textureC: Linear Rec.709 -> ACEScg

Without OCIO file rules, this config file behaves similarly to the RedShift config file. Karma assumes the .exr files are in Linear Rec.709 color space.

Optimized Textures

You can use imaketx or hoiiotool (or other utilities) to convert images to optimized format.
If you need to set the oiio:ColorSpace attribute on an .exr file explicitly, you can do this using
hoiiotool foo.exr --attrib oiio:ColorSpace "ACES - ACEScg" -otex fum.exr
imaketx will store color space metadata automatically, but if you have OCIO file rules, they take precedence.

Note: With HOUDINI_TEXTURE_DISK_CACHE, we considered using .exr files on output, but .rat files (like .tif/.tx files) support 8-bit and 32-bit data.
User Avatar
Member
433 posts
Joined: 4月 2018
Offline
Thanks for this very detailed breakdown, Mark. I did a few more tests and my conclusion is that it's best to bake your desired colorspaces manually using imaketx. By going that route I could get 100% reproduceable results with XPU and everything looked exactly right in render.

A simple command of imaketx <input_image> <output_image.rat> -c <input_space> <working_space> is all it takes. It's not too hard to wrap that up in a script with a loop that processes an entire folder automatically.
Subscribe to my Patreon for the best CG tips, tricks and tutorials! https://patreon.com/bhgc [patreon.com]

Twitter: https://twitter.com/brianhanke [twitter.com]
Behance: https://www.behance.net/brianhanke/projects [www.behance.net]
  • Quick Links