In mplay, you can actually click and drag a display window that shows data in the overscan area.
If you zoom out, you should see the blue border showing the data window.
Shift-click and drag out a region and you can see the data outside the display.
Found 662 posts.
Search results Show results as topic list.
Solaris and Karma » The "correct" way to set up overscan for Karma H20?
- mark
- 2590 posts
- Offline
Solaris and Karma » Karma OCIO file rules in XPU and differences between h19 and
- mark
- 2590 posts
- Offline
@pawel-fin, thank you for submitting the test files.
After going through the examples, there were a few things that I thought would be useful to share with the community. Understandings about how OCIO is handled in H20.0.
I hope this helps clarify some of the behaviours in H20.0.
USD/MaterialX
USD stores the color space information for MaterialX nodes as metadata on the parameter. The Hydra interface (used by Karma/husk) does not currently pass this metadata through to the renderer. So, the color space metadata doesn't make it through from VOPs to Karma.
If a renderer loads the USD directly (without Hydra), it's able to access this metadata, so behaviour might be different. Later versions of USD will have this fixed.
OpenEXR Color Space
To determine the source color space, Karma (and VEX) use the following rules:
a) If the color space is explicitly specified, use it (this doesn't work with USD/MaterialX, see above)
b) Match using the OCIO file rules
c) If there are no file rules, use metadata in the image
d) If there's no metadata, fall back to a default
If we get to the last stage, the default for 8-bit images is to assume the color space is sRGB. For OpenEXR, Karma assumes they are Linear Rec.709. This is according to the OpenEXR spec for files missing the chromaticities attribute (https://openexr.com/en/latest/TechnicalIntroduction.html)
Note that OpenEXR suggests the chromaticities metadata should be used to determine the color space. However, neither OpenImageIO, OpenColorIO or the Houdini core libraries are able to determine an OCIO color space from the chromaticities.
Houdini will use the
H19.5
H19.5 assumed scene_linear was Linear Rec.709 in all cases. It ignored the OCIO file. For the most part, H19.5 also ignored the OCIO config file entirely when interpreting color spaces. For 8 bit files, there was a built-in conversion from sRGB to Linear Rec.709. OpenEXR files were read as "raw", so no color space conversion happened.
As I pointed out, in your example, H19.5 rendered identical images, regardless of the OCIO config file. That's because H19.5 ignored OCIO.
H20.0
As mentioned above, H20.0 obeys the OCIO scene_linear ccolor space and uses the OCIO rules to perform color space computations.
None of the textures in your example were optimized (tiled and mip-mapped). In H20, Karma XPU automatically creates an optimized texture for rendering (see the HOUDINI_TEXTURE_DISK_CACHE variable). There was a bug with this process when scene_linear was not set to Linear Rec.709. This was fixed in H20.0.633 (March 1, 2024).
It's highly recommended that you use optimized textures since they consume less memory, have better filtering and are in general more efficient. Look at HOUDINI_TEXTURE_DISK_CACHE for options to automate this process.
Your Test Cases
For those readers without access, there were three textures provided:
- textureA = JPG image
- textureB = OpenEXR in ACEScg color space
- textureC = OpenEXR in Linear Rec.709 color space
As mentioned above, none of the textures had color space metadata and were not tiled/mip-mapped.
Default OCIO Config
This matches H19.5 exactly. textureB is interpreted as being in Linear Rec.709 color space, so comes through unmodified.
RedShift OCIO Config
The reason textureB and textureC are assumed to be in Linear Rec.709 is:
- There are no file rules in the OCIO config
- There's no meaningful color space metadata in the .exr file
Since the output image stores its colors in ACEScg space, textureC gets converted from Linear Rec.709 to ACEScg and appears correct. textureB also gets converted and has the ACEScg colors transformed again, so it appears incorrect.
There seems to be a bug in OpenColorIO handling the FileTransform rule for the "Input - Generic - SRGB Texture" to "ACEScg". It seems the gamma is not removed. This may be because I was missing a LUT in the test.
Houdini Config - ACEScg
Before the fix for HOUDINI_TEXTURE_DISK_CACHE, XPU would generate incorrect images. This has been fixed.
Since scene_linear matches the file rules for .exr files, the .exr texture data is unmodified and the output image is written in ACEScg color space. The
Houdini Config - ACEScg - No file rules
Without OCIO file rules, this config file behaves similarly to the RedShift config file. Karma assumes the .exr files are in Linear Rec.709 color space.
Optimized Textures
You can use imaketx or hoiiotool (or other utilities) to convert images to optimized format.
If you need to set the oiio:ColorSpace attribute on an .exr file explicitly, you can do this using
imaketx will store color space metadata automatically, but if you have OCIO file rules, they take precedence.
Note: With HOUDINI_TEXTURE_DISK_CACHE, we considered using .exr files on output, but .rat files (like .tif/.tx files) support 8-bit and 32-bit data.
After going through the examples, there were a few things that I thought would be useful to share with the community. Understandings about how OCIO is handled in H20.0.
I hope this helps clarify some of the behaviours in H20.0.
USD/MaterialX
USD stores the color space information for MaterialX nodes as metadata on the parameter. The Hydra interface (used by Karma/husk) does not currently pass this metadata through to the renderer. So, the color space metadata doesn't make it through from VOPs to Karma.
If a renderer loads the USD directly (without Hydra), it's able to access this metadata, so behaviour might be different. Later versions of USD will have this fixed.
OpenEXR Color Space
To determine the source color space, Karma (and VEX) use the following rules:
a) If the color space is explicitly specified, use it (this doesn't work with USD/MaterialX, see above)
b) Match using the OCIO file rules
c) If there are no file rules, use metadata in the image
d) If there's no metadata, fall back to a default
If we get to the last stage, the default for 8-bit images is to assume the color space is sRGB. For OpenEXR, Karma assumes they are Linear Rec.709. This is according to the OpenEXR spec for files missing the chromaticities attribute (https://openexr.com/en/latest/TechnicalIntroduction.html)
Note that OpenEXR suggests the chromaticities metadata should be used to determine the color space. However, neither OpenImageIO, OpenColorIO or the Houdini core libraries are able to determine an OCIO color space from the chromaticities.
Houdini will use the
oiio:ColorSpace
metadata. The example .exr files you included both had the oiio:ColorSpace
set to Linear
. Since there are multiple linear color spaces, we've found making the assumption that "linear" maps to "scene_linear" can often lead to bad results, so we don't count "linear" as being a meaningful value for oiio:ColorSpace
.H19.5
H19.5 assumed scene_linear was Linear Rec.709 in all cases. It ignored the OCIO file. For the most part, H19.5 also ignored the OCIO config file entirely when interpreting color spaces. For 8 bit files, there was a built-in conversion from sRGB to Linear Rec.709. OpenEXR files were read as "raw", so no color space conversion happened.
As I pointed out, in your example, H19.5 rendered identical images, regardless of the OCIO config file. That's because H19.5 ignored OCIO.
H20.0
As mentioned above, H20.0 obeys the OCIO scene_linear ccolor space and uses the OCIO rules to perform color space computations.
None of the textures in your example were optimized (tiled and mip-mapped). In H20, Karma XPU automatically creates an optimized texture for rendering (see the HOUDINI_TEXTURE_DISK_CACHE variable). There was a bug with this process when scene_linear was not set to Linear Rec.709. This was fixed in H20.0.633 (March 1, 2024).
It's highly recommended that you use optimized textures since they consume less memory, have better filtering and are in general more efficient. Look at HOUDINI_TEXTURE_DISK_CACHE for options to automate this process.
Your Test Cases
For those readers without access, there were three textures provided:
- textureA = JPG image
- textureB = OpenEXR in ACEScg color space
- textureC = OpenEXR in Linear Rec.709 color space
As mentioned above, none of the textures had color space metadata and were not tiled/mip-mapped.
Default OCIO Config
Scene Linear = Linear Rec.709
textureA: source space srgb_tx -> target space lin_rec_709
textureB: lin_rec_709 -> lin_rec_709
textureC: lin_rec_709 -> lin_rec_709
This matches H19.5 exactly. textureB is interpreted as being in Linear Rec.709 color space, so comes through unmodified.
RedShift OCIO Config
Scene Linear = ACES - ACEScg
textureA: Input - Generic - sRGB - Texture -> ACES - ACEScg
textureB: Linear Rec.709 -> ACES - ACEScg
textureC: Linear Rec.709 -> ACES - ACEScg
The reason textureB and textureC are assumed to be in Linear Rec.709 is:
- There are no file rules in the OCIO config
- There's no meaningful color space metadata in the .exr file
Since the output image stores its colors in ACEScg space, textureC gets converted from Linear Rec.709 to ACEScg and appears correct. textureB also gets converted and has the ACEScg colors transformed again, so it appears incorrect.
There seems to be a bug in OpenColorIO handling the FileTransform rule for the "Input - Generic - SRGB Texture" to "ACEScg". It seems the gamma is not removed. This may be because I was missing a LUT in the test.
Houdini Config - ACEScg
Scene Linear = ACEScg
textureA: srgb_tx -> ACEScg
textureB: ACEScg -> ACEScg
textureC: ACEScg -> ACEScg
Before the fix for HOUDINI_TEXTURE_DISK_CACHE, XPU would generate incorrect images. This has been fixed.
Since scene_linear matches the file rules for .exr files, the .exr texture data is unmodified and the output image is written in ACEScg color space. The
oiio:ColorSpace
metadata reflects this.Houdini Config - ACEScg - No file rules
Scene Linear = ACEScg
textureA: srgb_tx -> ACEScg
textureB: Linear Rec.709 -> ACEScg
textureC: Linear Rec.709 -> ACEScg
Without OCIO file rules, this config file behaves similarly to the RedShift config file. Karma assumes the .exr files are in Linear Rec.709 color space.
Optimized Textures
You can use imaketx or hoiiotool (or other utilities) to convert images to optimized format.
If you need to set the oiio:ColorSpace attribute on an .exr file explicitly, you can do this using
hoiiotool foo.exr --attrib oiio:ColorSpace "ACES - ACEScg" -otex fum.exr
Note: With HOUDINI_TEXTURE_DISK_CACHE, we considered using .exr files on output, but .rat files (like .tif/.tx files) support 8-bit and 32-bit data.
Solaris and Karma » "Render All Frames with a Single Process"
- mark
- 2590 posts
- Offline
jsmackKarma knows whether it's rendering in the viewport or from husk. If it's rendering from husk, it should redice every frame. The same behaviour as "IPR Continuous Dicing".mtucker
2. problems can occur when objects move from far away to up close - redicing may be required, but may not happen
Is there an option to force redicing every frame?
Solaris and Karma » Karma OCIO file rules in XPU and differences between h19 and
- mark
- 2590 posts
- Offline
jsmack
The metadata in the image is used before any rules so if your image has incorrect metadata that can throw it off. The metadata kind of has to take precedence since it's the only way multiple color spaces in the same file can be supported.
Small correction. It turns out that if there are any file rules, they need to take precedence over the metadata.
This post: https://www.sidefx.com/forum/topic/93002/ [www.sidefx.com] pointed out that if we relied on metadata first, textures like
ACEScg_ColorChecker.exr
would not be interpreted as being in ACEScg.This is a little unfortunate since if you have any file rules which match a filename, the metadata will be ignored.
Ideally, there would be some way to say apply priorities to file rules, so you could have something like: "If the filename has ACEScg, force the colorspace to ACEScg, otherwise, use metadata, otherwise fall back to the file extension rule.". Or maybe have a file rule that says to check the file's metadata.
This gets more complicated because there's no "standard" way of specifying color space metadata in OpenEXR files (there are various competing methods in play).
Solaris and Karma » Karma OCIO file rules in XPU and differences between h19 and
- mark
- 2590 posts
- Offline
pawel-finFile rules are supported in H20. OCIO support in H19.5 was not as complete as in H20. In H19.5, Karma had a hard-coded Linear Rec 709 working space. In H20, we use scene_linear. In H19.5 texture conversion may have used the "built-in" Houdini conversions rather than OCIO (in some cases). In H20, color handling is much more consistent and aligned with OCIO (throughout the whole package).
Hi,
Are file rule supported in Karma XPU? we are having a hard time moving production from H19.5 to H20 and getting XPU to match CPU.
out of the box H20 renders our hip files differently. This is a big problem for long forms that are in production for a year while we want to get new features of houdini. For tests we created a simple setup with 3 textures:
Trying to match H19.5 and H20 renders can be challenging because H19.5 was broken in many subtle ways.
If there's a particular use case that you need help matching between the two versions, please submit a bug/issue with an example and we can try to help sort things out.
Solaris and Karma » Husk holding exr open during post-frame script
- mark
- 2590 posts
- Offline
The raster productType is saved out by husk, the openexr is saved out by the prman delegate.
There was a fairly recent improvement to saving multiparts from husk, so you may want to upgrade to the fixed version if you haven't already.
There was a fairly recent improvement to saving multiparts from husk, so you may want to upgrade to the fixed version if you haven't already.
Solaris and Karma » Husk holding exr open during post-frame script
- mark
- 2590 posts
- Offline
Solaris and Karma » VR camera in Solaris
- mark
- 2590 posts
- Offline
If you're using Karma CPU, it's possible using VEX. Karma XPU has a fixed suite of lens shaders at the current time.
Solaris and Karma » Husk.exe stuck not finishing task
- mark
- 2590 posts
- Offline
Does someone has found a solution to husk getting stuck. I am having this problem when trying to render volume.
I have tried with all the Houdini 20 update. Now I am using 20.506 and I still have the problem.
https://www.sidefx.com/changelog/?journal=20.0&categories=58&body=&version=20.0&build_min=582&build_max=582&show_versions=on&show_compatibility=on&items_per_page= [www.sidefx.com]
There was a hanging issue with volumes fixed in 20.0.582 (which 20.0.506 seems quite old).
We have tried to reproduce the issue with husk hanging when rendering but without any success. If some renders complete, but take longer, then, it sounds like it might be something other than a hanging issue. We've also been unable to reproduce this.
As a note, if you're rendering on Linux or OSX, you can send husk a USR1 signal, which will cause husk to save out a snapshot of the current state. Not sure if this might help.
It is something we're continuing to look into.
Solaris and Karma » Husk.exe stuck not finishing task
- mark
- 2590 posts
- Offline
husk has a
By default, this uses the setting in
By default,
--fast-exit
option:--fast-exit arg (=2) 0 - Force a full tear down of the
USD scene and Hydra
1 - Fast exit lets the OS tear down
resources
2 - Use setting in UsdRenderers.json
to use delegate preference
By default, this uses the setting in
UsdRenderers.json
, but perhaps this isn't set for the delegates? Does manually adding --fast-exit
0 fix the issue?By default,
fast-exit
should be 0, so husk will tear down the delegate on exit.
Edited by mark - 2023年11月26日 18:00:58
Solaris and Karma » OCIO : Opengl vs CPU vs XPU
- mark
- 2590 posts
- Offline
Yes, mantra will follow the same rules:
a) Use the OCIO file rules
b) Use image metadata
c) Fall back to guessing based on the file type/data type and other heuristics.
a) Use the OCIO file rules
b) Use image metadata
c) Fall back to guessing based on the file type/data type and other heuristics.
Solaris and Karma » OCIO : Opengl vs CPU vs XPU
- mark
- 2590 posts
- Offline
This turned out to be an issue with the way Karma determined the color space of the texture. There was an inconsistent priority when looking at image metadata and parsing of the OCIO file rules. This should be consistent in the next daily build of H20.0.
Thanks for finding this.
P.S. Karma will now look much more like Houdini GL. For clarity:
-
-
Since the image data is the same between the two images, when they get converted to scene_linear, you'll see different values come through.
Thanks for finding this.
P.S. Karma will now look much more like Houdini GL. For clarity:
-
ACEScg_ColorChecker.exr
will be assumed to be in ACEScg color space-
A_C_E_S_c_g_ColorChecker.exr
will be assumed to be in Linear Rec 709 (sRGB)Since the image data is the same between the two images, when they get converted to scene_linear, you'll see different values come through.
Edited by mark - 2023年11月13日 13:54:13
Solaris and Karma » Sending USR1 to prman to pre-empt a Husk render
- mark
- 2590 posts
- Offline
`husk` traps USR1 to save out a snapshot - it continues rendering. snapshots are different than checkpointing.
I'm not sure whether the prman render delegate captures USR1, or whether it's the prman application. If it's the prman application, it's a different beast than husk.
In a future version of Houdini, husk will pass the fact it was signaled with USR1 to the delegate, and the delegate will be able to take actions (like saving a checkpoint and terminating). But it will need to be implemented in the render delegate.
I'm not sure whether the prman render delegate captures USR1, or whether it's the prman application. If it's the prman application, it's a different beast than husk.
In a future version of Houdini, husk will pass the fact it was signaled with USR1 to the delegate, and the delegate will be able to take actions (like saving a checkpoint and terminating). But it will need to be implemented in the render delegate.
Technical Discussion » Rendered EXR image ignores added metadata when "exrmode=0"
- mark
- 2590 posts
- Offline
When using legacy .exr files (exrmode=0), the metadata support isn't as complete as when storing multi-part images.
This is a known issue with the legacy OpenEXR driver, but submitting a bug/RFE would be helpful.
This is a known issue with the legacy OpenEXR driver, but submitting a bug/RFE would be helpful.
Houdini Lounge » Houdini 20 Rumors
- mark
- 2590 posts
- Offline
AhmedHindy
make Karma/ Mantra convert textures to .rat on the fly and use it just like Arnold. I think Arnold is the best in tersm of .tx files generation since I dont need to manually enter ".tx" at the end of every jpg on every single shader.
big hassle for big scenes with alot of megascans assets
This was added in H19.5:
% hconfig -h HOUDINI_TEXTURE_DISK_CACHE
HOUDINI_TEXTURE_DISK_CACHE
Some image formats will have better performance when used as texture
maps when rendering. The imaketx program can be used to create these
high-performance texture files. This variable controls how non-ideal
texture maps should be used for texturing, automatically running
imaketex on these images to create high-performance texture files. This
conversion is done once, resulting an overall performance increase.
* Unset or native: will use the raw texture map with no conversion
* local will create a high performance texture file in the same
directory as the source texture (if possible).
* temp will use a local disk cache to store the high performance
texture files. This cache can be controlled using the htexcache
command line utility.
* all will first try to create a local texture file and if that fails,
it will fall back to the temp disk cache. For example, a texture
file $HIP/maps/color.png would have a local texture file created in
$HIP/maps/color.rat, but a texture stored in an HDA cannot have a
local texture created, so a cached file would be created in the temp
disk cache.
Solaris and Karma » Karma EXR data window
- mark
- 2590 posts
- Offline
% husk
Usage: husk [options] usdfile
Build version: 19.5.441
...
--autocrop arg Pattern of AOVs to be considered when
computing the data window for
auto-cropping
...
Edited by mark - 2022年11月23日 22:21:32
Technical Discussion » Question: Enable rman style tesselation? (Karma)
- mark
- 2590 posts
- Offline
USD has a subdivision surface primitive type. Are you rendering Subdivision Surfaces or Polygon Meshes?
Technical Discussion » Question: Enable rman style tesselation? (Karma)
- mark
- 2590 posts
- Offline
Karma also does adaptive screen space tesselation.
For interactivity, continuous dicing is disabled in the IPR viewport. You can turn this on to get Karma to redice geometry while adjusting the camera in the viewport (open the display options for the viewport - the 'd' key).
For interactivity, continuous dicing is disabled in the IPR viewport. You can turn this on to get Karma to redice geometry while adjusting the camera in the viewport (open the display options for the viewport - the 'd' key).
Solaris and Karma » Texture issues XPU vs CPU
- mark
- 2590 posts
- Offline
As a note, in today's daily build of H19.5, Karma CPU should be respecting the default color when there are missing images in Mtlx shaders - so the initial scene should be much closer between CPU and XPU.
Solaris and Karma » .rat vs .exr in Karma
- mark
- 2590 posts
- Offline
The big difference between .exr and .rat textures is that the texture evaluation engine is different.
For .rat textures, we use the Houdini texture evaluation library. This does filtering one way.
For .exr textures, we use Open Image IO, which does texture filtering using a different way.
Since the two libraries use different algorithms for filtering, I don't think you'll ever get a 100% match between .rat and .exr files.
By default, we're using the "smart bicubic" filter for .exr files in OIIO.
You might get closer results by using "bicubic" filtering for both .exr and .rat files (as described in the
For .rat textures, we use the Houdini texture evaluation library. This does filtering one way.
For .exr textures, we use Open Image IO, which does texture filtering using a different way.
Since the two libraries use different algorithms for filtering, I don't think you'll ever get a 100% match between .rat and .exr files.
By default, we're using the "smart bicubic" filter for .exr files in OIIO.
You might get closer results by using "bicubic" filtering for both .exr and .rat files (as described in the
filtermode
keyword argument in the texture documentation: https://www.sidefx.com/docs/houdini/vex/functions/texture.html [www.sidefx.com] )
-
- Quick Links