I'm a bit on a crossroad here on if I want to use Solaris/Karma or go back to RedShift.. This is 3th or 4th time since it's inception that I tried really to make Solaris/Karma work for me and after the occasional sparks of hope I keep running into it's shortcomings. Performance, stability, workflow and overal design and implementation are all tripping over each other and the result is me trying to find the words to not turn this into a rant about what a convoluted mess it is.

All I want is to use Karma as a nice render engine for Houdini without all the USD drama getting in the way. Translating native Houdini on the fly into USD will always be a bottle neck. USD in general isn't a performant design, it's made for handling massive Pixar style scenes using cached out animations and instancing techniques .Everything turns in to thick mud with all the overhead and workflow issues, so much so it's removing Karma from the viable options list for using it as a 'normal' render engine for small shop stuff which a shame because it's actually quite nice.

Is this just me? What are the thoughts here? Ideally I'd like to see a native Karma that doesn't rely on USD. Not hiding USD from view but really rendering native Houdini content like Mantra use to do.

Hi Jonathan,
I Felt the same when tied to set SOlaris/Karma as my new workflow.. Buggy experience, heavy scene even with only 4 packs and sometimes a complicated workflow for simple things..
In an other hand, I love the way solaris is handling philosphie, I loved the way you can organise your scene with flexibility..
As you says, if we could use SOlaris and Karma without USD, it would be PERFECT!!!!

I find Solaris too clunky for now. There is much to like and SideFX is amazing, but there are too many quality of life features missing. Just check out the recent thread here about OCIO in Karma... Things that are smoothly implemented in other apps require extra thought and effort in Solaris/Karma.

I am not sure what are they thinking ,i guess now they felt they needed to score some points with the animators.

Absolutely agree, though had great hopes for this as houdini always missed a proper layout/lighting context, but sad truth its still missing it.

They released this 4 years ago in a pre-alpha state, to never really touch it again.

Jonathan de Blok
Translating native Houdini on the fly into USD will always be a bottle neck. USD in general isn't a performant design, it's made for handling massive Pixar style scenes using cached out animations and instancing techniques.

I think I understand your frustration, but I disagree that it's as big of a problem as you suggest at the moment. I fully transitioned to solaris+karma from redshift and could never go back. IMO you can really mostly ignore most of the USD stuff if you don't want to bother with it, but it can be incredibly handy especially when you are trying to scale up which as you say it's built for.


To me, the main gotcha here, is that at the moment, solaris can't really replace obj, at least for me and many other people I would assume, and a big part of that is due to the translation of native houdini to USD on the fly as you mentioned. I hope and trust that this will change and improve over time, but in the meanwhile, I treat solaris as PURELY an enviornment for doing my lighting, lookdev, and rendering. When it comes to the layout part-- i think most people would agree with me that it's just not there yet, and I wouldn't recommend using it for that.

If you just use obj and then scene import stuff to solaris for lighting etc, it's pretty dreamy. Especially if you end up caching USDs of your scene, which then makes lighting/rendering smooth as butter, and much nicer than working in obj for that. But its super easy for users to get confused about this and end up trying to do it all in solaris, which I think will usually end in regret (for the time being).

There's definitely a lot of UX improvements that can be made to solaris and I know sidefx is very invested in doing this, but it will take some time, and I also want to give them credit that solaris has gotten much much better with important UX improvements that come each release. I'm looking forward to those improvements, as well as optimizations, but in the meantime I do find Karma + solaris to be pretty great for my workflow, and I tend to be working as a solo freelancer or very small studio responsible for all aspects of cg produciton including rendering.

blakshep
I am not sure what are they thinking ,i guess now they felt they needed to score some points with the animators.

Absolutely agree, though had great hopes for this as houdini always missed a proper layout/lighting context, but sad truth its still missing it.

They released this 4 years ago in a pre-alpha state, to never really touch it again.

this is just incorrect, they have had plenty of updates and work that has been done / is being done on solaris over the past 4 years and prior. they have a completely different team of people who are dedicated to animation/rigging.

WowNems
As you says, if we could use SOlaris and Karma without USD, it would be PERFECT!!!!

I hear this sentiment expressed every now and then, but to be clear, neither Solaris nor Karma would exist without USD. The amount of infrastructure provided by USD and hydra is absolutely enormous. They can't simply be "replaced with native Houdini". Houdini never had these capabilities. A scene graph? Layered opinions? LODs? Variants? Composable scene graphs? An industry standard renderer-agnostic rendering API? A (nearly) universally understood data format for sharing between DCCs? Multithreaded scene construction? A complete python API for scene graph manipulation? An army of Pixar, Autodesk, nVidia, Adobe, Apple, and other developers working together on the underlying framework?

The alternative to embracing USD and hydra was never "stay on the current path". Does anyone remember trying to work with stylesheets for material assignments and overrides? And continuing to live with no actual overall scene description? Just a free floating cloud of objects that each needed to be represented by a separate node in /obj? That would never be able to compete with the likes of Katana.

The only actual alternative to adopting USD and hydra was writing our own equivalent to provide all this functionality ourselves. And then to also write a robust import/export system for interchanging with USD (since it really is the closest thing we have to an industry standard these days). After five years of working on that we would have had a much worse equivalent of USD with much worse tooling built on top because we would have had a lot less developer time to devote to the tooling with more resources (but nowhere near enough) focused on the framework. I have never doubted our decision to put USD and hydra at the core of Solaris. Not because USD and hydra are uniformly wonderful, but because I have given a great deal of thought to the alternative.

Now, do I wish we could have accomplished more by now? Of course! I have a list of tasks a mile long that I am eager to get to. If you ask me again in five years, or ten years, I can almost guarantee I'll be saying exactly the same thing. But I also think we will have made a great deal of progress, just as I think we have over the past five years. I think we have a pretty good idea where Solaris comes up short, and we'll keep plugging away trying to make it a little better with each release, for what we hope is an ever-growing set of users. Thank you all for your thoughts!

mtucker
I hear this sentiment expressed every now and then, but to be clear, neither Solaris nor Karma would exist without USD. The amount of infrastructure provided by USD and hydra is absolutely enormous. They can't simply be "replaced with native Houdini"

Yeah, it's clear that USD brings a lot of tech to the table and it's obviously a sound decision to take that route for Solaris, that's all fine and good. The "Karma without USD" sentiment is not so much a technical issue, it comes from the UX when you simply want to dial in some materials and render from /obj using Karma. I know you guys take your feedback seriously and I might have been a bit frustrated when I wrote the OP, it's just that I see all these great ingredients but the final dish is slightly undercooked and missing a pinch of salt!

Anyways, just thinking out of the box.. technically it should be doable to skip the USD intermediary and go directly from /obj to Hydra primitives and feed that into Karma. I'm probably oversimplifying thing a bit but If for example the SDK's obj-level GeometryNode's class gets a method to return Hydra primitive data directly that should work just fine to render /obj as-is in any Hydra delegate. It should open the door to much more efficient way into Karma for 'simple' usage since it would know what to recook and what not using all the existing time-dependency and caching code that's already in native Houdini. It's only a hand full of base classes that would need this (geo, light, cam, volume) Not sure about materials but they should carry over quite easily.

That would basically create the option to choose between a performant no-fancy-pants-render-as-is and an all-you-can-eat-USD path without compromising each other. Once that works maybe blur those lines a bit and carry over some requested features.

Having a USD "scene delegate" that translates SOP geometry directly to hydra prims is something that has always been on the big wish list, but it's a huge amount of work, so we've always shied away from it. Especially with the major upgrades that have been drifting into hydra over the past couple of years it was extra hard to justify that effort. Now that the new hydra framework has settled down it at least becomes feasible to think about this again, but it's still a huge amount of work, and so would take a lot of developer time away from other things. The other knock against it is the limited contexts in which it would be useful. It's basically only going to work for in-Houdini-process render delegates. Add to this the work we're doing to improve workflows where Houdini GL is rendering a stage and the SOP viewer is rendering SOP geometry into the same viewport, and the use case gets even smaller.

So yeah, it's a good idea in theory, but has so far been rejected because of the calculation of effort versus how much is gained by how many people. A "SOP scene delegate" has just never made the cut. It really hasn't even been close, to be honest. But I'd never say never.

For now we are just focusing on the "Karma ROP" workflow where we have tried to make the translation process from OBJ to karma seamless and robust (and as fast as it can be given the inherent limitations of importing to USD before sending to the renderer).

Ok, well then just make it all a lot faster so there is no need to bypass USD in the first place

But seriously, if I animated 25 individual boxes in /obj and open up the Karma ROP node it plays back at +-20fps at the candidate SceneImport node and around 10 at the end of the Karma ROP graph. I'm not really convinced that that is as fast as it can possibly be. The animated transforms trigger a recook of nearly the entire Karma ROP graph and why the scene import node is choking on this I do not even dare to guess!

I guess I'm not telling anything new here, just want to express my concern for the entire /obj->Karma workflow.


(btw: the Karma ROP doesn't like /obj level path nodes. Draw a path, add a sweep to turn it into a tube.. doesn't translate into USD. Copy the path node's content into a geometry node and it does render.)

FWIW it has always felt to me like the translation from obj/sops is pretty snappy, but the granularity of the change tracking is no where near fine enough. Thus seeming small changes to the sop data (adjusting a single point or single uv coordinate for example or adjusting an xform on a single packed prim) results in a retranslation of way more data. That’s always been a general sop issue anyway I guess, compared to systems that automatically track everything at attribute granularity.

antc
the granularity of the change tracking is no where near fine enough

There are places in Solaris where we're (reasonably) clever about minimising the frequency/impact of recooking, but it is fair to say we generally are still recalculating a lot more often than should be necessary. It is something we try to continually invest in, and certainly invite everyone to shout at us loudly regarding what they see as the worst offenders.

Jonathan de Blok
btw: the Karma ROP doesn't like /obj level path nodes

Thanks for flagging this. You can try using the attached as the starting point of a translator (put it in $HOME/houdiniX.Y/husdplugins/objtranslators). Please note it's definitely not complete in terms of all the flags/parms it should check in terms of "should this end up in the render or not"

robp_sidefx

Jonathan de Blok
btw: the Karma ROP doesn't like /obj level path nodes

Thanks for flagging this. You can try using the attached as the starting point of a translator (put it in $HOME/houdiniX.Y/husdplugins/objtranslators). Please note it's definitely not complete in terms of all the flags/parms it should check in terms of "should this end up in the render or not"

Should a path node end up in a render? I thought it was more like a rig to create a path that would get used in some other sop geometry object.

Also, path node, in 2024?

jsmack
Should a path node end up in a render?

It does have a "Renderable" parm ... soooooo ... sometimes?

Even if it's not meant to end up in a render, its presence in /obj shouldn't trigger warnings from Scene Import.

Thanks for the translator!

I use path nodes to create some simple animated electronic cables. Paths basically are like rigged bezier curves where you can parent the control nulls to powerplugs and other equipment. Works surprisingly well for these kind of things!

robp_sidefx

antc
the granularity of the change tracking is no where near fine enough

There are places in Solaris where we're (reasonably) clever about minimising the frequency/impact of recooking, but it is fair to say we generally are still recalculating a lot more often than should be necessary. It is something we try to continually invest in, and certainly invite everyone to shout at us loudly regarding what they see as the worst offenders.

I was thinking (and I'm sure you guys did too ) but just to float the idea.. would an APEX like approach make things much faster for USD? So basically a scheme where the USD nodes only add things to a 'todo' list, which gets optimized/pruned etc and only executed at the end and/or at nodes that have some sort of 'force execute' flag enabled.

Jonathan de Blok
So basically a scheme where the USD nodes only add things to a 'todo' list, which gets optimized/pruned etc and only executed at the end and/or at nodes that have some sort of 'force execute' flag enabled.

The HoudiniProceduralAPI / "husk procedurals" kinda sorta wanders in this direction in allowing for deferred/on-demand evaluation.