Hi;

I have a scene in which I'm generating a dozen or so procedural trees. At the moment, I lump all these trees together in one SOPnetwork; each tree has a path like:


When I do a SopImport to bring all these in, I see the USD heirarchy I expect, tree_01, tree_02, etc.

I'd like to write all of these out to disk, to use them elsewhere. At the moment, I put them all in one USD file, trees.usd. When I import this file elsewhere, my heirarchy is preserved and everything works fine.

However, I 1) would like to animate my trees, and 2) suspect that lumping everything into one file isn't very USD-like. I'm aware of the concept of layers, and see some options in the USD export node in LOPS that suggests that if I have a layer save path specified, I can generate one top-level file that contains N references to layer files. So, it seems like I should consider having:


I can't tell if there's an attribute I can set on my tree geometry to help split them all into separate layers though; the sopimport has one layer save path attr...can this be varied somehow based on a geometry attribute? Or, is there another way I could accomplish this splitting?

Thanks!

It sounds like what would suit your purposes best would be to store each of your trees as a variant of a main trees asset. Take a look at the attached file, which uses the Component Builder to store multiple rocks as variants of an asset.

Setting it up isn't terribly elegant, as you'll have to make a connection between the Component Geometry node and the Component Geometry Variants for each tree. If you have a lot of them, that could get quite tedious.

Hmm, thanks for this! Though, to clarify my original description of what I'm doing: there is no "main tree"; I have ten wholly-different trees, each of which is different geo, different branch arrangement, etc. My understanding of variants is that I might use them in the case of, say, changing the seasons (i.e. leaf color) of a single tree, such that I have tree_01:summer, tree_01:autumn, etc.

But, what I'm trying to do first here is just manage these 10 different trees properly/efficiently. I think that means storing each tree in a different file, which I think == layers. Perhaps within each layer I can add variants to each tree for seasons (as an example).

I'm still green enough with USD that it *seems* like I should be able to specify an attribute on my tree geometry like @layerSavePath="/path/to/tree_01.usd", but I can't find any notion of this in the docs. All I can find is a description of the layerSavePath param on a sopImport node, which suggests I would need 10 different sopImport nodes in order to specify 10 different layer paths, which seems oddly un-Houdini-like. I'm sure I'm just not understanding something though...

Continuing trying to explore this, here is a hip file that I think might be close but doesn't actually work yet (it writes out separate files per tree, but the files all seem to be empty!)

Curious what I'm doing wrong...

Here's a modified version of that scene which does the SOP Import inside the loop, thus creating one layer per tree (and only importing one tree each time). Hope this helps!

Amazing, thank you so much @robp_sidefx! I'm curious if I could trouble you to explain what I was doing wrong, or how this conceptually differs from what I was trying in my scene, as I'd love to understand this better. Inspecting the Scene Graph Layers panel, the data all seems the same (at least, to my untrained eye)...what is it about this setup that allows it to work?

I beg Rob to correct me if I'm wrong about this, but in your Begin Context Options Block LOP, you have Perform Layer Break checked. The Layer Break ensures that everything above that node (that is, your entire scene) is not saved. That's why all of your files are empty.

If you were to turn that off, however, you'd find that each of your sublayer files would contain all of the trees instead of just the one you were targeting because USD is designed to be non-destructive; removing prims from the scene graph is not easy.

You could get around that by using a Configure Stage LOP to create a temporary stage with only the prims you are interested in. The attached example has those minimal changes to your starting point (I did adjust the save paths a bit so as not to pollute my working folder.) There seems to be a bit of a bug somewhere in there with ITERATIONINDEX; tree_01 is empty. I'm not sure what caused that, but I haven't taken the time yet to chase down why it's happening.

Rob's file, of course, is much cleaner and more efficient.

Thank you so much! To try to say that back to you: past the point at which I'm importing my geo into LOPS (my single sopImport node), I am working with a stage that - at least in USD parlance - is designed to be non-destructive; changing layers past that point would constitute a destructive edit, and so while of course Houdini offers ways around this, I am bucking the intended way of accomplishing this to some degree.

I'm curious how one is meant to use the Configure Layer node - its existence sort of hinted to me that I should try doing this the way I did, but clearly I misunderstood its purpose a bit.

The "USD is meant to be nondestructive" co-existing inside Houdini where one can pretty much do anything at any time really twists my head around sometimes! I'm sympathetic as to how hard this must be to resolve from a developer standpoint.

I believe the intent was to enable you to break off a portion of your stage to work on in isolation for better interactivity. Even deactivated prims still have to be processed if there's a recomposition, so entirely removing them from the stage can improve performance.

Stage Manager can do the same kinds of things, so I caution anyone using either node to be careful. It would be really easy to break a pipeline by reconfiguring a stage!

Thank you so much for the additional insight, truly appreciated!

A followup question - which I realize is wholly different from my original one: when exporting my treees (which now works great: I get a separate file for each tree), I notice that when I export a frame range rather than a single frame, I seem to get absolutely enormous files (or, well, they are quite a bit larger than the bgeos from which they derive). I know USD makes it very easy to get a lot of complexity going quickly, but I feel compelled to ask if docs exist that discuss strategies (if any) for optimizing this?

To be clear, my trees are swaying gently in the breeze: so, constant point count, no IK involved, just some moving vertices. Am I doomed to massive files?

I'm not real strong on time samples in USD, nor on how it compresses, but I know a bit. The geo portion of your usd file is really just an ordered list of 3d position triplets. Since USD doesn't know anything about whatever rigging produced the motion*, your point cached animation is going to have to be the new positions for every single one of those points, so every time sample will be nearly as large as the original.

If you can reduce the number of samples you create, you can therefore reduce the size of the usd file. The simplest way to do that would be to use the Cache LOP and increase the Increment value to something greater than 1. Positions will be interpolated between samples, so you won't get steppy motion, but you might still see the occasional discontinuity in motion if some of the discarded samples contained critical parts of an eased curve.

Definitely don't turn on subframe sampling in the Cache LOP or use the Motion Blur LOP when saving an asset—doing that will at least double the number of samples you're saving.

Make certain you're not saving usda files, as those are always larger than the binary usdc. If there are portions of the tree that aren't moving, like the lower part of the trunk, you could consider separating those to a subcomponent and ensure you're only getting time samples for vertices that are actually moving.

You might also be able to get some optimization by instancing leaves. That way, only the points that they're attached to have animated transform data, and the leaves themselves are static.

But again, I'm hardly an expert. Learning USD and Solaris is my full time job right now, but I haven't been doing it very long!


*I believe UsdSkel is a method for using skinned meshes in USD, but I have no clue yet how it works or what kind of support we have for it in Solaris. Theoretically, that would vastly reduce the file sizes since you'd only be tracking joint transforms instead of all of the mesh point.

Ok this is terrific, thank you; I'm learning so much here. I did indeed fear that what I'm effectively doing is baking 100 copies of my tree into a single file if I'm exporting a 100-frame animation sequence of the tree (I understand these are time samples, but conceptually there's not a lot of opportunity for compression here). The tip about reducing the time samples is a good one, I'll experiment with how much I can get away with by stashing every, say, 4th frame or so.

Another question: Once I've written out my tree animations, loading them back in via a reference brings Houdini to its knees. This seems like the exact opportunity to leverage one of USD's allegedly most-powerful features, which I think is deferred loading of all this animation data...is that right? Is the term "payload" related to this idea? If so, do I need to do anything differently when exporting the animation? Or is it as simple as ticking off the 'load payloads' option on the Reference LOP? How do I, um, 'declare' something as a payload?

p.s. I was talking over my scene with a fellow coworker who has much more intimate experience with USD than I do, and he too brought up the UsdSkel idea, but then advised that my trees probably aren't an ideal case for that mechanism, unless I'm doing something very simple like just wiggling around the trunk a bit (which I am not).

BryanRay
The simplest way to do that would be to use the Cache LOP and increase the Increment value to something greater than 1. Positions will be interpolated between samples, so you won't get steppy motion, but you might still see the occasional discontinuity in motion if some of the discarded samples contained critical parts of an eased curve.

I'm trying this this morning, though it's not clear to me exactly how to set this up. If, say, I want to reduce my time samples to 1/5 the original amount, and interpolate in between each sample (something USD itself does?), do I set my Cache LOP behavior to "cache up to cooked frames", and the start/end/inc to something like $F-5, $F+5, 5?

No, you want your start and end frames to be the actual start and end of your animation. So for instance, if it's a 100 frame animation, set it to 1, 100, 5. Or $RFSTART, $RFEND, 5 if your timeline is set up to the length of the anim.

And yes, I think USD itself does a linear interpolation between the time samples.

Payloads: There are two steps to using payloads effectively: First, in the Scene Graph Tree panel, you'll want to disable viewport loading of payloads:



Then, in the Reference LOP, change Reference Type to Payload File. Do both steps before actually loading the file. That will prevent the geo from being loaded in the viewport. However, there's another step you might want to take before saving them. If you add a Loft Payload Info LOP, then some metadata about the bounding box will be added, and you'll at least be able to see the extents of the geo when it's not loaded.

It seems to me, though, that it should be able to load data for just the current frame, so I wouldn't expect the presence of other time samples to slow things down so much. It may be that I need to do some more reading and experimenting along those lines.

Something else you might do to help yourself is to save a static version of the trees for use in layout and lighting, with the intent to layer the animation over in a later step. That way you need not load the heavier files until you actually need them, perhaps not until right before rendering.

You're my hero, this is all so enlightening!

BryanRay
Something else you might do to help yourself is to save a static version of the trees for use in layout and lighting, with the intent to layer the animation over in a later step. That way you need not load the heavier files until you actually need them, perhaps not until right before rendering.

Oh I like this idea; how do I specify the separation of static tree geo vs. animation? Sounds like you're saying I export one (smaller) single frame USD per tree, then one (much larger) usd containing the animated tree, and then...?

The way I am doing this is though a sop modify lop. And exporting out the first frame with added prim Vars such as rest etc, then further down killing the normal and uv data, then exporting the next part out as the anim file. The two resulting usd files can be sublayerd together. I am saving a ton of space this way. Best. Mark

This is a great idea, thank you! I'm going to try this.

How does one reconfigure the layerSavePath after it's set? Like, ideally I would do a sopImport, using Rob's elegant method above, assign my materials and other render settings, then export a single frame. Then, set my layerSavePaths to different filepaths, remove extraneous data, and save the full length of the animation.

Also, how are you removing N and uv in a sopModify? I'm doing what I think is the obvious thing (unpacking geo as polys, attrDelete all vertex attribs), but it seems to not affect my geo, which is puzzling...

Hi. Here is a screenshot from within my HDA. For creating a single frame usd export. I am using the "flush data" option. The attribute delete is doing just what you said. Removing N, and UV. Best. Mark

Ah, you're doing the USD export from within the SOP modify; that's the bit I was missing. Thank you! Trying to remove the attribute and push it back over into LOPS doesn't seem to result in those attrs being removed (for reasons I could only educatedly guess at...nondestructive workflow something something).