Hi all
I am new to solaris. If anyone can give me a hand with this it would be awesome.
I have exported a large USD file from the software City Engine. It is a model of a city.

It is a large scene where each Mesh has attached its own USDPreviewSurface material to which are connected a set of PBR textures and the UV sets which must be used with each.
I have successfully imported this USD in Solaris as a reference using the reference node.

What I would like to do is to convert all of these USDPreviewSurface materials to 1 single Principled Shader, where each texture path is variated by Primitive Mesh.
In mantra i guess this would be a Material Override.
In Maya Arnold a similar thing can be done with the Tokens.

So, for instance if I have a

Road_GEO,
a Roof_GEO
and a Wall_GEO,

in each imported USDPreviewSurface diffuse Color attribute there would be

inputs:diffuseColor
inputs:file @textures/Mud_BaseColor.png@
inputs:diffuseColor
inputs:file @textures/Tiles_BaseColor.png@
inputs:diffuseColor
inputs:file @textures/Stone_BaseColor.png@

I would want to grab the path of each texture input file and store it into a variable and assign that variable to the path of the new texture.
Is such a thing possible?

I wanted to start by assigning a default Principled shader to all the meshes, but I get this error message:
Authoring to an instance proxy is not Allowed.

Many thanks

Indeed, you can't edit the contents of instanced USD primitives (or else they wouldn't be instanced any more).

The best way to address this is to add an "inherit" arc to all the individual buildings, and then put an opinion in the inherited primitive which sets the new material on the "mesh" child of each building. Note that to do this, you'll want to mark /reference1 as _not_ instanceable. Making it instanceable isn't really accomplishing anything anyway, since there is only one copy of the city in your scene. This will make all the "instanceable" root of each building editable, allowing you to add the inherit arc to them. Because it's an inherit arc, the opinions (like the material binding) set on the mesh sub-prim will be stronger than the opinions coming in through the references (I assume each building is being referenced in from a separate file, and that each building appears many times in the city so you want to preserve the instancing of the individual buildings).

Thanks for the help mtucker!
I tried using the configureprimitive LOP to set the first Level (which is /reference1) to not Instanceable and it has done what you said.
Question 1: Reference LOP vs FILE LOP
Would it have also worked the same if instead of a Reference Node I would have imported the USD with a File LOP?

Question 2: How?

mtucker
The best way to address this is to add an "inherit" arc to all the individual buildings, and then put an opinion in the inherited primitive which sets the new material on the "mesh" child of each building.

I am not sure I understand how to do these 2 steps.
I could find some information on what Inherit [www.sidefx.com] means in USD, but nothing on how practically do that, no inherit LOP, so to speak, nor on how to add an opinion to each primitive and what such opinion should be.
Could you offer some guidance on that please?

Question 3: After this step is done, how do I extract the string of the texture path from the Preview material of each Mesh?

1. A "File" LOP is actually just an alias to the "Sublayer" LOP. If you don't know the difference between these two, you might want to start here: https://www.sidefx.com/docs/houdini/solaris/usd.html [www.sidefx.com]

2. The Reference LOP can create an Inherit arc from a bunch of instance prims to a common class prim.

3. Once you have all the buildings bound to the same material, then you'll want to use inherited primvars on each building to control the texture map(s) to use in the shared material for each instance. I'm pretty sure there are a number of examples of doing that around the forum. Extracting the texture map from the existing material so you can set it as a primvar on each building is a job best suited for a VEX or python LOP (depending on which one you're more comfortable with).

mtucker
1. A "File" LOP is actually just an alias to the "Sublayer" LOP. If you don't know the difference between these two, you might want to start here: https://www.sidefx.com/docs/houdini/solaris/usd.html [www.sidefx.com]

Thanks Mark, I studied the document now. You are right it was fundamental, tutorials won't just do.

mtucker
The Reference LOP can create an Inherit arc from a bunch of instance prims to a common class prim.

Ok, so this means that the material will be passed down to the instances in the reference I got that.
I had no idea that the Reference node could do that too, seems a wildly different thing. Man, they really should make a separate node for that.

I made a few attempts with it but it is not really working.

I am not sure I am plugging the things in the right order and the documentation gives no real example sadly.
It seems that the inherit setting needs 2 elements, a reference and the things you want to inherit the shader from, but if I plug it in the second input it gives a warning.
Also I do not understand what to assign the material to if I am assigning it before the reference/inherit node.

mtucker
Once you have all the buildings bound to the same material, then you'll want to use inherited primvars on each building to control the texture map(s) to use in the shared material for each instance.

I am looking around on the forum, but I must say everything is quite confusing.
What I did understand is that I must use the Material Variation node to generate prim var (#babySteps). Which is a step further, as there is a lot of info on the docs on what prim vars are, but not a lot on how to actually make them.

PaoloGiandoso
I am not sure I am plugging the things in the right order and the documentation gives no real example sadly.
It seems that the inherit setting needs 2 elements, a reference and the things you want to inherit the shader from, but if I plug it in the second input it gives a warning.

Inherit can only come from the first input. In your scene you have the materials connected to the second input, and nothing to inherit from.

PaoloGiandoso
Also I do not understand what to assign the material to if I am assigning it before the reference/inherit node.

The material assignment needs to be applied to the primitives that are the source of the inherit arc. They can be 'none' primitives that are overs, with the root inherited prim being a class 'none' prim. 'none' meaning the prim has no opinion on the type of primitive. The primitives under the class should match the hierarchy of the target.

For example, to change the material assigned to /reference1/Sigil/Shape_4/shape_6/mesh (if that is the primitive with the assignment), create dummy primitives:
(class) /_class_mat_override
(over) /mesh
and assign the new material to this prim named "mesh"

Then create a reference lop set to "Inherit from first input" and set the target to the matching prim (/reference1/Sigil/Shape_4/shape_6) and the source to /_class_mat_override

The source could replicate higher up the graph, performing multiple assignments. This example is just for one assignment.

Thanks for the help you are giving me!
I am so determined to crack this nut
Here is what I gathered from the last set of guidance.
PART A

PART B

PART C

jsmack
create dummy primitives:
(class) /_class_mat_override
(over) /mesh

I did create empty prims with the Primitive LOP I am fairly sure it is the correct way to do it, but I might be spectacularly wrong here

jsmack
The material assignment needs to be applied to the primitives that are the source of the inherit arc. They can be 'none' primitives that are overs, with the root inherited prim being a class 'none' prim. 'none' meaning the prim has no opinion on the type of primitive.

In the Primitive LOP, the "none" tag can be applied to 3 things. Primitive Kind, Parent type and Primitive Type. My guts says you are referring to Kind here.
The term class instead appears under the Primitive Specifier: Class parameter. But there is not a Primitive Class: None parameter I can select. Therefore I went for the Primitive Specifier: Class parameter.
I am not sure I understand what is the "root inherited prim". I assume you are talking about mesh and _class_mat_override but I might be wrong.

jsmack
The primitives under the class should match the hierarchy of the target.

I am not sure what you mean when you say "under the class".
I interpreted this with recreating the structure of the reference file above mesh and _class_mat_override so that is similar to the one inside the Referenced file. Not sure it is the rigt thing to do though.

jsmack
Inherit can only come from the first input. In your scene you have the materials connected to the second input, and nothing to inherit from.

I have shifted the materials to the first input. Now there is nowhere to plug in the file to Inherit, but if I understood you correctly it is enough to type in the Primitive path to it in the Stage, as long as it is already loaded, even without a visible linking line.

jsmack
Then create a reference lop set to "Inherit from first input" and set the target to the matching prim (/reference1/Sigil/Shape_4/shape_6) and the source to /_class_mat_override

Target is the Primitive Path and Source is the Reference primitive Path I guess. I went for that.

It is still not working but I feel I am getting there

Inherit and specialize arcs (unlike reference and payload) cannot be made to external files. Inherit and specialize are always specified as pointing to a primitive path on the current stage. That's why the menu option says "Inherit from first input", and there is no "inherit from multi-input" or "inherit from file" option.

The "Specifier" (rough) equivalent of "None" is "Over". The specifier is really quite different from wither Kind or Type because it controls how USD handles the prim at a higher level. Kind and Type are just bits of metadata that you can put on a prim or not. Every prim _must_ have a specfifier. Have a look at "Over", "Class", and "Define" in the USD glossary (https://graphics.pixar.com/usd/release/glossary.html). Not that reading those three entires will clear everything up, but in general I highly recommend that everyone read that glossary as much as possible. There is so much terminology in USD with very specific meanings which may not match up exactly with anyone's "intuitive" understanding of the terms.

Primvars can be created in SOPs and imported into LOPs (usign SOP Modify), with an Edit Properties node, or with an Attribute Wrangle node. As much as possible LOPs tries to hide the distinction between USD "attributes" and "primvars" (which are, at bottom, attributes with a particular prefix and a little bit of metadata). So generally in LOPs to create a primvar, just create it like you would create any other attribute, just remember to name it "primvars:foo" instead of "foo". Again, I'd recommend reading the USD glossary entry on primvars.

Also, if you could provide some simple sample data and a hip file, it would be far easier to assist than trying to pore over the jpgs of small sections of your network...

Here's a hip file I had lying around that demonstrates both these concepts... It has three instanced tori. One has a material parameter overridden by setting a constant primvar on the root of the torus with an Edit Properties node. And one torus uses an inherit arc to add a velocity blur geometry setting to a prim inside the top level instanceable torus prim. This is done with a render geometry settings node, but you can actually make _any_ kind of edit to the class primitives (such as changing the material binding). USD will just apply the inherited edits on top of the original referenced primitive definitions.

You may be interested to turn on the "Show implicit prototyope primitives" option in the scene graph tree to see the implicit prototype definitions USD creates at various points in the LOP network. You'll see that there is only a single prototype even when you add a constant primvar to one of the instances. It's only when you add the additional inherit composition arc that the "instancing" really gets broken and you have two separate prototypes. Then you can RMB on the prototype primitive and choose "Select All Instances" to see which instances are using each prototype.

Hi @mtucker

Thanks for the example, is much easier to follow a file .
I think I am very close now: the primvars are inherited, however the process still is not working.
I will try sending you my scene: The node tree on the left is what I am trying to do, the node tree on the right is the correct process applied to the scene after removing all instances (in that case it works as there is no need for any inheritance).
I placed it on a wetransfer so the link should be active for a few days now.

Click here to download it... [wetransfer.com]

I compressed the houdini file, the .usdc file and the textures. Finger crossed, I never sent a USD before.

mtucker
It's only when you add the additional inherit composition arc that the "instancing" really gets broken and you have two separate prototypes.

Does it means that by doing the inherit we are creating a second, modified "original" model for the instances, that lives aside the first one? Does that make the scene heavy?

The wrangles are not working because their primitive patterns will not match instanced primitives that cannot be modified. As you say in one of your sticky notes, when we add a VEX function to get descendants you'll be able to have your primitive pattern match the shape xform prims instead. In the mean time, you can work around these limitations by creating a "selection rule" (https://www.sidefx.com/docs/houdini/ref/panes/scenegraphtree.html#edit_rule). In the Edit Seletion Rule dialog, create a new selection rule called "texture_shaders". Make sure to turn on "Allow instance proxies", and set the Pattern to "/*/Sigil/*/*/material/texSampler_*". Then change your Wrangle node to have a Primitives parameter of "%rule(texture_shaders)".

Also, in order to affect the instanced prims, the inheritance needs to be put on all the instanceable prims, not way up at the root. The inheritance of material bindings into instances I believe does not work. Fortunately all of your prims have a "mesh" directly under the xform, so they can all inherit from the same class prim that overrides the material binding on that mesh child prim.

I'll also say that while working on this file I found that when I added a new inherited primvar to a prim, I often had to restart Karma to make it pick up the new value. But in the end I think I got it working in the left hand branch. New version of your hip file attached.

Hi @mtucker, thank you so much for all the help! It works now and I would never have solved this by myself

mtucker
The wrangles are not working because their primitive patterns will not match instanced primitives that cannot be modified.

I have looked into Selection Rules now. What seems weird to me is that both Selection Rules and regular Primitive patterns are based on the USD primitive matching syntax. I wonder why one method has the option to affect instances and why the other has not.

mtucker
In the mean time, you can work around these limitations by creating a "selection rule"

Does it means that this is a temporary bug and it will be fixed in future versions?
Does using Selection Rules slow down the scene in the case of big scene files?

mtucker
Also, in order to affect the instanced prims, the inheritance needs to be put on all the instanceable prims, not way up at the root.

Got it, you set Primitives : %instance in the Reference node.
Can I ask why the action was set to Edit?

mtucker
I'll also say that while working on this file I found that when I added a new inherited primvar to a prim, I often had to restart Karma to make it pick up the new value. But in the end I think I got it working in the left hand branch. New version of your hip file attached.

Interesting. I also noticed that Karma starts the render almost immediately but then stops for 20-30 seconds after the first couple buckets in the center before starting again. Also, all snapshots give a "failed to save image" red error.
Do you think these small drawbacks will improve in later versions?

mtucker
Also, in order to affect the instanced prims, the inheritance needs to be put on all the instanceable prims, not way up at the root.

In fact, I saw you changed the class from /__class__Inheritance/Sigil to /__class__Inheritance/mesh
I mistakenly thought that by placing the inheritance on the top level it would have trickled down to all the things nested under it.

mtucker
The inheritance of material bindings into instances I believe does not work. Fortunately all of your prims have a "mesh" directly under the xform, so they can all inherit from the same class prim that overrides the material binding on that mesh child prim.

This is the difficult phrase I am not sure I am getting. We have set up the inheritance on "mesh", which are all instanceble proxy prims. So how come that the Xforms are the ones to be effected?

PaoloGiandoso

mtucker
In the mean time, you can work around these limitations by creating a "selection rule"

Does it means that this is a temporary bug and it will be fixed in future versions?
Does using Selection Rules slow down the scene in the case of big scene files?

Yes, this is something that should be changed about the way the wrangle works. I will add an option on that node to allow it to match instanced primitives with the Primitives pattern. It is up to each LOP node to decide if it's Primitives parameter should match instance primitives or not. Nodes that edit the matched primitives generally don't allow matching instanced primitives, because instanced primitives can't be edited. But as your hip file demonstrates, often a Wrangle LOP can modify primitives other than the matched primitives, so there are cases where matching instanced primitives would be useful.

The fact that you can use a selection rule this way is just a lucky accident that lets you get around this limitation.

There are not significant performance downsides to using selection rules.

PaoloGiandoso

mtucker
Also, in order to affect the instanced prims, the inheritance needs to be put on all the instanceable prims, not way up at the root.

Got it, you set Primitives : %instance in the Reference node.
Can I ask why the action was set to Edit?

Because the other mode (Create) doesn't support primitive patterns. It will only let you edit one primitive at a time.

PaoloGiandoso

mtucker
I'll also say that while working on this file I found that when I added a new inherited primvar to a prim, I often had to restart Karma to make it pick up the new value. But in the end I think I got it working in the left hand branch. New version of your hip file attached.

Interesting. I also noticed that Karma starts the render almost immediately but then stops for 20-30 seconds after the first couple buckets in the center before starting again. Also, all snapshots give a "failed to save image" red error.
Do you think these small drawbacks will improve in later versions?

Not sure about the delay. Maybe the first couple of buckets happen to not contain many textures, but the next set of buckets contain lots of textures? I'm not seeing anything like that. I also don't have any problem when I take a snapshot. Does the error message say where it is trying to save the file? Is there any reason why Houdini might not be able to write to that location?

PaoloGiandoso

mtucker
The inheritance of material bindings into instances I believe does not work. Fortunately all of your prims have a "mesh" directly under the xform, so they can all inherit from the same class prim that overrides the material binding on that mesh child prim.

This is the difficult phrase I am not sure I am getting. We have set up the inheritance on "mesh", which are all instanceble proxy prims. So how come that the Xforms are the ones to be effected?

If I have a prim /foo/bar, and another prim /ref/bar, and I add a reference or inherit to /foo that points to /ref, then the opinions on /ref get composed on top of /foo. The opinions on /ref/bar get composed on top of /foo/bar. The whole referenced tree get layered over top of the referencing primitive and it's descendants. That's why adding an inherit to the xforms adds the material binding from /class/mesh to xforms/mesh, which is where you want the material binding to go (to replace the binding that's already there). The inherit arc needs to be added to the xforms because the mesh prims themselves are instance proxies, and therefore uneditable. If the mesh prims were not instance proxies, you could have added the inherit arc to the meshes themselves, and the material binding could have been authored directly on the target of the inherit (rather than on a mesh child prim).

mtucker
I'm not seeing anything like that. I also don't have any problem when I take a snapshot. Does the error message say where it is trying to save the file? Is there any reason why Houdini might not be able to write to that location?

Thanks @mtucker! The message does not specify it unfortunately and the path is a valid one. . I add a screenshot.


I have another question.
THIS [we.tl] is the updated file right now.

I made displacement work with the workflow. If I switch to XPU using either the Material X or the USD preview MTR all the BaseColor render white, (but the displacement works).

Any idea of what I might be doing wrong?

Oh, I bet the issue with not being able to save the image is that your viewport resolution (or the resolution of the image being generated by karma to show in the viewport) is larger than the output resolution allowed by an apprentice license?

As for XPU, it is Alpha software, so lots of things aren't going to work. I don't know the state of XPU with displacements, but if you can make a simple test case (something smaller than a whole village) and submit it as a bug I'm sure the XPU team would approeciate it.

mtucker
Oh, I bet the issue with not being able to save the image is that your viewport resolution (or the resolution of the image being generated by karma to show in the viewport) is larger than the output resolution allowed by an apprentice license?

GENIUS! Thanks you are 100% right. It works now that I shrunk the viewport

mtucker
As for XPU, it is Alpha software, so lots of things aren't going to work. I don't know the state of XPU with displacements, but

I do not think that the problem is the displacement actually. That is the one thing that seems to be working with XPU in the scene (if you see in the last image it renders the height correctly). The problem is everything else (base color, normal etc..).
If I remove the displacement the renders stay white with XPU.
I think the problem might be with the inherit instead: You see, if I disable the inherit (ergo going to the default), it renders correctly all the textures.

mtucker
if you can make a simple test case (something smaller than a whole village) and submit it as a bug I'm sure the XPU team would approeciate it.

How can I submit a bug?

PaoloGiandoso
What I would like to do is to convert all of these USDPreviewSurface materials to 1 single Principled Shader, where each texture path is variated by Primitive Mesh.

Two things XPU is currently not doing, which is why you see grey

1) per-instance properties
2) on demand texture loading (ie texture path being resolved in the shader)

links
https://www.sidefx.com/docs/houdini/solaris/karma_xpu.html#features [www.sidefx.com]
https://www.sidefx.com/forum/topic/82989/ [www.sidefx.com]

apologies its not working right now, but this stuff will come with time
Cheers

brians

PaoloGiandoso
What I would like to do is to convert all of these USDPreviewSurface materials to 1 single Principled Shader, where each texture path is variated by Primitive Mesh.

Two things XPU is currently not doing, which is why you see grey

1) per-instance properties
2) on demand texture loading (ie texture path being resolved in the shader)

links
https://www.sidefx.com/docs/houdini/solaris/karma_xpu.html#features [www.sidefx.com]
https://www.sidefx.com/forum/topic/82989/ [www.sidefx.com]

apologies its not working right now, but this stuff will come with time
Cheers

Hi Brian
no need to apologize. It's in alpha after all.
But good to know you guys are aware of it
What puzzles me is that the Displacement works, meaning SOME per instance on demand texture loading (the heightmap) is actually working. Otherwise the model would be completely white and also without Displacement.

PaoloGiandoso
What puzzles me is that the Displacement works, meaning SOME per instance on demand texture loading (the heightmap) is actually working. Otherwise the model would be completely white and also without Displacement.

Displacement is currently done on the CPU using regular Karma

brians
Displacement is currently done on the CPU using regular Karma

Got it, thanks!