I am unclear about how to best do complex shading and materials in Houdini.

Sometimes shading and materials can be assigned to polygon groups, but there are situations where the shader and textures have to cross polygons. Of course one can add loop cuts or clip nodes and try to create polygons that match the textures, but that is sometimes not a good solution. I am not sure the best method for handing this. In Modo, it is simple, since shading and texturing is layer based, unlike any other 3D or rendering program (excepting 3Dcoat) that I know of.

The methods I am aware of for Houdini are:
1. Creating a series of textures in Photoshop for each channel (diffuse, metallic, roughness, etc.) using Photoshop's layers, modes, masks, and opacities. This works if the surface is flat, but doesn't work for normal maps or when one one wants to mask based on curvature or normals.

2. Exporting a FBX from Houdini with generic principled shaders and creating the various textures in Mari or Substance painter, where there is better visualization of the effect on the model and the ability to create masks based on curvature or normal direction. Back in Houdini, load these textures in the appropriate channels. This is what I've generally done. It is pretty easy, but one can't visualize directly the look in the rendering engine used in Houdini when working in Mari or Substance.

3. Try to set up nodes directly in the material network of Houdini based on the rendering engine used (Mantra, Arnold, Redshift, 3Delight, Octane, etc) and attempt to simulate layer modes with multiplications, additions, etc of nodes in Houdini. This has the advantage of allowing adjustment directly in Houdini, but gets really complex if one is using masks, layer modes, different opacities, etc. See second attachment for what I mean.

4. Dive into the principled shader itself (as Varomix did to add a bind based on an attribute for compositing after render). This is really complicated with all the nodes.

5. Trying to use a material builder or layer shader. I don't know when these are helpful, so I haven't used these.

An example what I am trying to do issomething like this (grey lines are polygon borders):

Island
5. Trying to use a material mixer or layer shader. I don't know when these are helpful, so I haven't used these.

that's the easiest way to mix whole materials (including normals and displacement) using simple masks procedural or not
so you should have probably started with this

Thank you. The layer mix helped reduce node clutter. When is the material builder used?

After trying the layer mix (and even collapse selected into material to create a material builder at the end), there are a few problems I'm having with this workflow. It does indeed work fine to reduce complexity. However, it does not show up when one selects "display material on objects" of the scene viewport. It requires dragging and dropping from the material network onto the object, as it neither shows up as an option on a material node or as an options on the objects render material properties. The former limits use to situations where the object has only one material (so one can't use material groups). Theoretically one could use Houdini's procedural features to create masks, but the render viewport is slow anyway (and has the annoying property of not displaying the right brightness when sets the gamma in the mantra render node to something other than 1 (for jpg or png outputs). mPlay doesn't have that deficiency but is even slower than the render viewport. So all that brings me back to using Substance painter or Mari even though one will not get an exact match of Houdini's render. They have fast display when one is adding masks and materials and have the advantage of layer mixing modes which have be be set up with several complicated math modes in Houdini. Am I missing something?

Island
as it neither shows up as an option on a material node or as an options on the objects render material properties

make sure the orange Material flag is on any VOPs you want to use as materials otherwise they will be filtered out in those lists as not materials to reduce clutter

Island
and have the advantage of layer mixing modes which have be be set up with several complicated math modes in Houdini

not for full layers, but for color data like textures you can use Composite VOP to get access to blending modes

The orange Material flag doesn't seem to matter. Here is a simple layer mix of a principled and cloth shader combined with an alpha mask, and you can see that the options are only for the principled or cloth shader, not for the layer mix. In my example, I used turbulent noise as the alpha, but generally in projects, I use a texture alpha image instead.

The composite VOP looks interesting, but a quick try of the various modes show that it must be using a different color space than the usual other programs (Photoshop, etc.). Maybe it is because it is linear, but A over B with a white rectangle and transparency saved as a png is not creating a good alpha mask.