Hi there,

My TOPnet is running on our renderfarm with a couple of clients using HQueue.

On the clients I want to use the Perforce TOP to sync certain files. The files differ per workitem.
Those synced files are the input for an HDA Processor. The files are actually textures and set as parameters on the HDA by using workitem attributes so they don't use the PDG input and output files, if you know what I mean.
The problem is that I want to run the Perforce sync on the same client as the HDA Processor because else the input files might not be there or not the latest version.

So I'm looking for a way to sort of group the Perforce and HDA Processor TOPs together in a way they are executed on the same client.
Or maybe get some info from the workitem coming from the Perforce TOP and figure out which client was used. And then wrangle the workitems before they go into the HDA Processor to overwrite the job parameters to only execute on that specific client.
But I guess there might be a case where TOP nodes after the HDA processor would want the work items to run on any of the available clients again.

The problem is more general than just this case of Perforce and HDA Processor.
This is my first PDG network so I could be missing something obvious

cheers,
benS

There's no special mechanism to set the farm client for work items, other than the scheduler job parms.

You could set an expression on the hqueue_clients parm which picks up an attribute from the upstream item, the attribute could be created by querying the client name with the hqueue xmprc api from the completed job (using @hqueue_jobid).

Another way would be to create a little topnet that uses localscheduler and has a topfetchinput->perforce->hdaprocessor chain and then use a topfetch node to fetch it. Since it uses localscheduler it should all happen on the one client.

Hi Chris,

Yes I realize now that there is not such a mechanism because:
A: it would be HQueue specific and not work for other shedulers
B: renderfarms seem to be made with share drives in mind.

I've tried the second option this morning and believe I saw it working once.
But I can't reproduce that case again.
It is nice to see the TOP nodes, that should run on a single client, in isolation in their own network.
The downside is that debugging the nodes can be tricky in a similar to debugging of Invoke networks because no workitem/data is flowing through them.

I'm now investigating the first option.
I've got both Perforce and HDA tops set up to use the HQueue server and trying to limit the hqueue_clients of the second one to the client of the first one. But unfortunately doesn't work yet.
I've tried the following expression in the hqueue_clients parameter (under Override Job Parameters):
And I've managed creating in attribute using simular code and set the hqueue_clients parameter to @upstream_client.
Both the pythonprocessor and the hda processor are set to Dynamic as I thought that would make sense.
But no success yet. The hda processor is executed on a random client.

Any further leads?
cheers,
benS

ben5ch
I've tried the second option this morning and believe I saw it working once.
But I can't reproduce that case again.
It is nice to see the TOP nodes, that should run on a single client, in isolation in their own network.
The downside is that debugging the nodes can be tricky in a similar to debugging of Invoke networks because no workitem/data is flowing through them.

The topfetchinput node will read in the topfetch work item and the topfetch will read back the workitem of the output node of the fetched network, so there is data flowing through the network as normal. However it's pretty hard to debug because as you say it's a separate process. There should be log files generated you could look through for hints.

The hda processor is executed on a random client.

Any further leads?

You could look at the properties link on the hqueue job page to see if the client setting is actually coming through, maybe there's something not right with the expression.

Is it not possible to setup the Perforce node to use git, rather then Perforce ?

The perforce node only supports repositories that are compatible with the p4 tool. Please submit an RFE if you would like to see a Git TOP node.