I'm a bit out of my league here with technical knowledge but in a situation where the developer doesn't know enough Houdini to effectively pinpoint issues, whereas I don't know enough networking terminology to really understand the flow. That said here goes:

I'm looking for the best practice to set the same port. The developer would like the same port to be used both locally and on the remote task. Says he's getting a different remote port each time the task is run. Is there a way to "stabilize" the port being used. For instance, always set the port to, localhost:54411

I've tried the environmental edit to no avail and not exactly sure what to set that to or if it's even the correct node.

I'm using this to try and ascertain the current port being used,
in the python scheduler

Could someone point me in the right direction or where to look into such issues, thanks!

Here's the exact issue we're experiencing in the words of the developer. Hopefully it's something simple!

I’m trying to set the address of the PDG RPC server in such a way that it is predictable every time, so that the firewall rules for my cloud worker nodes can be limited and secure by only opening a single port for RPC communication.
I’ve tried setting the PDG_RESULT_SERVER directly in my master node’s environmental variables, but still I can observe that the result of str(self.workItemResultServerAddr()) changes each time I setup and run Houdini on the master. I’d prefer to always set it to something like localhost:54411 every time it runs.
How would I ensure that the PDG RPC server always runs on the same port by default? Also, is there a way to change this per job or per execution run?

@tpetrick or @BrandonA -- are either of you able to point us in the right direction here?

This has become a blocker for us to continue with our development so would really appreciate any help you can provide.

Thanks!

It currently isn't possible to customize the address of the internal result server used by PDG, unless you write a custom scheduler that constructs its own RPC endpoint at a fixed port. However, when submitting jobs to the farm the PDG MQ service should make it possible to avoid firewall issues. Every submission using one of the farm schedulers starts an MQ service job that the PDG graph connects to when the cook begins. Work items running as part of the cook also connect back to the same MQ to send results to PDG, which means that the work items don't need to directly communicate with the result server at any point in time. Each of the farm schedulers (Tractor, Deadline and HQueue) expose parameters to customize the ports used by the MQ service.

In the future, for urgent questions like this one it's much better to contact SideFX support instead of using the forum. The forums are not directly tracked by support or QA, and developers check them at their own leisure. This sticky thread has more details about submitting bugs and RFEs, which also applies to questions: https://www.sidefx.com/forum/topic/61697/ [www.sidefx.com]

The developer is currently writing a custom python scheduler - any chance you could point us to some documentation on how we might integrate a custom RPC endpoint with the rest of PDG so that it can handle the scheduling/result RPC communication?

We are not using any of the farm schedulers - we are running our jobs as child Hython processes on remote servers, setting PDG parameters by passing environmental variables, and ensuring that the master and all workers use the same shared network drive that is mounted in an accessible place by all.

We also noticed that, although we're not using it, the HQueue scheduler has some functionality to provide a custom TCP port to use as the result server. Is it feasible to add this feature to the base Python scheduler?

andrewlowell
We also noticed that, although we're not using it, the HQueue scheduler has some functionality to provide a custom TCP port to use as the result server. Is it feasible to add this feature to the base Python scheduler?

Which parameters are you referring to? As far as I know, the parameters for configuring ports on the HQ scheduler are for customizing the ports used by the message queue service.

andrewlowell
The developer is currently writing a custom python scheduler - any chance you could point us to some documentation on how we might integrate a custom RPC endpoint with the rest of PDG so that it can handle the scheduling/result RPC communication?

There isn't any documentation on that since the RPC server is mostly an internal implementation detail. However, since all of the schedulers are written in Python, the code for the schedulers is still available as part of a Houdini installation. All schedulers subclass from the same two classes -- the pdg.Scheduler.PyScheduler base class, and the CallbackServerMixin mix-in class that defines the RPC server logic. For example, the Local Scheduler defined in $HH/pdg/types/schedulers/local.py. The callback server mix-in itself is defined in $HH/pythonX.Ylibs/pdg/job/callbackserver.py.

The documentation for writing a custom schedulers includes examples that are derived from the Local Scheduler: https://www.sidefx.com/docs/houdini/tops/custom_scheduler.html [www.sidefx.com]

In order to change the RPC server construction you'd need to write one from scratch that uses your own implementation of CallbackServerMixin that constructs the RPC server in the way that you want.