Robert Vinluan

rangi
Hi,

I'm rendering on a second workstation using hqueue. Submitting from the stage with a fetch rop grabbing the "USD Render Rop", which is then wired in to an hqueue rop.

The behaviour I'm seeing is that it creates the job with the first task generating the subsequent tasks which are the frame batches.

I'm used to a workflow where that would be generating ifds, then the subsequent tasks are just mantra. Quick loads, only requires render license.

What I see is those tasks for rending the frames are just pointing back to the original hip. Fine - means it uses the license but it's a one-node farm so who cares.

However - the first tasks creating those frame tasks is incredibly slow - tying up the machine for perhaps half an hour. Really significant. Since I'm not waiting for it to generate USDs, what is it that is taking the time here?

( I know the proper response is to write submission and wrapper scripts - which I've not done for hqueue and wanted to avoid doing. This is the out-of-the-box experience which I was hoping would suffice for my one man show )

Thanks in advance for any tips.

r.

Hello,

Would you be able to post your original .hip file where you encountered these issues?

The first task that the HQueue Render ROP submits to HQueue should be labelled as "Generate USDs". The task should generate .usd files and also submit followup render tasks that render directly from the generated .usd files. The render tasks should only take up Karma licenses instead of Houdini licenses (if rendering with Karma).

Note though that the "Generate USDs" workflow wasn't added until Houdini 20.0 so if you are using Houdini 19.5 then that would explain why your tasks are rendering directly from the .hip file.

However, if you are using Houdini 20.0, then the tasks should generate and render from .usd files by default. Check that the USD Render Options -> USD Output Filecheckbox parameter on the HQueue Render ROP is checked on.

The other possibility is that the Fetch ROP is confusing the HQueue Render ROP causing the "Generate USDs" workflow to be skipped over entirely.

As to why the first task is slow when not generating .usd files, it's hard for me to say. However, the first task needs to evaluate a few parameters in order to gather information that's stored in the followup render tasks. Perhaps evaluating one of the node parameters triggers node cooking or some other heavy operation?

Cheers,
Rob

QuentinRoux
_The webpage don't have a login user system and the full control is available, does something exist to fix that ?
_How to allow only some Houdini users to send jobs on the farm ? (not to have all the students to send jobs).

Hello,

HQueue is currently completely open for good and bad. Anyone that can connect to the HQueue Server can submit jobs to the farm. User authentication is something that's been on the roadmap for a while but has not been addressed.

QuentinRoux
_And the IT from the school mentioned he had to open all the ports to make the pcs slaves to be visible on the master pc, what are the port we need to open excepted 5000 and 5001 ?

The HQueue Server listens on port 5000 by default. The HQueue Client attempts to listen on port 5001 by default but if 5001 is reserved by another process, then the HQueue Client tries 5002, then 5003, then 5004, etc. until it finds an available port.

Note that you can specify what port the HQueue Server listens on by setting the portkey in the server config file, hqserver.ini. And you can specify the HQueue Client listen port by adding the --listenportoption to the command that launches the HQueue Client. Where you add the option exactly depends on the client machine's operating system.

QuetinRoux
_Any resources about Hqueue who are more oriented production and not personal ?

I don't know of any resources but I can say that those who use HQueue in production and are concerned about security typically install HQueue on a segmented portion of their network and machine farm. They limit access and control to the smaller HQueue farm through the use of network firewalls, restricted network file systems, and sometimes with locked down VMs. Basically, they build security at the system admin and network level and not with HQueue itself.

QuetinRoux
_ I saw a GitHub about sending other jobs like Maya stuff and nuke renders to Hqueue, any resources for that ?

You can have a look at the HQueue Python API -- https://www.sidefx.com/docs/houdini/hqueue/api.html [www.sidefx.com] . You can use the API to submit custom jobs to the farm.

And also have a look at the HQueue docs that talks about job specifications -- https://www.sidefx.com/docs/houdini/hqueue/jobdetails.html [www.sidefx.com] . The docs explain how to build up a custom (JSON) job specification that can be submitted to HQueue. At its core, a job simply contains a series of (shell) commands that are to be executed by the machine.

QuetinRoux
And in the end is Hqueue a good solution ? I would like to keep things simple like what Hqueue give but with that security layer to make it more safe for a school usage.

What do you guys think !

HQueue is a good solution for some cases -- small farm usage, lightweight installation, etc., however, if security is a priority then you may want to look at other solutions like AWS Thinkbox Deadline as eguquansuggested, which has security features available out of the box.

I hope this helps.

Cheers,
Rob

Hello,

Windows Server has never been supported. It's even mentioned in the Houdini 19.0 systems requirements:
https://www.sidefx.com/Support/system-requirements/19.0/ [www.sidefx.com]

We don't support Windows Server because we don't actively test on that OS. Some users still run Houdini on Windows Server but they do so at their own risk. If they encounter a bug that's specific to Windows Server (i.e. it can't be reproduced on Windows Pro), then there's no guarantee that they will receive a fix for it.

Cheers,
Rob