Hello Everybody,

just bought the Indie Version of Houdini last week and I am trying to get Hqueue up and running but I don`t have any success.

For now it just on my workstation where I installed the whole deal.
Main Installer from License Server to Main App to Hqueue Server and Client.
I thought I want to start small and just set my workstation up as server and client to submit render jobs which can be render in the background like a simple backburner setup.

I got the webinterface working and i also see my workstation as a client but after submission it fails rendering.
My guess is that the Variables are set wrong…

I got the Shared Network folder working and mounted it on a H: Drive which in my case might not even be necessary since I am rendering locally right?

I even can submit the job which I am able to see in the webinterface but after assigning the job I get an error and it switches instantaneously to “failed”

Somehow in Houdini I get the $HQROOT and $HQCLIENTARCH Variables marked in red so I am guessing that there is something wrong with the Variables but I cant get it working…

Houdini is installed on the Basic Path:
C:\Program Files\Side Effects Software\Houdini 14.0.249
Server is also on the Path suggested by the installer
C:\HQueueServer

# The shared network.
hqserver.sharedNetwork.host = 192.168.1.xxx
hqserver.sharedNetwork.path.windows = \\HQueueServer\shared
hqserver.sharedNetwork.mount.windows = H:

Can U guys give me a hint what I am doing wrong here?
I did some Pipeline work integrating C4d and After Effects in Backburner but I am not a guru….

Hqueue makes me feel like I am stupid.

Sidefx wrote on the Houdini Indie Page "With the ability to set up to 6 machines , you can quickly set up a small farm."

It absolutely not quick and I get the feeling that You really have to be a trained TD to get anything done at all.

It can be mighty frustrating, I totally get where you're coming from! I've had a little experience, and got it to work using two machines, but here is what made it easier for me:

- I used Linux, and I used the same version of Linux on all machines (Linux Mint)
- I've forgotten the name of the package that lets you setup shared dirs through the installer (I think it tells you when it's running, so you can open another shell and apt-get install it, then proceed with installation)
- Since they're all the same OS, and have houdini installed in the same location, I changed the target hfs to /opt/hfs14.0.249, for example
- I cannot stress enough that hqueue is far easier to maintain/run if the machines are all the same OS (for me)!

What exactly is the error message you're seeing on your job? Mine were usually related to permissions; because the client is being run under another user (hquser by default), your folder permissions have to allow for it.

I wish I could remember the package name (I'll post it if I remember/find it), as I'm sure the installer tells you what it is when installing the server, but that was the only thing I had to do manually when getting setup under linux. What OS are you using? Windows I assume?

*EDIT* The tool for mounting the shared hq folder is nfs-utils, but on Debian/Ubuntu/Linux Mint, you install it via sudo apt-get install nfs-common!

dani2055
Houdini is installed on the Basic Path:
C:\Program Files\Side Effects Software\Houdini 14.0.249
Server is also on the Path suggested by the installer
C:\HQueueServer

# The shared network.
hqserver.sharedNetwork.host = 192.168.1.xxx
hqserver.sharedNetwork.path.windows = \\HQueueServer\shared
hqserver.sharedNetwork.mount.windows = H:

Hello!

Your setup looks pretty good. My only suggestion is to take out the “\\HQueueServer\” part of the hqserver.sharedNetwork.path.windows entry. HQueue already knows about the server IP address from the sharedNetwork.host line and uses that to build the final shared network folder path.

So set:

hqserver.sharedNetwork.path.windows = shared


Be sure to restart the HQueue Server service after making the change.

As for $HQROOT and $HQCLIENTARCH, these variables are not recognized by Houdini because they do not exist locally. They become set when the job runs on the client machines.

For example, when a job runs on a Windows client then:
- $HQROOT evaluates to “\\192.168.1.xxx\shared” (for your farm setup)
- and $HQCLIENTARCH evaluates to “windows-x86_64”

These variables allow HQueue to find Houdini (via the Target HFS parameter) and other network files in an OS-independent way.

But if your farm consists of only Windows machines and if Houdini is installed locally on all of the machines, then I suggest keeping things simple and setting the Target HFS parameter to “CProgram Files/Side Effects Software/Houdini 14.0.249”.

I hope this helps.

Let us know if this fixes the failed job.

Cheers,
Rob

Sorry, for my late response, I got caught up at work.
I am still trying to get HQ to run and I actually finally got it to run at least on my local machine.


@goldleaf
Thank You for Your kind words!
I am running all machines on Windows 7. I got HQ running on my local machine but it will only work if i copy the Project Files to the shared Folder and start the HQ ROP from there. Also this will only work for my local machine. As soon as I ad a client I will get an error which says for example:

“ERROR: Cannot open file KHoudini/network_test/network_test_02.hiplc”

This Error must be caused by the Permissions I guess but I can`t figure out how to fix this. I read a lot spreaded all throughout this form about chancel the HQClient Service and start hqclientd.bat instead if you run in permission errors but somehow I am not able to start the service manually.
Cmd Console will flicker up a second but will shutdown immediately without starting the client.


@rvinluan
Thank You for Your tips!

I tried changing the ini the way You described but I could not get it to work.
I played around a little more by myself and now the setup looks like following:

hqserver.sharedNetwork.host = 192.168.1.xxx
hqserver.sharedNetwork.path.windows = \\dani\hq
hqserver.sharedNetwork.mount.windows = H:

I have a created the shared Network Folder like it was suggested by the installer “C:\HQueueServer\shared”
and granted all Users full Access to it.

This Setup let`s me add Jobs to the HQ and render locally but only if I put my Project files to “C:\HQueueServer\shared\projects” which is equal to
“H:\projects” since the Network drive is mounted on H:

As soon as I add A client I will get an Error in HQ saying:
ERROR: Cannot open file //dani/hq/projects/t01/network_test_localOK2.hiplc

This has to mean that there is something wrong with the permissions I guess.
I double check my permissions on my client machine and everything seems to work out fine. I can access the shared Folder, it`s mounted correctly.

I am searching all over the Internet on some Info about this issue but literally just find bits and pieces from people either ditching HQ and go with Deadline which is not my intention or just gave up on it.

Luke Letellier said in this post:
https://www.sidefx.com/index.php?option=com_forum&Itemid=172&page=viewtopic&p=132653&highlight= [sidefx.com]

If you see an ‘Access is denied’ error early on in the job output then the HQueue Client could not access files on the shared folder. To fix this try changing the user account running the HQueue Client service to the Windows user account that you are logged in with.

Open the Windows Services window (Control Panel -> Administrative Tools -> Services) and locate the HQueue Client service in the list. RMB-click the service and choose Properties.

In the Properties dialog select the Log On tab and then select the “This account:” radio button. Change the user account and password to your Windows account and password. Click OK to save the changes and close the dialog.

Now RMB-click the HQueue Client service again and select Restart to restart the service.

If you still encounter an ‘Access is denied.’ error after this change then you can try the following workaround:

Run the HQueue client process outside of Windows services.

Stop the HQueue Client service (Control Panel -> Administrative Tools -> Services). Then start the HQueue Client process manually by opening the Windows File Explorer, navigating to C:\HQueueClient and double-clicking on the hqclientd.bat file.

Note that with this workaround you must manually start the HQueue Client every time you reboot the Windows machine.

You know what would be really really great? If somebody could make a Video Tutorial Step by Step setting up just 1 Server and 1 Client and get it up and running under Windows 7 environment with all trouble shooting.

I am even willing to pay for that!!

Best Regards to You all and thank You again!

Hey Dani,

On your HQueue output ROP, see the dropdown menu at the bottom of the panel where it says “Render Current HIP file”? Try changing that to “Render Target HIP File”. That will open up the “Target HIP” slot right below it, and use that to navigate directly to the project file that you're currently rendering. This might help.

Also - did you try out what you quoted from my post? It's a new addition to the Hqueue manual. Worked for me!

In regards to the tutorial: I really don't know that much! The help I did get was through Side Effects support, and while I learned enough to troubleshoot things with my particular setup , much of it doesn't directly apply 1:1 to your setup, which is quite different from mine.

– Luke

Hi Luke,

thank you for Your reply!
Yes I did try to your suggestion with target hip but I get the same “Cannot open file” ERROR message.

I also tried to change the user account and password in the Service Options but it also gives me an error telling me that this user doesn`t exist which I know for sure it does.

I also tried stopping the Service manually and executing the hqclient.bat which also gives me an error in the cmd console telling me
ERROR: Could not stop existing client
I already stopped the running client service by myself though so I don`t understand the cause of this error.

Do You have any other ideas about how to fix this “Cannot open file” issue?

Best Regards
Dani

*
Perhaps I should mention that I use Windows in a Japanese environment maybe that is the cause of the errors?

Guys I finally got it sorted out!
I am going to post a short “How to” later on this evening or the latest by tomorrow morning!
Thank You again!!
Dani

That's good to hear Dani!

I was going to make some suggestions about the server configuration and setup because I didn't think it was a permission problem. Typically permission problems on Windows appear as generic “Access is Denied” errors which as Luke mentioned can be solved by changing the Log On account on the HQueue Client service.

But anyway I could be wrong in this case. So I'll just wait to hear how you fixed the issue.

Cheers,
Rob

Glad to hear you got it sorted out!

Sweet! I too can't wait to see how you solved it!

please say how you fixed it, i have the same “cannot open file” problem

dani2055
Guys I finally got it sorted out!
I am going to post a short “How to” later on this evening or the latest by tomorrow morning!
Thank You again!!
Dani

Hi Dani. I have troubles running Hqueue render with 2 PCs too.
It would be awesome if you could explain what you did to make it work.

Any updates? I have hqueue working in the past on Win7 and Linux but for some reason I can't get it happening in Win10 at all..

The system cannot find the path specified..

from what I can see every path is fine. Houdini is installed in the same path on all PCs

# The shared network.
hqserver.sharedNetwork.host = 192.168.1.2
hqserver.sharedNetwork.path.linux = %(here)s/shared
hqserver.sharedNetwork.path.windows = design
hqserver.sharedNetwork.path.macosx = %(here)s/HQShared
hqserver.sharedNetwork.mount.linux = /mnt/hq
hqserver.sharedNetwork.mount.windows = Z:
hqserver.sharedNetwork.mount.macosx = /Volumes/HQShared

shared folder is located on 192.168.1.2\design and is mounted as z:

From HQueue logs i get this:

HQCOMMANDS { “hythonCommandsLinux”: “export HOUDINI_PYTHON_VERSION=2.7 && export HFS=\”CProgram Files/Side Effects Software/Houdini 15.0.244.16/bin\“ && cd $HFS && source ./houdini_setup && hython -u”, “pythonCommandsMacosx”: “export HFS=\”CProgram Files/Side Effects Software/Houdini 15.0.244.16/bin\“ && $HFS/Frameworks/Python.framework/Versions/2.7/bin/python”, “pythonCommandsLinux”: “export HFS=\”CProgram Files/Side Effects Software/Houdini 15.0.244.16/bin\“ && $HFS/python/bin/python2.7”, “pythonCommandsWindows”: “(set HFS=C:\\Program Files\\Side Effects Software\\Houdini 15.0.244.16\\bin) && \”!HFS!\\python27\\python2.7.exe\“”, “mantraCommandsLinux”: “export HFS=\”CProgram Files/Side Effects Software/Houdini 15.0.244.16/bin\“ && cd $HFS && source ./houdini_setup && $HFS/python/bin/python2.7 $HFS/houdini/scripts/hqueue/hq_mantra.py”, “mantraCommandsMacosx”: “export HFS=\”CProgram Files/Side Effects Software/Houdini 15.0.244.16/bin\“ && cd $HFS && source ./houdini_setup && $HFS/Frameworks/Python.framework/Versions/2.7/bin/python $HFS/houdini/scripts/hqueue/hq_mantra.py”, “hythonCommandsMacosx”: “export HOUDINI_PYTHON_VERSION=2.7 && export HFS=\”CProgram Files/Side Effects Software/Houdini 15.0.244.16/bin\“ && cd $HFS && source ./houdini_setup && hython -u”, “hythonCommandsWindows”: “(set HOUDINI_PYTHON_VERSION=2.7) && (set HFS=C:\\Program Files\\Side Effects Software\\Houdini 15.0.244.16\\bin) && (set PATH=C:\\Program Files\\Side Effects Software\\Houdini 15.0.244.16\\bin\\bin;!PATH!) && \”!HFS!\\bin\\hython\“ -u”, “mantraCommandsWindows”: “(set HFS=C:\\Program Files\\Side Effects Software\\Houdini 15.0.244.16\\bin) && \”!HFS!\\python27\\python2.7.exe\“ \”!HFS!\\houdini\\scripts\\hqueue\\hq_mantra.py\“” }


HQPARMS { “frames_per_job”: 1, “dirs_to_create”: , “project_name”: “Josh_Girls_001”, “hip_file”: “ZProjects/Houdini/Girls/Girls_001.hip”, “output_driver”: “/out/mantra_ipr”, “max_hosts_per_job”: 1, “min_hosts_per_job”: 1, “render_frame_order”: “linear_order”, “use_render_tracker”: 0, “render_single_tile”: false, “enable_checkpoints”: true }

Hi Josh,

Your hqserver.sharedNetwork settings look good.

One thing I noticed though is that your HFS path seems a little off. Try setting the Target HFS parameter to this instead:

CProgram Files/Side Effects Software/Houdini 15.0.244.16


Also, it seems strange that the hip_file setting in HQPARMS is pointing to Z: instead of the UNC path equivalent (i.e. //192.168.1.2/design). I suspect that is causing the error since the HQueue Client service can't access mapped network drive letters.

I would try restarting the HQueue Server service just to be sure it is picking up your latest hqserver.ini settings (which again look pretty good).

And if that doesn't work, then please post or PM me the diagnostics information for the failed job. This is new to HQueue 15.0 and appears on the web interface when viewing the job.

Cheers,
Rob

Actually I seem to have problems with hqueue recently as well. I have contacted support who have tried to help, but right now the only course of action I have is to uninstall and reinstall which I'm going to do later on this week when I can - but I don't see what has changed and am pulling what hair I have left out.

I have made some progress. some jobs will run, some will not..

The same job can run fine, then if I run it again, it will give an error.

Traceback (most recent call last):
File “<string>”, line 8, in <module>
File “CPROGRA~1/SIDEEF~1/HOUDIN~1.16/houdini/python2.7libs\hou.py”, line 28, in <module>
import _hou
ImportError: DLL load failed: The specified module could not be found.
Traceback (most recent call last):
File “C:\Program Files\Side Effects Software\Houdini 15.0.244.16\houdini\scripts\hqueue\hq_render_from_hip.py”, line 4, in <module>
hqlib.callFunctionWithHQParms(hqlib.renderFromHip)
File “C:\Program Files\Side Effects Software\Houdini 15.0.244.16\houdini\scripts\hqueue\hqlib.py”, line 1864, in callFunctionWithHQParms
return function(**kwargs)
File “C:\Program Files\Side Effects Software\Houdini 15.0.244.16\houdini\scripts\hqueue\hqlib.py”, line 969, in renderFromHip
import hou
File “CPROGRA~1/SIDEEF~1/HOUDIN~1.16/houdini/python2.7libs\hou.py”, line 28, in <module>
import _hou
ImportError: DLL load failed: The specified module could not be found.

I have attached the Diagnostics Information from a test job that worked once and then failed.

It also appears that HQROOT is not being set correctly when a job is submitted and run.

I have also tried to setup tiled render but only one machine seems to render a tile and it never renders correctly. After a while the job completely fails as all clients are blocked due to failing too many jobs.

I can run tiled renders and pretty much anything else fine with Deadline using exactly the same setup but I'd like to get Hqueue going properly for distributing sims.

job_73_diagnostic_information.txt is from a tiled render job that should have been run on Stalepopcorn but it fails and renders on the other client instead. Normal renders seem to run fine on it though

Hi Josh,

I looked at the diagnostics and just had a couple of notes/questions:
1) First off, I would leave Min. Clients per Frame and Max. Clients per Frame parameters on the HQueue Render ROP to their default values of 1. Those parameters were created before Mantra supported tiled rendering and were designed to handle the same problem that tiled rendering now solves.

2) $HQROOT looks fine to me for the most part though it looks like that variable is expanding out differently on your machines. On StalePopcorn, $HQROOT => \\192.168.1.2\design, while on Blackzilla, $HQROOT => \\Blackzilla\design. I'm not sure if Blackzilla and 192.168.1.2 are the same machine. If not, then you need to fix the sharedNetwork settings in hqserver.ini and hqnode.ini and then restart the HQueue Server and Client services.

3) If you set the Min. and Max. Client parameters back to 1, do you find that jobs fail for all machines on the farm? Or do they fail only on one of the machines on the farm?

4) The “DLL load failed” message is interesting. That almost appears as if that hython cannot startup. Out of curiosity, are you able to run Houdini manually (i.e. outside of HQueue) on each of the Windows machines? Are you able to launch ‘hython’ from the Houdini Command-line Tools terminal?

5) Also, by any chance is the PATH environment variable set on any of the machines? If so, then that could explain the DLL failure. As a test, try temporarily unsetting PATH and restart the HQueue Client service.

Cheers,
Rob

The PATH variable on Stalepopcorn actually does have Python2.7 in it as I need python for a few other things running on that particular PC. Could that be part of the problem.

I didn't realize I had even changed Min. and Max. Client. I did have some success when i changed them back to 1 though.

The PATH variable could be causing the DLL failure.

As a test, I would try temporarily unsetting PATH, restarting the HQueue Client service and then submitting a job.

If that works then you can restore PATH on your machine and just unset it for HQueue jobs. You can do that on your HQueue Render ROP (Advanced tab - > Environment). Add an entry mapping PATH to either blank or whatever your PATH is normally set to but without the Python parts.

I have it working for the most part. I was having issues with Computer names resolving to IP addresses for a bit so I changed everything to use IP instead of names.

I tried setting PATH to blank in the Hqueue node and everything still works ok with the 2 original clients, but when I tried to add my third client, I get the same DLL error on that machine. Hython runs fine on that machine, but nothing works using Hqueue.