Drew Whitehouse

drew

About Me

Expertise
Not Specified
Location
Australia
Website

SciViz programmer, have been a SideFX user since Prisms. Minor claim to fame: I wrote the original Houdini Ocean Toolkit which has now been ported to pretty much every 3D platform under the sun. Thankfully Sidefx have released much better Ocean tools for Houdini since then :-)

Connect

Recent Forum Posts

Houdini 18 HQueue Error Dec. 5, 2019, 6:34 p.m.

Grrrr….

After spending a few days investigating what was going on with H18 HQueue and submitting bug reports detailing how hqnode.ini files were being overwritten on startup, it's really frustrating to find out this has been moved into the web server. Why was this done? Why was nothing put into the configuration files mentioning the fact that they were now were being ignored. Nothing in the Whats New about HQueue changes.

I've spent a good deal of time building automated tools for:

- spinning up new linux VMs
- rolling out new versions of HQueue, keeping versions and configurations up to date
- installing hqnode clients that start on machine reboot, and restart on errors
- running up the web server on boot, _not_ as the root user
- creating a hquser on all the machines with the same uid/gid that allows a team to use HQueue without having the network mount completely open from a security standpoint
- not requiring someone to put passwords in for every client they add, frankly it's scary that the web server wants to ssh into machines and do stuff behind my back, and doesn't scale to a large number of nodes

So the new changes just seem to be going in the opposite direction. It's obviously not testable at the moment as evidenced by the fact that it was completely broken on day one of the H18 release. Philosophically I just don't think that HQueue should be developed in this direction, it seems to me un-Houdini like. When administering multiple machines there are much better tools for system orchestration than buggy web servers holding user passwords. Configuration should be in one place, clearly documented, updateable, and automatable. At least give system admins a chance to do things the right way. Sorry for the rant but I had to get this off my chest!

BTW sometimes we have to nuke the server and database and start from scratch due to large render jobs being nearly impossible to delete. The server goes into 100% cpu and hours go by without a successful delete operation. I'm assuming that would mean putting the network shares back into the database by hand through the web GUI?

Maybe I'm expecting too much of HQueue? It's been a really useful, albeit fragile, tool that has rendered hundreds of thousands of frames for my small group over the years. We're hardware rich, but software poor so it has filled a gap when we couldn't afford Deadline, Tractor etc. But maybe now we should be focusing on developing a custom PDG scheduler for task distribution in our environment, something that I'm seriously considering?

-Drew

render farm linux houdini 17.5 Dec. 5, 2019, 4:21 p.m.

So i suggest you

> mkdir /opt/hqueue/shared/houdini_distros

Then install a full distro of linux houdini into this, ie something like the following

> tar xvfz houdini-17.5.360-linux_x86_64_gcc6.3.tar.gz
> cd houdini-17.5.360-linux_x86_64_gcc6.3
> . houdini.install # note the space after the period here

You shouldn't need to be the root user to do this.

… then from the menus you're going to select option D and change the installation directory to be something like /opt/hqueue/shared/houdini_distros/hfs17.5.360 at which point go ahead and install.

Now

> cd /opt/hqueue/shared/houdini_distros
> ln -sf hfs17.5.360 hfs.linux-x86_64

Then you should be on your way.

render farm linux houdini 17.5 Dec. 4, 2019, 7:01 p.m.

On your linux machine, do this and tell us what you see.

> ls /opt/hqueue/shared

Also,

> ls /opt/hqueue/shared/houdini_distros