render farm linux houdini 17.5

   3747   18   3
User Avatar
Member
141 posts
Joined: Sept. 2013
Offline
Hello ,

I managed to make a farm between my two linux machines, now I can start a job on the farm but I have another problem after two seconds my job fails.

I don't understand why?

A shared folder problem?

Please help i dont understand

”hythonCommandsLinux“: ”export HFS=\“$HQROOT/houdini_distros/hfs.$HQCLIENTARCH\” && cd $HFS && source ./houdini_setup && hython -u“,
”pythonCommandsMacosx“: ”export HFS=\“$HQROOT/houdini_distros/hfs.$HQCLIENTARCH\” && $HFS/Frameworks/Python.framework/Versions/2.7/bin/python“,
”pythonCommandsLinux“: ”export HFS=\“$HQROOT/houdini_distros/hfs.$HQCLIENTARCH\” && $HFS/python/bin/python2.7“,
”pythonCommandsWindows“: ”(set HFS=!HQROOT!\\houdini_distros\\hfs.!HQCLIENTARCH!) && (set PATH=!HQROOT!\\houdini_distros\\hfs.!HQCLIENTARCH!\\python27;!HQROOT!\\houdini_distros\\hfs.!HQCLIENTARCH!\\bin;!PATH!) && echo Accessing \“!HFS!\\python27\\python2.7.exe\” … && \“!HFS!\\python27\\python2.7.exe\”“,
”mantraCommandsLinux“: ”export HFS=\“$HQROOT/houdini_distros/hfs.$HQCLIENTARCH\” && cd $HFS && source ./houdini_setup && $HFS/python/bin/python2.7 $HFS/houdini/scripts/hqueue/hq_mantra.py“,
”mantraCommandsMacosx“: ”export HFS=\“$HQROOT/houdini_distros/hfs.$HQCLIENTARCH\” && cd $HFS && source ./houdini_setup && $HFS/Frameworks/Python.framework/Versions/2.7/bin/python $HFS/houdini/scripts/hqueue/hq_mantra.py“,
”hythonCommandsMacosx“: ”export HFS=\“$HQROOT/houdini_distros/hfs.$HQCLIENTARCH\” && cd $HFS && source ./houdini_setup && hython -u“,
”hythonCommandsWindows“: ”(set HFS=!HQROOT!\\houdini_distros\\hfs.!HQCLIENTARCH!) && (set PATH=!HQROOT!\\houdini_distros\\hfs.!HQCLIENTARCH!\\bin;!PATH!) && echo Accessing \“!HFS!\\bin\\hython\” … && \“!HFS!\\bin\\hython\” -u“,
”mantraCommandsWindows“: ”(set HFS=!HQROOT!\\houdini_distros\\hfs.!HQCLIENTARCH!) && (set PATH=!HQROOT!\\houdini_distros\\hfs.!HQCLIENTARCH!\\python27;!HQROOT!\\houdini_distros\\hfs.!HQCLIENTARCH!\\bin;!PATH!) && echo Accessing \“!HFS!\\python27\\python2.7.exe\” … && \“!HFS!\\python27\\python2.7.exe\” \“!HFS!\\houdini\\scripts\\hqueue\\hq_mantra.py\”“
}”

Attachments:
Capture d’écran de 2019-12-03 16-03-28.png (38.0 KB)
Capture d’écran de 2019-12-03 16-04-11.png (56.6 KB)
Capture d’écran de 2019-12-03 16-08-14.png (83.6 KB)

Sorry for my english but i learn too ^^

1 ryzen 3970X 64go
1 xeon hexacore 32go
1 i7 hexacore 16o
User Avatar
Member
141 posts
Joined: Sept. 2013
Offline
someone help please ?
Sorry for my english but i learn too ^^

1 ryzen 3970X 64go
1 xeon hexacore 32go
1 i7 hexacore 16o
User Avatar
Member
117 posts
Joined: July 2005
Offline
Drill down into that failed job entry (click on the “1” in the Job Id column) and see what errors are being thrown.
User Avatar
Member
141 posts
Joined: Sept. 2013
Offline
I have this message “ /bin/bash: ligne 0 : cd: /opt/hqueue/shared/houdini_distros/hfs.linux-x86_64: Aucun fichier ou dossier de ce type


I think is a shared folder problem ?
Edited by bgood - Dec. 4, 2019 14:12:31
Sorry for my english but i learn too ^^

1 ryzen 3970X 64go
1 xeon hexacore 32go
1 i7 hexacore 16o
User Avatar
Member
117 posts
Joined: July 2005
Offline
Do you have a houdini distribution installed in /opt/hqueue/shared/houdini_distros/hfs.linux-x86_64 ?

I'm assuming you have /opt/hqueue/shared as your hqserver.sharedNetwork.path.linux in hqserver.ini.

I typically install houdini into the shared directory /opt/hqueue/shared/houdini_distros, then symbolic link to the one I want to be used by default.

So assuming you've installed the latest distro into /opt/hqueue/shared/houdini_distros/hfs17.5.425

> cd /opt/hqueue/shared/houdini_distros
> ln -sf hfs17.5.425 hfs.linux-x86_64


-Drew



bgood
I have this message “ /bin/bash: ligne 0 : cd: /opt/hqueue/shared/houdini_distros/hfs.linux-x86_64: Aucun fichier ou dossier de ce type


I think is a shared folder problem ?
Edited by drew - Dec. 4, 2019 15:59:49
User Avatar
Member
141 posts
Joined: Sept. 2013
Offline
Under windows when I install the server it does it to me alone why under linux it did not do it to me?

I have to reinstall houdini in this directory or just the server I don't understand?

if I understood correctly, give him the houdini installation directory but when I go to this directory I don't find “hfs.linux-x86_64”
Edited by bgood - Dec. 4, 2019 18:12:31
Sorry for my english but i learn too ^^

1 ryzen 3970X 64go
1 xeon hexacore 32go
1 i7 hexacore 16o
User Avatar
Member
117 posts
Joined: July 2005
Offline
These things happen …

1. The Hqueue server is installed somewhere, either windows or linux. The server does not need to mount the shared filesystem.

2. The Hqueue clients are installed on the machines you want doing the work, windows and/or linux. The clients _must_ all be able to see the shared filesystem. Note: this means if want your server machine to also do work, you'll need a Hqueue client installed on it as well and hence the shared network mount to be present. I'm guessing from what you've desrcribed this is what you'll be doing.

3. A full Houdini distro must be installed on all the clients machines, and the easiest way to do this is to install it once onto the shared filesystem that is accessible to all. ie into /opt/hqueue/shared/houdini_distros/ directory like I detailed above.

You may also eventually need a Windows distro installed on that shared mount as well, but I'm not a Windows user so others will need to jump in here for the detalis. While you're getting things set up I'd just have linux be the client. One you get that running proprely you can think about adding Windows as a client as well.


Hopefully this is a bit clearer. BTW I don't recommend going to H18 yet.

-D



bgood
Under windows when I install the server it does it to me alone why under linux it did not do it to me?

I have to reinstall houdini in this directory or just the server I don't understand?
Edited by drew - Dec. 4, 2019 18:23:45
User Avatar
Member
141 posts
Joined: Sept. 2013
Offline
So yes it's clearer for the 18th I'm aware of the problems.

So you advise me to install houdini in my shared directory?

The problem is that once installed I don't have this famous “hfs.linux-x86_64” file

This is where I block

I would like my share directory to be in my nose I tried to install houdini on it but it didn't work.

On the other hand if I do all the installation of the server from windows by itself it installs houdini in the sharing directory and I can see the famous file in the distros directory.

Why when I do it on Linux it doesn't do it?

Translated with www.DeepL.com/Translator (free version)
Sorry for my english but i learn too ^^

1 ryzen 3970X 64go
1 xeon hexacore 32go
1 i7 hexacore 16o
User Avatar
Member
117 posts
Joined: July 2005
Offline
On your linux machine, do this and tell us what you see.

> ls /opt/hqueue/shared

Also,

> ls /opt/hqueue/shared/houdini_distros
User Avatar
Member
141 posts
Joined: Sept. 2013
Offline
Hello i have this

Attachments:
Capture d’écran de 2019-12-05 11-11-56.png (26.2 KB)

Sorry for my english but i learn too ^^

1 ryzen 3970X 64go
1 xeon hexacore 32go
1 i7 hexacore 16o
User Avatar
Member
117 posts
Joined: July 2005
Offline
So i suggest you

> mkdir /opt/hqueue/shared/houdini_distros

Then install a full distro of linux houdini into this, ie something like the following

> tar xvfz houdini-17.5.360-linux_x86_64_gcc6.3.tar.gz
> cd houdini-17.5.360-linux_x86_64_gcc6.3
> . houdini.install # note the space after the period here

You shouldn't need to be the root user to do this.

… then from the menus you're going to select option D and change the installation directory to be something like /opt/hqueue/shared/houdini_distros/hfs17.5.360 at which point go ahead and install.

Now

> cd /opt/hqueue/shared/houdini_distros
> ln -sf hfs17.5.360 hfs.linux-x86_64

Then you should be on your way.
Edited by drew - Dec. 5, 2019 18:53:30
User Avatar
Member
141 posts
Joined: Sept. 2013
Offline
Well now nothing works anymore the server doesn't work anymore. I'm going to drop it thanks for your help

Side fx = why make it simple when you can make it complicated

very frustrating
Sorry for my english but i learn too ^^

1 ryzen 3970X 64go
1 xeon hexacore 32go
1 i7 hexacore 16o
User Avatar
Member
19 posts
Joined: Oct. 2011
Offline
Drew has done a great job helping out with all the Linux stuff, It's been a wee while since I've run a linux distro (Thanks Adobe ) But I can try and help from a Windows perspective.

It's rather hacky, but….

Have a full version of Houdini on each machine and direct the HQRender node to use that. It seems your issue is putting the distro into the shared folder. You don't need to reference a shared distro, just any working copy of Houdini.

My Shared folder for Houdini is a Synology NAS so obviously I cant install a distro to that so each of my machines has a full Houdini install that is referenced. They are all installed in the same location on each machine so I dont have to worry about HQServer looking in different places. If you were mixing and matching OS's this wouldn't work but as both your machines are Linux you should be OK.
Edited by SkippyLink - Dec. 6, 2019 08:11:19
User Avatar
Member
141 posts
Joined: Sept. 2013
Offline
Yes, no worries about the help I get here, it's great.

I'm testing a render farm with linux I'm using two pcs under linux.

I would like to use my nose as a shared directory I have a qnap.

But today the server doesn't start anymore I will uninstall and reinstall everything but for now I will continue my houdini training and put it aside for a few days

thank you again for your help
Sorry for my english but i learn too ^^

1 ryzen 3970X 64go
1 xeon hexacore 32go
1 i7 hexacore 16o
User Avatar
Member
19 posts
Joined: Oct. 2011
Offline
Yeah, best to reinstall and start over, I've lost count of how many times I've installed HQ in the last few days…

If your QNAP is anything like Synology, you can't install a distro to it. Shared folder for projects and rendering is fine. But the distro will have to be on a proper OS.

No problem, good luck!
User Avatar
Member
141 posts
Joined: Sept. 2013
Offline
so if I understood correctly I can change the shared directory once hqeue is installed.

For example my two PCs each with houdini installed and the client and on a server
and in my ini file I modify the link of the shared folder to have my rendering files on the nas?
Sorry for my english but i learn too ^^

1 ryzen 3970X 64go
1 xeon hexacore 32go
1 i7 hexacore 16o
User Avatar
Member
19 posts
Joined: Oct. 2011
Offline
Yes, you can change it.

Take my setup for example. (Each machine has a full install of Houdini as if I were going to use it normally)
All Houdini installs at “C:\Program Files\Side Effects Software\Houdini 18.0.287”

Main Desktop - HQClient / HQServer
Laptop - HQClient
Server 1 - HQClient
Server 2 - HQClient
Synology NAS - Shared folder called HQueue)


Each machine has the HQueue folder on the NAS mapped to H:

Each HQClient on each machine has the hqnode.ini file that reads

****

server = IP Addess of Main Desktop
port = 5000 (Whatever port you've set allowing for firewalls etc)
sharedNetwork.mount = //IP Address of Synology NAS/HQueue

****

And the hqserver.ini file on the Main Desktop reads

****

use = eggaste#http
host = IP Addess of Main Desktop
port = 5000 (Whatever port you've set allowing for firewalls etc)

hqserver.sharedNetwork.configLoaded=True
# The shared network.
hqserver.sharedNetwork.host = IP Addesss of Main Desktop
hqserver.sharedNetwork.path.linux = %(here)s/shared
hqserver.sharedNetwork.path.windows = //IP Address of Synology NAS/HQueue
hqserver.sharedNetwork.path.macosx = %(here)s/HQShared
hqserver.sharedNetwork.mount.linux = /mnt/hq
hqserver.sharedNetwork.mount.windows = H:
hqserver.sharedNetwork.mount.macosx = /Users/Shared/HQShared
# Server port number.
hqserver.port = 5000 (Whatever port you've set allowing for firewalls etc)
****

A couple of housekeeping tips to bear in mind
Always stop the HQServer or HQClient service when messing with the ini files

Also, you might be ok on Linux, no idea, but on Windows, H18 messed this all up! Check out thiss thread https://www.sidefx.com/forum/topic/70788/ [www.sidefx.com] for info on the network folders.

However, you should be closer with the info above, the biggest takeaway is to not bother installing a single distro on the shared folder. Just have a copy of houdini on each machine.

The “Target HFS” default on the HQueue Render ROP is always
$HQROOT/houdini_distros/hfs.$HQCLIENTARCH
I change it to
C:\Program Files\Side Effects Software\Houdini 18.0.287

Do whatever is appropriate for Linux here.

Ah, one more tip. Think it's a licence thing, but I have to disable Generate IFD's on the Mantra Options. I use Houdini Indie and my allotted 3 copies of Houdini Engine Indie for rendering on the farm.
Edited by SkippyLink - Dec. 6, 2019 09:06:33
User Avatar
Member
141 posts
Joined: Sept. 2013
Offline
I have some news but it's always the same I'm moving forward a little bit another problem happens I'm going to go crazy. I have a feeling it's a permission issue. If I disable generate ifds I feel it's better but it doesn't make my image.

Attachments:
Capture d’écran de 2019-12-13 20-20-58.png (46.4 KB)
Capture d’écran de 2019-12-13 20-22-03.png (69.5 KB)
Capture d’écran de 2019-12-13 20-27-09.png (81.9 KB)

Sorry for my english but i learn too ^^

1 ryzen 3970X 64go
1 xeon hexacore 32go
1 i7 hexacore 16o
User Avatar
Member
141 posts
Joined: Sept. 2013
Offline
Now its ok en its run Thank you for your help en de support team too
Sorry for my english but i learn too ^^

1 ryzen 3970X 64go
1 xeon hexacore 32go
1 i7 hexacore 16o
  • Quick Links