Distributed simulation tracker

   5765   8   3
User Avatar
Member
19 posts
Joined: Nov. 2012
Offline
Is it necessary to manually run simtracker.py on some computer? Documentation says there must be a tracker to synchronize simulation. For some reason it doesn't work for me. It looks like nodes don't talk to the tracker and stop simulation after one frame. If I don't enter tracker IP in Control node then simulation works, but result is different from what I get with single computer.
User Avatar
Member
19 posts
Joined: Nov. 2012
Offline
It looks like hqueue automatically starts tracker on one of clients. Is it really so hard to write this in documentation? :roll:

Attachments:
Capture.JPG (19.1 KB)

User Avatar
Member
7025 posts
Joined: July 2005
Offline
On a related note, we don't (and can't) use Hqueue, I can't find any docs or discussion about how to set up distributed sims (Flip mostly, some Pyro) in an existing studio.

Any pointers appreciated
User Avatar
Staff
4177 posts
Joined: Sept. 2007
Online
I'm really surprised to hear Hqueue doesn't work with Indie licenses; have you guys asked support? I considering we can purchase Engine indie licenses, it seems like a big oversight (or a bug) that Hqueue isn't allowed.
I'm o.d.d.
User Avatar
Member
7025 posts
Joined: July 2005
Offline
I haven't really asked, will do that now
User Avatar
Member
7025 posts
Joined: July 2005
Offline
pbowmar
On a related note, we don't (and can't) use Hqueue, I can't find any docs or discussion about how to set up distributed sims (Flip mostly, some Pyro) in an existing studio.

Any pointers appreciated

FYI, all you need to do is run a single python file: simtracker.py in $HFS/houdini/python2.7libs

The comments in that file tell you how to point your web browser to it etc.

You still need to set up your own ROPs but it's easy peasy, just set the env var SLICE to be the appropriate slice on each ROP and Bob's Your Uncle.

Sure wish that was documented
User Avatar
Member
373 posts
Joined: March 2009
Offline
https://www.sidefx.com/index.php?option=com_content&task=view&id=2271&Itemid=363. [sidefx.com] I used this a while back to help me get it set up on a studio system. Looks like you figured it all out though.

I didn't find much use for it, however. I guess the main reason to use it for memory requirements. I had no sim speedups even when slicing over 8 machines all in the same rack. It was as fast as using one machine for me. This was a while back though when it first came out. Please let me know if you actually find it useful because I might try and set it up again if its gotten better somehow.
Ian Farnsworth
User Avatar
Member
7025 posts
Joined: July 2005
Offline
Hey Ian,

My initial tests (in H13 btw) show about a 4x speedup with 60million particles in a tank, using 6 machines.

However, the seams are still a problem. The default setting for the seams makes the seams so obvious it's not even vaguely useful. However, changing to Transfer Attributes and Average seems to help, though I'll know more Tuesday morning.

also, the tracker crashed with some sort of socket error, which is concerning too.

All in all it seems like one of those “promising but not robust” type things.

Unfortunately, to do proper oceans, we need to run 100's of millions of particles, and only distribution will make that feasible. Hopefully some efforts will be put into fixing the seam problem and making it overall very robust to compete with other water-oriented studio's technology.

Cheers,

Peter B
User Avatar
Member
373 posts
Joined: March 2009
Offline
Thanks for the response Peter! I'll look into setting it up again and see how it goes. I've been able to run 60 mil particles on a single machine with no problem. I could see needing more in some cases, so its nice to see that the distribution is actually useful. I was able to do some Sims in a sequential style slicing (not done at the same time) to maximize detail, and the overlap areas weren't really too much of an issue when we meshed or rendered together. However that wasn't with an ocean where it may be more visible.
Ian Farnsworth
  • Quick Links