I'm aware of more cores/GHz are better then combined.
But in my test case, my machine is 12 cores, all dist sim machines are either 8 cores or 12 cores(actually 2 8 cores and 2 12 cores, they all have same 48G ram), the only difference is my workstation is better version Xeon, while the 8 cores might be a i7 with hyper thread. Cause I want to see how the load balance work.

I thought that only particles on boundary are transferred not everything(cause when I check tracker output, transfer only take miliseconds, and maybe up to seconds, with 240 frame this only takes minutes total out of entire simulation.)

If dist sim is intended to just make sure over sized sim can actually be simed, then I think it still need to be worked on.(cause the tank_initial is actually taking up quite a lot of time as well even sim on a single machine.(if not using flat tank)

Just to pop a question here, I setup a small queue and run a distributed sim using the master class video as my guide, it ran well but it is very slow compare to simulate the sim on a single machine.

From 1 ~ 16million(runs for 23 hours) particles, where the 16m sim runs about 20minutes for 10 frame(out of 240, so it will run about 8 hours on single machine). It is true that the machines run distributed sim is a bit slower than my workstation but still it's quite slow when you have 4 slower machines stack up to run one sim.

Is there a threshold in particle counts that distributed sim start to show advantage?(cause 1m sim is about 10x slower, while 16m sim seems to be only 3x slower) Or is there something wrong in my setup?

Note, this is for the fluid sim only, no white water sim.

the master class does limited the cpu to 12(according to their machine spec). But however, this does not solve the issue. From what I can see when I do my test(might post another thread) distributed sim is very slow compare to a single machine if it's below certain threshold(say even at 15m particles it's still slower to run sim distributed), this is build 15.0.347.

Also, master class do have a part that Jeff walk through how to also limit the tank_initial part, so make sure you also use that to reduce memory usage.

your queue says waiting for machines, if you have 4 slices, you need to have all 4 client pick up the hip and started before distributed sim can progress.

Just tried to crash it again, and no crashlog created at all.
On a side note, the original hip file is created with 13.0.198.21, and then later onto 13.0.237, when I try to reproduce with a simpler setup, I couldn't get it to crash, just the hip I worked on will crash and reproducible.

would mini dump from Process Explorer be helpful?

I'm currently using the vc11 build.
There is a lot of crash but still manageable if I tread carefully.
Some of the most common crash I found is following(if Auto Update is on):

• disconnect node in DOP network when display/render set to down stream nodes get disconnected.
  
  change display while in SOP network that used by AutoDopNetwork.( a particle field), including Alt-LMB to just set display.
  
  change current timeline frame before start frame of AutoDopNetwork, or just change start frame while Auto Update is on.( flipfluidobject's creation frame is expression bind to AutoDopNetwork's start frame, solve on creation is disabled. )
  
  always crash after alembic export finished using alembic rop.(doesn't seems to happen in vc9 build of 13.0.198.21 )
  
  I'd like to share the scene but it's a production scene, I wonder where is the crash dump/log saved so I can send those to the bug report? Houdini did dump a stack I think, but window gray out and Windows force to close it before I can copy paste it somewhere, I can use a ProcessExplorer to export a mini dump file, don't know if this would be more useful?
  
  System Spec:
  OS: Windows 7 Ent 64bits
  Intel Xeon X5690x2
  Mem: 88 GB
  GPU: Quadro 4000, Driver 320.92
  
  

Thanks to everyone who reply my post first, although your solution didnt correct my problem, but narrows down the possible of what cause this all.

It's my custom compiled kernel 2.6.13-4, when I change back to stock kernel of Ubuntu 5.10, everything runs so smooth, that I even tried it 3~4 times with rebooting and switch kernels.(and smesh Fn keys like hell XD)

Finally I would like to try 2.6.14, the newest one but failed to boot, so I just use the last linux-tree provide by Ubuntu archives(which is 2.6.12-xx).

Things turns out quite smoothly using 2.6.12 version of kernel, maybe nVidia should try to keep up with new kernels?(I pull out useless modules which enabled with default settings,turn on SMP, change arch,etc..etc..).

Anyway, a big THANK to you guys!!!
I can finally start playing with new houdini 8~~~YEAH~~~

Best,
Ren-Wei Yang

It's very annoy especially about the geometry part.
After I do something like change modes(OBJ->ROP->OBJ),
or do somthing like preview render, maximize help window, etc.
The icons of both tool bar and network pane are missing, also geometry OGL render disappear after I move camera.

System spec below:
HP xw9300 with Quardo FX1400
Dual Opteron 252, 2G RAM
nVidia driver version 1.0.7676
OS: Ubuntu 5.10
Gcc: 3.4.5
glibc: 2.3.5
Houdini: houdini-8.0.383-linux_rhEL4


See screenshot taken below.
I've already tried with HOUDINI_OGL_DBL_BUFFER_FIX.
And I have nvidia CIOverlay option on in xorg.conf.