For your initial configuration, you might look at the Tetrahedral setting for Point Configuration on the PointsFromVolume SOP. This gives a fairly tight sphere-packing and is used for setting up Grain simulations.

Turn on Add Scale Attribute and set the Particle Radius Scale to 0.5, then use the Copy SOP to copy 1-unit spheres onto the points. They should stack together nicely. Increase the Particle Radius Scale slightly and they'll overlap.

Quick update: this issue has been fixed in the latest daily build:

Houdini Journal
Wednesday, February 3, 2016
Houdini 15.0.376: Fix refresh problems when interrupting the cooking of various VDB filtering nodes. This also fixes the refresh problems of the Particle Fluid Surface SOP, which internally used these VDB nodes.

Please submit a bug if you see any other refresh problems.

nigelgardiner
set HOUDINI_OCL_DEVICETYPE=GPU
set HOUDINI_OCL_DEVICENUMBER=0

The first step is to check About Houdini..| Show Details and make sure the OpenCL device is listed. Or with H15 run:

hgpuinfo -c

in a command-line shell to get the same information.

nigelgardiner
But when I run my fluid simulation which is using a particle fluid solver I see no or very little activity on the graphics card monitor.

Do only certain solvers use open cl? OR am I missing something here?

Only certain solvers use OpenCL, the main ones being Smoke/Pyro, Grains, and FLIP for pressure and viscosity solves. But you generally have to enable Use OpenCL on the solvers, often under the Solver or Advanced tab.

Actually the Intel GPU will be picked up with DEVICETYPE=GPU, so you really need to use VENDOR. The problem is likely that the string should be “NVIDIA Corporation”, not just NVIDIA. And DEVICENUMBER should stay 0, since that's per vendor.

In H15 there's a command line tool called hgpuinfo that can help debug OpenCL device problems without launching Houdini:


$ hgpuinfo -c
OpenCL Platform NVIDIA CUDA
Platform Vendor NVIDIA Corporation
Platform Version OpenCL 1.2 CUDA 7.5.8
OpenCL Device Quadro K6000
OpenCL Type GPU
Device Version OpenCL 1.2 CUDA
Frequency 901 MHz
Compute Units 15
Device Address Bits 64
Global Memory 12278 MB
Max Allocation 3069 MB
Global Cache 240 KB
Max Constant Args 9
Max Constant Size 64 KB
Local Mem Size 48 KB
2D Image Support 16384x16384
3D Image Support 4096x4096x4096

$ HOUDINI_OCL_DEVICETYPE=CPU hgpuinfo -c
OpenCL Platform Intel(R) OpenCL
Platform Vendor Intel(R) Corporation
Platform Version OpenCL 1.2 LINUX
OpenCL Device Intel(R) Xeon(R) CPU X5650 @ 2.67GHz
OpenCL Type CPU
Device Version OpenCL 1.2 (Build 57)
Frequency 2670 MHz
Compute Units 24
Device Address Bits 64
Global Memory 24102 MB
Max Allocation 6025 MB
Global Cache 256 KB
Max Constant Args 480
Max Constant Size 128 KB
Local Mem Size 32 KB
2D Image Support 16384x16384
3D Image Support 2048x2048x2048


I've been meaning to add an RFE for an additional command-line switch that will list all available OpenCL devices. I'll add that to the bug db now.

meeotch
Thanks for the reply. Will that give the correct result? Aren't the frequency & phase spectra necessary to evaluate the whole thing?

As long as the first seven parameters of the Ocean Spectrum nodes match (i.e. Resolution, Grid Size, Depth, Gravity, Seed, Loop over Time, Loop Period), then the phase and frequency will match and you can just add all the amplitudes as Andrew suggests. Probably the biggest limitation here is Seed, since it controls the random phases of the different frequencies. Also I believe only the second Chop parameter will have an effect.

Note that you can do a fair amount of tweaking to the amplitudes with the Remap Amplitude ramp on the Wave Amplitude tab. Also you could use as many spectra as you want and add all their amplitudes, again assuming those first seven parameters match.

If you can get the desired result, adding the amplitudes is likely the best approach since none of the spectrum nodes are time-dependent and will typically only evaluate once, and you're still only ultimately sending one spectrum into the OceanEvaluate so that's no more expensive.

What Jeff said.

The only thing I'd add is that we specifically write the OpenCL kernels to avoid branching as much as possible, since divergent code paths drastically slow down the GPU. For example for boundary conditions during the multigrid solve we use padded grids rather than have any “if” statements in the kernels.

Fortunately avoiding branching also makes the kernels much easier to automatically “vectorize” to SSE/AVX on the CPU by the generally excellent Intel compiler technology embedded within their driver. Regular c++ compilers have a more complex language and more general code to deal with, so have a harder time with auto-vectorization, meaning we'd likely have to do any similar vectorization by hand.

There's some information about getting the best performance from OpenCL CPU in this thread [sidefx.com], as well as some benchmark files you can test with. Also here [sidefx.com].

In general you can use the Performance Monitor to see where the time is going. OpenCL should generally be faster for advection and the multigrid solve, which are often the most expensive parts of a large smoke / pyro sim. However if you're spending all your time doing other things such as deforming collisions, sourcing, drawing the viewport, caching the simulation to memory, you'll see less of a speedup.

Just wanted to point out that the new Nvida 352.09 beta drivers seem to fix the 4GB OpenCL limitation! I got them for Linux here:

http://www.nvidia.com/download/driverResults.aspx/85057/en-us [nvidia.com]

I don't know the status under Windows, I'm afraid.

I ran a 200M voxel smoke sim last night that used 11GB in a K6000 and solved in under 4 seconds / frame.

We'd be very curious to hear experiences successful or otherwise if anyone has a chance to try these.

MartybNz
As a side note to the SP to DP disparity; interesting talk at GDC, by Amber molecular dynamics, where they compute in single precision and accumulate in double precision IIRC i.e. comparing DP, DPFP, SPFP, SPXP etc

FWIW we do the same thing. Most of the internal multigrid computations are single-precision, but if we're looking at total error to determine whether we can stop iterating, we use double precision for accumulation, dot product totals, etc.

Luke Letellier
What about running the simulation by clicking the “render” button on a DOP I/O node? How does this compare to the memory usage of the command line setup?

Render will just render in the current process on top of all the current memory use. Background Render does launch a separate process and is essentially the same thing as rendering from the command line, so I suppose you could start it that way then quit your main Houdini session to free up memory.

But for something like big overnight sim on a single machine I usually recommend the command-line, though setting up your own local hqueue server is also an option (requiring a bit more setup, of course)

That's looking really nice.

There are a few things you can do to reduce memory for big whitewater sims. In particular a lot of the memory use comes from loading the FLIP sim into Whitewater Source.

So, some things to try:

• Delete all possible attributes from the original FLIP particles to save both disk space and memory when reloading into Whitewater Source. With something that big, I'd get rid of everything but v. If you need pscale for surfacing, you can re-create it later after load. (Usually I do this on the Compression tab of the DOP I/O SOP, but you don't need to re-sim; just load the existing geo in and run through an Attribute Delete, although the next step might make even that unnecessary in this case…)
  
  
• Cache out the Whitewater Source results from the whitewatersource_cache SOP. This will take extra disk space since you're saving the volumes to disk twice, but then you'll only need to load in a very small subset of the original FLIP particles when running the whitewater sim. If that's too much disk space in general, you can save the original FLIP particles and volumes separately, but that takes more setup. I can elaborate if you don't have the disk space.
  
  
• Turn off Allow Caching for the Whitewater Object, since you won't be playing it back in real-time obviously. This just prevents making any copies of the whitewater object data, which includes the volumes. (You can do the same for the FLIP object under the Creation tab).
  
  
• Maybe copy/paste the static object from the FLIP sim into Whitewater to get terrain collisions, though I'd try it the way you have it set up first.
  
  
• Run your simulations from the command line. It takes a lot of memory to load and display the particles and terrain and such. Rendering from the command line avoids any display-related memory use. For the whitewater sim in this file it would just look like:
  
  $ hbatch newRiverWhitewaterRamTest.hiplc
  hbatch Version 14.0.260 (Compiled on Mar 10 2015)
  WARNING: Entered limited commercial session mode.
  / -> render /obj/sim/rop_geometry2
  
  or
  
  $ hrender -d /obj/sim/rop_geometry2 newRiverWhitewaterRamTest.hiplc
  
  
  
  There are a few additional steps you can take, but they generally require more setup and hopefully these will get you started. Let us know if it helps!

riviera
Btw, today I happened to do something related (particle sims with/without impact data), and particle-geometry collision feels way faster in H14 than before (I had a heavy collision geo and quite some substepping, and the difference was noticeable). Which is great news, and really appreciated

Yes, we did some significant optimization of particle / SDF collisions. Glad to hear you're seeing an improvement! I've included my post from the Beta forum describing some of the changes and including a file that shows the difference pretty starkly:

From the beta forum:
———————————————

Tomorrow's build(14.0.184) has the following change:

Houdini journal
Houdini 14.0.184: Mon Dec 15 15:06:54 CST 2014
Particle / SDF collisions have been optimized, improving collision
performance and decreasing DOP cache memory for the POP, POP Grains,
Whitewater and FLIP Solvers:
- Particle / SDF collision impulse calculation has been optimized and
multithreaded.
- The GasIntegrator, POP Solver, and POP Grains DOPs have a new
Add Impacts Data parameter that controls whether the particle object
receives Impact data, defaulting to off.
- Particle collisions now only create Feedback data on actively solved
collider objects, not static ones, similar to the RBD solver.
- Collision velocity lookup in the GasCollisionDetect DOP has been optimized.

Basically we've optimized particle/SDF collision response and changed some default behavior to decrease DOPnet cache memory use in typical particle scenarios. If you have the chance, please test tomorrow's build with some typical particle collision workflows to look for any strangeness.

Generally you should just see much better performance for setups involving lots of collisions, but there are two other slightly controversial changes that we'd like to hear opinions on as soon as possible.

1) By default we no longer create Impacts data on particle objects. This was required in H13 to get hit attributes via GasImpactsToAttributes, but in H14 the attributes are created by GasCollisionDetect, so Impacts data is usually just taking up time and space. There is a new option to enable Impact creation.

2) We no longer create Feedback data on static objects. Before this change, every particle impact with something like a static groundplane created a Feedback record, which was fairly wasteful as it could never have an effect. Now we only create Feedback on actively solved objects, similar to the RBD solver.

As an example, the attached file emits particles onto a groundplane and RBD box with feedback. In 14.0.182, advancing to frame 100 takes 190 seconds and 2038 MB DOPnet cache memory. In 14.0.184 with this change, it takes 21 seconds and 770 MB cache memory.

POP Grains particularly benefit, since we can take a very fast path with them when all Impact creation is off. For typical big sand sims the collision processing should be down below 5% now, even with a fast GPU solving the constraints.

Alireza Soleimanikhah
but when i simulate the ship in the tank the liquid is not splashi or not behave like water. its like a viscouse fluid like creame i changed this

Does it help if you turn off Treat as Wind on the advect_by_ocean node in the created DOP network?

We probably should default that to off, since it can act as a drag against water that doesn't match the underlying ocean velocities (for example, in a big splash from a boat).

goldleaf
- Is kvisc an attribute the solver knows about? I thought the FLIP Solver had a chart of attributes it inherits (ala Bullet and POP Solvers), but maybe I'm imagining it?

Sorry no, I just meant you might find a viscosity setting you like, but want the foam to float more. If you change the density you have to change the viscosity as well. So in your AttribWrangle it might be easier to do something like:

float kvisc = 1.5;
@density = 150;
@viscosity = @density * kvisc;


Then just tweak kvisc and not worry about it the viscosity behavior changing when you change density.

goldleaf
- The optimization tips are great; for viscosity accuracy, is there a reason the default is 64-bit, and not 32-bit?

It's more accurate for high viscosity (and matches cloth's similar default). But for low viscosity like this (and if you want to run the solver on the GPU) it can be a bit faster to run in 32-bit.

goldleaf
- For visualizing the different fields, is it possible those could get a 3d mode that isn't smoke? I'd really like to see it with a field of points changing color (or maybe even scaling up/down); it's so handy looking at vel this way, it'd be great to see scalars in 3d as well. I peaked inside, but couldn't find if those Visualizers could be modified…

That's an interesting RFE (please submit it), not sure how hard it would be. I usually just use planes and slide them around to see the parts I'm interested in.

Cool test.

You've got the right idea with your setup, but your Mix Method for Divergence and Viscosity attributes are set to Multiply, which I don't think is what you want. That method multiplies the particle attribute value with the default field value as specified on the FLIP Object's Physical tab. So in this case you're multiplying your density and viscosity attributes by the default values of 1000 and 0, respectively. You end up with very high density values and zero viscosity everywhere. With your setup just use Copy for both, since you're explicitly setting the particle values to represent the final density / viscosity values.

In general it's a *really* good idea with a complex setup like this to turn on the visualization for the Density and Viscosity fields, set the Guide Ranges appropriately, and inspect the field values directly. The field values are what the solvers actually use.

Some other notes:

- You likely won't need Stick on Collision; likely you had to turn it on since viscosity wasn't working.

- You don't need a mass attribute as it has no effect for FLIP.

- I turned off Particle Separate as I don't like the look, but of course that's subjective.

- With Reseeding you probably don't want to interpolate viscosity and density. For the most part this is a “two-phase” simulation of different liquids, so when new particles are created they should be one fluid or the other, not somewhere in between (unlike, for example, a melting simulation)

- Because it's a two-phase simulation, you might try decreasing the Grid Scale to 1.5 or so, which just makes it more likely a voxel is purely one fluid or the other. If you do that, likely change the Particles per Voxel to 4 (well, really pow(gridscale, 3) but 4 is close enough)

- For speed, I moved the bricker on your box option into the source geometry, changed substeps back to 2 and switched to 32-bit accuracy for viscosity.

- You don't need very high viscosity for this setup. I got decent results with density = 150, viscosity = 250 for the foam, but obviously you'll need to tweak.

- When tweaking your density and viscosity values, remember that we use dynamic viscosity units, which are then divided by density. So a given viscosity value will have a greater effect on less dense fluid. It might make sense to tweak kinematic viscosity values, then just set @viscosity = kvisc * @density;

- Finally this is another situation where you might want higher viscosity values but not want the foam to stick to the paddle that much. You can use the trick / hack / cheat I described here [sidefx.com] to make the foam stick to the paddle less.

I've attached a file with these changes as a start.

Also try increasing the Particle Radius Scale to the 1.5-1.7 range and set Under-Resolved Particles to None. That basically turns off all the ballistic-type behavior of particles on the edges of the fluid.

Oh, and turn off Reseeding for high viscosity.

Your setup was very close. The problem is that the FLIP particles are moving through space and picking up whatever color is currently closest from the animated noise map, so the colors don't track with the particles.

There's a few ways to fix this:

Since you have Reseeding off, the easiest way in this case is use an Attrib Copy rather than Attrib Transfer. The former just copies color directly from the source geometry by matching point numbers, and in this case you've got a one-to-one mapping.

A more generic approach (e.g. supports reseeding or other color sources) keeps the Attrib Transfer but first applies a Rest Position SOP to the original particles, creating a “rest” attribute with their original positions. Then in your SOP Solver move each particle back to its rest position before doing the AttribTransfer, then Attrib Copy to the particles in their current positions in the simulation.

Or you could do the whole thing in one AttribWrangle using point clouds.

The attached shows each approach.

Netvudu
I think this kind of solutions should be more prominently mentioned. I know this node skipped under my radar for almost a year

Fair point. I've attached a simple example of transforming hi-res fractured geo from a low-res RBD simulation. This will be an example for the Transform Pieces help for tomorrow's build.

By the way, Transform Pieces actually shows up in the Point Replicate SOP: all the points are created and sampled at the origin then transformed by the template points using Transform Pieces. POP Replicate is also based on Point Replicate, so Transform Pieces gets a fair bit of use, just somewhat hidden.

Luke Letellier
Ah, this explains a lot. For some reason I had thought that spray/foam calculations only needed the geometry - so I thought “Why I am a caching what I don't really need?” Optimization FAIL on my part, haha.

Well, strictly speaking you can send just the points into Whitewater Source and have it rebuild those fields for you (see the Sources tab), but that's more for use with external sims. Rebuilding does the exact same thing as FLIP, so better to just save the FLIP output in the first place. Also, the whitewater sim itself requires these fields: it's how it advects particles along with the original FLIP sim, and figures out depth of particles for things like particle classification, buoyancy, etc.

Luke Letellier
This is very interesting, as I had previously thought that that particular HIP file option was only in effect at sim time, rather than at scene setup time. How exactly did you go about resolving this issue? Did you simply delete the whitewater setup & redo the shelf tool with the correct HIP file settings - or did you manually go in and fix the gravity, buoyancy, etc?

Right, it applies at node creation time, scaling the various parameters that are marked with units. The assumption is that the units setting is more of a studio- or at least project- wide setting that doesn't change very often. It's often used at studios that also work with Maya and have adopted a centimeter-based unit scale.

To fix it I just used Revert to Factory Defaults on the parameter tab then restored just the Volume Source parameter (and turned on Volumetric Foam).

Luke Letellier
In terms of “correct Houdini usage” isn't it preferable to perform PSR adjustments in a transform node rather than at the object node level? I've heard differing views on this, and I'm guessing that it's really just a case of “different situations call for different settings”.

For non-deforming geometry, you're generally better off at the object-level. In that case DOPs imports the geometry and builds a collision SDF only once, then just applies any updated object-level transforms on each timestep. But in the scheme of a large FLIP sim and smaller collision object like this the cost is not really that great either way.

Hi Luke,

Sorry, just looked at this (somehow the did bug not show up in the bug database).

When debugging whitewater problems, the first place to look is always Whitewater Source, so you can see where you're actually emitting whitewater. In your case if you look at that node, you'll see the entire FLIP sim is showing up there as emission points!

This is because you're not importing and writing out your surface and vel fields to disk along with your FLIP particles. When Whitewater Source calculates acceleration, it compares the current points to the previous frame's velocity field. If the vel field is missing, it's always comparing the particles' velocity to 0, so there's always acceleration (admittedly we should do a much better job of erroring out if the vel or surface fields are not there.) Anyway, in the current file you're sending in your entire FLIP sim as emitter particles (over 12M points), and it's replicating whitewater points at each particle; not surprisingly you're quickly running out of memory. (Houdini is also particularly memory-hungry under Windows, unfortunately.)

However, even if you fix this, you've got your initial particles situated a fair bit above the bottom of fluid tank for the FLIP sim, so they immediately fall down due to gravity and make a big splash, adding more acceleration. You need to make sure your Volume Limits align with the edges of your initial particle box, or just use the FLIP tank shelf tool.

With those issues fixed your whitewater sim should no longer run out of memory. However for some reason the .hip file preferences have Unit Length 10M = 1M (Preferences | Hip File Options menu item). It appears that was set before you created the whitewater sim but after the FLIP sim, so those two sims are operating at different scales and your gravity, buoyancy, foam depth, etc. on the whitewater sim are all in the wrong units and look strange.

Finally you might consider animating your ship at the Object level and turning off Use Deforming Geometry. Not a really big deal, but it would avoid re-calculating the collision SDF each timestep.

I've attached a file with a few of these fixes that yields a reasonable whitewater sim.

Edit: sorry, forgot one more thing: you're also visualizing the different whitewater particles by type, which is fine, but internally actually creates a copy of all the points with colors attached. So that makes the memory use worse if you're displaying the points while simulating.

For big whitewater sims you're better off writing straight to disk, then if you want to visualize using the various point groups for foam, spray, etc. and coloring them in SOPs for visualization.