Is there a better way to pause and resume caching? Let's say I have 240 frames. I keep working on something else in the background while caching keeps happening. I realize that 170 frames have been cached and now I need to pause it to keep working on something else so that I have enough CPU power and RAM.

Hours pass by and I want to close the day, shutdown my machine and go to sleep.

Next morning I wake up, I want to start caching at 171st frame onwards.

What's the ideal way?

Specifically asking for bgeo.sc type caching, not for .sim caching.

Leverage all your time. Run your caching overnight while you sleep. Get a second machine. With Indie you get two licenses.

I see. I thought there was some ideal workaround to overcome this. I only have one machine as of now.

How about having Thinkbox's Deadline Submitter(free) on your machine? so you can schedule simulation and rendering scenes.
You can also add dependencies. After sim scene has been finished, the rendering scenes are automatically started even midnight.

if the simulation is still on-going during your working time, set houdini's CPU low-usage(maybe half usage) from Task Manager.
if you pause the simulation, you need to restart the sim, so the idea is keep the simulation but make CPU low usage.
After you finished working, you can set it back to normal.

If you have a lot of memory, you can set concurrent task from 1 to 3. meaning you can render 3 machines at same time.
Sometimes this boost your rendering time

Haven't heard of it before. Can you please share some good and authentic resources to learn about this deadline submitter workflow for Houdini?

The only way I know to resume sim after the crash or deliberate kill on the farm is to cache checkpoint files alongside
https://www.sidefx.com/docs/houdini/dyno/cache.html#explicit-dop-disk-caching [www.sidefx.com]

You can play with Checkpoint Trail Length and Checkpoint Interval to save only small amount of trailing .sim filés exactly for the purpose of resuming
While you still filter down the sim data and save normally as bgeosc or other format

Ok. So which method is more robust when dealing with heavy sim caching? Using deadline or using checkpoints?

from the description of the Deadline approach it looks like it will still need to keep the sim fully in memory and can potentially just free up some CPU resources, so for heavy sims that may be a no go, unless you just need to perform low memory tasks while the sim is on low CPU
On top of that it doesn't protect you from crashes

so checkpoints may be a safer way and also the only way I know that should allow you to truly resume the sim, even though I didn't have a need to use them in production, so not speaking from experience, but just in how I understand it should work
the .sim files may be heavy, but you may need to keep only 1 last rolling frame on disk at a time so it may be worth for long sims

and you can always combine the techniques, that way you can use deadline or other scheduler if it allows you to decrease the resources on the fly but you can also kill the job if needed if you are checkpointing and then submit from that frame later

I find that sim workflows with checkpoints are very easy to screw up. It's a bit like takes, it sounds great at first and then things go wrong and you don't notice until it's too late. That's just my personal experience and it may work well for some.

On Linux you can also put your machine to sleep and the just awake it at a later point and things will continue where they stopped. Not 100% sure that works with opencl on GPUs.