I'm trying to load a bunch of photogrammetry scans into a single hiplc file for a batch processing thing. Each scan is about 12 million polygons, and I bring it in via a batch script that uses importFBX(). Works great with less than ten scans, starts falling apart once I approach 30 or so.

All of the file nodes are set to be unlocked, visibility tags are all turned off, and delay load geometry is on to try to make sure nothing gets loaded until I want to visualize or process something. I even set the import to bring everything in at half floating point precision in case that lightens the load a bit. I have also cached all of the geometry further downstream after a bit of processing into .bgeo format, so the stuff being shown on the screen shouldn't even have anything to do with the imported .fbx file.

But still, despite all of that the HIP file is becoming gigantic (we're talking 3 gigabytes) and things take forever to load (I timed with a stopwatch and it took an hour or so before the interface is all caught up and ready to use - on a AMD 5950x, RTX 3080, 128 gig ram system).

I thought that everything was supposed to be by reference only? What on earth is it doing with all these unlocked nodes that's gumming the whole system up despite them having turned off visibility and delayed loading? Shouldn't it just ignore them as if they don't exist until I turn on the visibility flag and only then prompt Houdini to start trying to visualize it? I thought that lightweight file references was the whole point of Houdini.

Is there anything I'm missing here? I know that fbx files can have a lot of additional weight to them, but again, I'm only importing geometry and camera positions and I'm generally sanitizing the whole import to the bare minimum. There has to be a way to force houdini to just ignore the file outside of keeping a simple reference to it unless something I'm doing actively requests the data. The setup I'm trying to get to is where despite the fact that I may have a whole bunch of large photogrammetry models in a single hip file, but at any given time only one of them is ever visible so when everything is just a reference that should still be lightweight no matter how many photogrammetry scans I have in there. But clearly houdini is choking on something.

if your file nodes are unlocked it shouldn't bloat up your file as much
a lot of nodes can definitely add up, so it also depends on how many objects each FBX contains and additional baggage like materials etc
however, if your file is 3GB there likely is some locked or stashed geo in there

I'd suggest try importing only cameras through importFBX()
- then the whole geo in a single File SOP rather than Object/file SOP per object
- or run through TOPs one at a time to convert to bgeosc and import just those
- if your batch processing is the ame for all you can also apply some of that during the processing using TOPs so that the geo you need to import is as lightweight as possible