Hi there, pardon my ignorance in these matters.

Is there any way to increase the data chunk size that mantra asks to load from the network?
For example, when mantra reads TGA's from disk it seems to do it by sending truck loads of requests for 4096byte chunks of the files (according to process monitor).

This seems to be the cause of really long render times and is stressing the network no end, the cpu's spend 90% of the time just waiting for textures to load!
The render times are a lot slower when I render the textures in RAT format. Mantra then loads even smaller chunk sizes, mostly 512 bytes but never more than 4096. Our network setup really doesnt seem to like it

I observed how other aplications load files, Photoshop for example, will load one of the 98MB tga's in about 5 seconds. I suspect because its chucks are much larger at 24,576 bytes

is there any way to tell mantra to load bigger chunks? or am I barking up the wrong tree completely?

Thanks
Sergio

I'm working alongside Serge here, and as the company render manager we're somewhat confounded by this behaviour.

We are trying out .tgas as opposed to .rats since the slowdown experienced by the .rats selectively loading small chunks at a time was slowed down considerably and tweaking the rat caches env setting didn't make a huge difference.

It's the thousands of instructions to load 4096bytes across our network which are being slowed when the servers drives are being polled so much so that it's slowing down the disks response time and building huge disk queues.

Our servers are very capable and the network backbone is large enough to handle anything. It's a little confusing to see a 58mb tga take an age to load on such a strong infrastructure.

The CPUs on the nodes are practically idle whilst the load texture process seems to take an absolute age when compared to normal loading operations of other renderers.

Whilst I'm not professing to be a Houdini Expert on these matters I am very open to ideas and info which will help us streamline these load times.

Is this a behaviour that's present because we had .rats in there at an earlier stage?

Is there a reference being made to a texture format flag that isn't being used?

Is there a way to increase the read size of the texture chunks for the mantra.exe? - (either in the ifds or on mantra's commandline)

We're using process monitor (formerly by systernals) to record the process behaviour and this has proven to be invaluable in tracking down what a node is doing when things take an age or go wrong.
http://www.microsoft.com/technet/sysinternals/default.mspx [microsoft.com]

This behaviour has also been observed on workstations when rendering the frame on the desktop so it's very unlikely it's our farm/nodes that are at fault.

Cheers

Jamie

Version and platform(all MS all the way?)?

Cheers,

J.C.

Houdini 8.2.58 on XP pro 64.
MS all the way.
You can tell it's my 1st post.
Cheers

J

Unless that's a typo, definitely upgrade. 8.2 is up to 238, lots of room for fixes in there, although of course that might not be the problem. In case you don't know, as a rule running the later cuts of Houdini tend to have fewer bugs, better performance, rather than the other way ‘round. A lot of software packages, daily cuts means flakier, not here.

So *all* your machines are XP64? I’m not even sure there was a 64 bit version of Houdini for 8.2 - we're all linux here so I'm out of my area with windows in production. What sort of scale we talking about here? 10 machines? 100? 1000? I know there are shops running mantra on windows farms and not noticed something so profound being mentioned. How are you farming out your renders? if you're on a really big farm, I'd be taking this directly to support rather than the forums, sounds like something's wrong somewhere…

Cheers,

J.C.

Yea, all the machines concerned with rendering are XP64, the farm is larger than 100cpus and we are running mantra on these systems happily for quite some time. Maybe it's got something to do with the fact it's a 32-bit process on a 64-bit os.

Thanks for your reply, we may take this to side-fx support. Just wanted to see if there was any MS experiences like ours on certain shots.

Upgrading can be, as your pointed out, prone to introduce more bugs than we already live with. We're mid project and if we're going to update to the latest 8 version it'll be on a testbed initially etc. Again tho thanks for your reply.


Cheers

Jamie

Oh yeah, there's that whole mid-project thing where you can't upgrade. Now I remember why we stayed small.

We're running about half your size here on Linux, a mix of 32/64 bit, and certainly haven't hit any I/O issues like that, apart from IFD generation which of course can always be a bear. Still, networking on MS is another world compared to Linux, so I suspect the problem is tied to that.

Good luck with it. I checked the dev logs and didn't see anything between your version and latest that would be directly related to your problem, but SESI might have some thoughts.

Cheers,

J.C.

i can also confirm that we absolutely hammer the network with no houdini related slowdowns. this certainly sounds like a windoze thing. i seem to remember a similar problem eons ago when we used windoze (if thats any comfort).