I know this is RFE for a long time. Just additional vote of mine. Both Shake IFFs and DPX are somehow the standards in industry. Are there any chances to support them in Houdini?

I know people advocate for dpx for a long time this is rather obvious hole in a pipeline.

I'd like to argue also for iff format, which is used extensively in Maya/Shake facilities and lack of support of it in Houdini is a real. Unfortunately EXR is not so good for Shake compositing (quite opposite to Nuke) yet Shake still holds its position in business (and remains my favorite comp package of course ops: ). There is a difference of seconds in big file access between EXR and IFF thus IFF is a choice if any heavy compositing in Shake.

As long as this is not a terrible challenging shouldn't it be supported? I know there is made Iff reader on Exchange (ready to compile for every single new build), but this is minimal solution. Read/Write and stable support would be great.

Unless I wish too much or I'm alone in that wish…

thanks,
Sy.

You are not alone, sir. I can't say we use the IFF format here, but DPX is getting to the point of hurting. Red One is starting to inject(crap) everywhere and it seems DPX is the most obvious choice.

Cheers,

J. C.

I know it's naive but I would prefer to see Shake/DF/Nuke recognize the .pic format finally. With it's deep rasters it is just as capable as dpx.

As much as it would be nice for us, I'm unsure the world needs yet another standard that doesn't really bring anything new to the party.

Cheers,

J.C.

probbins
I know it's naive but I would prefer to see Shake/DF/Nuke recognize the .pic format finally. With it's deep rasters it is just as capable as dpx.

Well, DPX is something that comes from Telecine. I don't think pic is capable for that purpose (at least practically).

The only reason I bother myself (and you) with IFF is performance issue. IFF is crazy fast. Whenever you comp seriously in >2K, IFF makes a huge difference. As far as I know this is related with its structure and it's not just a matter of native support in Shake. It's not a matter of my favorite format. It's a matter of composting Houdini's render in a most preferable comp package in industry. For now all that comes from Mantra have to be converted to be useful in Shake.

Not a big deal anyway. If I'm alone, I'm gonna live with that

Are you sure you're comparing apples to apples here? OpenEXR stores images as 16-bit floating point whereas AFAICT .iff files are only capable of (up to) 16-bit integer. I'd be more interested in a comparision between 16-bit integer .tiff's and 16-bit integer .iff's.

edward
Are you sure you're comparing apples to apples here? OpenEXR stores images as 16-bit floating point whereas AFAICT .iff files are only capable of (up to) 16-bit integer. I'd be more interested in a comparision between 16-bit integer .tiff's and 16-bit integer .iff's.

This is well know issue. Shake's Iff outperforms any other format. The speed boost specially in multi-layered, heavy scenes is very noticeable. As to OpenEXR, I was comparing Nuke and Shake support for that format, not iff and exr. The second one is very slow in Shake (opposite to Nuke which rocks with exr files)

What if the .iff files were 8-bit? Then that might account for the speed. Compression may also differ. I'd just say I have my doubts that IFF has any particular speed advantages that stems from its format.

I'll admit to being a little skeptical too, I'd be curious if this means ‘iff is fast’ or ‘iff in shake is faster than other formats’. Shake is dead/dying, what matters is the speed in packages that have active development being done.

Cheers,

J.C.

edward
What if the .iff files were 8-bit? Then that might account for the speed. Compression may also differ. I'd just say I have my doubts that IFF has any particular speed advantages that stems from its format.

Sorry, that's my confusion: Shake reads 8bit *.iff files much faster then any other 8bit format supported in it.

*.iff supports also 16bit & 32bit integer. In those cases, Shake outperforms also any other format with particular color depth. This format *was designed to allow fast I/O disk operations and it's noticeable faster. In case of single sequence read it's like a 0.5 second faster for 2K, multiply it by 10, and you have the difference.

I'll admit to being a little skeptical too, I'd be curious if this means ‘iff is fast’ or ‘iff in shake is faster than other formats’. Shake is dead/dying, what matters is the speed in packages that have active development being done.

This is reasonable opinion J.C. and I haven't much to say on this except that many facilities buying Shake's code reinforced the position of that deadly ship (WETA, ILM, TIPPET among others).

As to speed of *.iff files: they are faster then anything else in perhaps the fastest comp application out there. All I know about it is that the layout of the data in a file was designed to allow speedy read.

Anyway, I really understand your skepticism, sir. I'm just sure Shake gonna be here for a long time anyway…

Just to raise a point about OpenEXR performance: compositing software likes to access data in different ways - tiles versus scanlines.

COPs renders in tiles and might be suited to the “piz” compression scheme, but Nuke reads images in scanlines and you get a HUGE speed boost if you set your EXR compression to single scanline zips (“zips”) compression. Not sure about DF.

The slowness of Nuke accessing OpenEXRs saved in piz compression format gets a lot worse when your EXRs have many layers.

Change the options in the FBoptions file for permanent effect. Oh, and I don't think the FBoptions yet have the (commented) options declared in it - perhaps SESI can post in here what those options are.

Definitely need DPX support, that's our standard image format at CIS Vancouver for scans and filmouts etc. EXR otherwise for renders.

Currently I can't Mplay a sequence of DPX files I'm sure I could do an FBio hack but then speed would be an issue vis a vis the conversion.

Cheers,

Peter B

Another vote for DPX. As for IFF, I really don't care.

Dragos

Pardon my ignorance, but I've only heard about this DPX format a minute ago.

Would any one care to shed some light on the advantages of DPX and potential technical challenges that's preventing it from being supported in Houdini COPs/Mplay?

I am surprised to hear that iff format is faster than exr (in everyday use). In an FX environment, where an image might be 90% black, an exr with an embedded DOD, (we have a util that runs an autoDOD as a post-process on each CG render), loads hella fast. I think the autoDOD utility was some of the best $$ we ever spent.

mrCatfish
I am surprised to hear that iff format is faster than exr (in everyday use). In an FX environment, where an image might be 90% black, an exr with an embedded DOD, (we have a util that runs an autoDOD as a post-process on each CG render), loads hella fast. I think the autoDOD utility was some of the best $$ we ever spent.

I meant iffs are faster then exr *inside Shake*. In fact, Shake's IFF was supporting DOD, AOV etc, the way before OpenExr saw the sunrise of open source and plain Maya software render saves iffs with DOD without any care or addons.


EDIT:

just made simple test:
simple composition of two 2K plates (grainy - to avoid compression):
32bit float for IFFs – 0.5 second per frame (frames between 8-24 MB, final comp ~36 MB per frame)
16bit float for EXRs – 1.5 second per frame (5-13MB, final 11 MB per frame)

Pardon my ignorance, but I've only heard about this DPX format a minute ago.

Would any one care to shed some light on the advantages of DPX and potential technical challenges that's preventing it from being supported in Houdini COPs/Mplay?

I don't know anything about potential technical challenges, but as to DPX its self, Digital Picture Exchange is a 10bit logarithmic format designed to be used by film scanner facilities. DPX is an standard in digital intermediate. If you work with life action footage you will most likely get it in dpx. Besides storing logarithmic colour it handles timecode from telecine or any other custom data (like exposure of film stock).

SYmek
I don't know anything about potential technical challenges, but as to DPX its self, Digital Picture Exchange is a 10bit logarithmic format designed to be used by film scanner facilities. DPX is an standard in digital intermediate. If you work with life action footage you will most likely get it in dpx. Besides storing logarithmic colour it handles timecode from telecine or any other custom data (like exposure of film stock).

A simple way to think about DPX'es is that they are basically .cin files with more header info. Cineons, however, were never particularly fast to load. DPX'es now support compression and allow for some bit/log/lin variation whereas Cineons were always uncompressed 10-bit log format files, as coming from a scanner, the “compressed” file was often bigger than the uncompressed one due to noise.

AFAICT, there's no compression in IFF's. Try the test with uncompressed EXRs.

edward
AFAICT, there's no compression in IFF's. Try the test with uncompressed EXRs.

And with the same bit depth. Any operations on 16 bit floats will probably have some overhead converting to full 32 bit floats.

edward
AFAICT, there's no compression in IFF's. Try the test with uncompressed EXRs.

I did. There was no compression on Exrs (sorry for confusion with compression note). Ok, I'll make test with exr 32 float, but after all I'm pretty sure all tests will show the same thing for Shake. I saw it many times in many compositions. This is the main reason for preparing comp footage before work in Shake. Any other format then iff, is much slower, what can be clearly seen specially on a few plates. Imagine x3 speed difference on ten or twenty 2K images… I made the test just for sake of what I said before. As I said I understand your skepticism and I'm not convincing you guys any more.


Simon.

BTW Does Mantra saves DOD in Exr?