This is a little post about expanding the Contents section of a Digital Asset. But first a little background.

Hip files, the Contents section of a Digital Asset and the files create by the hscript command opwrite are nothing more than CPIO [en.wikipedia.org] archives. cpio is both the name of a binary utility, and a form of digital archive. A cpio archive is essentially a stream of files and directories in a single archive.

110 ~/Contents.hpio.dir % ls -1 *
node_type
subnet1.def
subnet1.init
subnet1.net
subnet1.spare

subnet1:
model.def
model.init
model.net
model.parm
model.spare


The cpio command can be a pain to use so Houdini ships with a tool called hexpand which is a wrapper script for cpio. Using hexpand and hcollapse you can break up hip files into individual parts. (Each op is made up of multiple files, a defintion file, a init file, a parameter file, etc.) This allows you to change any aspect of your scene without having to load up Houdini or hscript, which makes this a very handy tool for debugging. For example there have been times when a custom op will cause Houdini to crash everytime it loads the hip file. To work around this we hexpand the hip file, remove the OP, then hcollapse the hip file again.

Like hip files, the Contents section of a Digital Asset is a cpio archive too, which means you can expand it change some stuff then collapse it and readd it to your digital asset. This is also a handy trick for finding out the difference between two versions of a digital asset. If you want to see what changed between versions you can extract the Contents of the two Digital Assets and the use a tool like diff to compare all the files.

There is a small snag though. At some point between Houdini 6.2 and Houdini 7.0, SESI added support for compressing the Contents section of a digital asset. (You'll see a Contents.gz instead of a Contents section in the Operator Type Manager) While this is a good thing since it makes the size of OTLs smaller, it requires a few extra steps to expand them. First you have to unzip the Contents.gz. Now if you try to hexpand the Contents it won't work because Contents that have been compressed have 4 bytes at the head of the Contents which act as a flag. Sadly this flag prevents hexpand from working. To work around this you can use a built-in linux tool like dd to skip the first 4 bytes so you now have a valid cpio archive for hexpand.

Here are two shell scripts (tcsh) which are handy for expanding compressed contents and then recollapsing them. Each take one argument which will generally be Contents.gz

ExpandContents.csh

#!/bin/tcsh

if ( ! -f $1 ) then
echo “$1 does not exist or is not a file”
exit 1
endif

set base=`basename $1 .gz`
echo ungzipping $1
gunzip $1

echo Splitting $base
# Ignore the first 4 bytes and copy the rest to new file
dd if=$base of=$base.hpio bs=4 skip=1 status=noxfer
# Take the first 4 bytes and save them as a new file
dd if=$base of=$base.header bs=4 count=1 status=noxfer
# Expand the cpio archive
hexpand $base.hpio

echo Expansion finished


CollapseContents.csh

#!/bin/tcsh

set base=`basename $1 .gz`

if ( ! -f $base ) then
echo “$base does not exist or is not a file”
exit 1
endif

echo Collapsing $base
hcollapse -r $base.hpio
# concatenate the header and the cpio archive into one file
cat $base.header $base.hpio > $base
/bin/rm $base.header $base.hpio

echo Compressing $base
gzip $base


Sample Run:

95 ~ % ExtractContents.csh Contents.gz
gunzipping Contents.gz
Splitting Contents
1233+1 records in
1233+1 records out
1+0 records in
1+0 records out
7 blocks
table of contents for Contents.hpio stored in Contents.hpio.contents
7 blocks
Contents.hpio expanded into the directory Contents.hpio.dir
Expansion finished


96 ~ % CollapseContents.csh Contents.gz
Collapsing Contents
old Contents.hpio stored as Contents.hpio.bkp2
collapsed Contents.hpio.dir into Contents.hpio
removed Contents.hpio.contents and Contents.hpio.dir
Compressing Contents


Note: Sadly none of the above will work with files saved from Apprentice. Apprentice files are a different format that prevent it from working with cpio.

Wolfwood! Nice One! :-)

One little hiccup here: it seems on Fedora Core tcsh is in /bin not /usr/bin (Linux 2.6.17-1.2142_FC4smp)
So some ppl may need to change the first line to:
#!/bin/tcshcheers,
ben.

Nice catch….actually its in /bin/ on SuSE too, the tcsh in /usr/bin is just a symlink. I blame windows…..

I fixed the post above.

Wiki! Wiki! Wiki!