Digital assets are stored on disk in an Operator Type Library file. The OTL can contain many HDAs, but it can also store other operator definitions, such as scripted operators, though that is less common. When Houdini starts, it uses the HOUDINI_OTLSCAN_PATH to look for the OTL files or HOUDINI_OPLIBRARIES_PATH to look for OPlibrary files that specify the list of OTLs to load. In Houdini, the OP_OTLManager is responsible for managing the digital assets, and it holds a list of OP_OTLLibrary instances that in turn contain a list of OP_OTLDefinition objects, each corresponding to an HDA.
There may be different situations, when you may want to interact with HDAs in the HDK code. You may want to inspect the current Houdini session to see what HDAs are used and save a backup version for them, or you may want to programatically add or remove some data to or from an HDA. You would use the three classes mentioned above, and the sections below describe them in more detail.
There is only one instance of OP_OTLManager class and it is owned by the OP_Director global object. You can use OPgetDirector() global function to access that object (for more info on OP_Director please refer to About OP_Director) and then you can use OP_Director::getOTLManager() to obtain the OTL manager instance.
The following code lists the files of the loaded operator type libraries:
// get the global instance of the OTL manager OP_OTLManager &mgr = OPgetDirector()->getOTLManager(); // iterate through the list of libraries and print their file name for( int i = 0; i < mgr.getNumLibraries(); i++ ) { UT_String full_path; OP_OTLLibrary * library = mgr.getLibrary( i ); const char * file_path = library->getSource().buffer(); OP_OTLManager::fullyQualifyLibFileName( file_path, full_path ); cout << full_path << endl; }
If you need to load a new OTL into Houdini, you should use OP_OTLManager::addLibrary() method, which does extra processing such as checking whether the library has been already loaded, warning about non-commercial assets it contains, and adding it to the meta source (see Metasources). After adding the library, Houdini will be able to use the operator definitions it provides.
UT_WorkBuffer errors; OP_OTLLibrary * lib; lib = OPgetDirector()->getOTLManager().addLibrary( "myassets.otl", // OTL file name OTL_INTERNAL_META, // install library for current .hip file only true, // modify meta source to include this library false, // if failed including to the specified meta // source, then don't include it in the // internal meta (since that's the one // that we already specified). errors ); // outgoing error messages if( lib == NULL ) cerr << "Failed to load the OTL library: " << errors.buffer() << endl;
Sometimes, the OTL file may have changed on the disk, when, for example, a new file has been downloaded or an old backup version has been copied from another directory. The Houdini will pick up this new copy of the file only the next time it starts up. But, there is a way to explicitly update the new definitions using OP_OTLManager::refreshLibrary() method. It will make sure Houdini uses the freshest copy of the OTL.
OP_OTLManager & mgr = OPgetDirector()->getOTLManager(); int index = mgr.findLibrary( "myassets.otl", OTL_OTLSCAN_META ); UT_WorkBuffer errors; if (index >= 0) { // Houdini is aware of "myassets.otl" library, so refresh it if( !otl_manager.refreshLibrary( index )) cerr << "Error while refreshing the OTL" << endl; } else { // Houdini never loaded 'myassets.otl' library, so load it now if( !otl_manager.addLibrary( "myassets.otl", OTL_OTLSCAN_META, false, false, errors )) cerr << errors.buffer() << endl; }
And finally, if Houdini no longer needs to use a certain library during the current session, you can unload it using OP_OTLManager::removeLibrary(). This call does not delete the file, but rather removes the library from the manager's list.
OPgetDirector()->getOTLManager().removeLibrary( "myassets.otl", "", // empty string means to use meta in which the // library was originally specified when loaded false ); // don't modify the meta source
The OTL file name can be obtained with OP_OTLLibrary::getSource() and its meta source with OP_OTLLibrary::getMetaSource(). There are several utility methods to access information about the definition, but the basic one is OP_OTLLibrary::getDefinition().
OP_OTLManager & mgr = OPgetDirector()->getOTLManager(); int index = mgr.findLibrary( "myassets.otl" ); OP_Library * library = (index >= 0) ? mgr.getLibrary( index ) : NULL; if( library ) { for( int i = 0; i < library->getNumDefinitions(); i++ ) { OP_OTLDefinition op_def; library->getDefinition( i, op_def ); cout << i << ": " << op_def.getName() << endl; } }
Because it is cumbersome to obtain OP_OTLDefinition first before accessing name, there are a few utility methods for most common class members, which skip that step. For example, a single OP_OTLLibrary::getDefinitionName() method could be used in the above example instead of first invoking OP_OLTLibrary::getDefinition() and then following it with OP_OTLDefinition::getName().
The OP_OTLLibrary class is fairly simple, but it inherits some extra functionality from its FS_IndexFile base class. All of the inherited methods deal with data stored in an index file (see HDK_HDAIntro_Management_IndexFile below). One thing worth mentioning here, though, is that not all of the OTL file is loaded into the memory, because that would use up prohibitively large amount of RAM. Instead, Houdini reads in only the basic information about the asset definitions (e.g., name, label, and icon) and the rest of the data stays on the disc. Though, in addition to the name, label, and icon, the base class, FS_IndexFile, also reads in and stores the information about offsets and sizes of each index file section defining each asset.
There is however, one other piece of data that is very important. It is the path, OP_OTLDefinition::getPath(), which provides the information about the location of a file that contains all description of all the remaining details for this operator. This file will contain information about the parameters, handles, scripts, etc. Usually, the file path looks like "oplib:/Object/myoperator?Object/myoperator" which means "go to the file of the operator library that defines the operator 'Object/myoperator' and look for the section whose name is 'Object/myoperator'". That's because Houdini interprets 'oplib:' as the library file that contains a particular operator, and the question mark delineates the index file name from the section name (see HDK_HDAIntro_Management_IndexFile for details), and, of course, because the library file is an index file.
So, the file that describes an operator resides in one section of an operator library index file. Moreover, the file that describes an operator (ie, an HDA) is itself an index file, so we are dealing with an index file (ie, operator definition file) contained within a section of another index file (ie, the OTL file). Each section of the (inner) index file that describes an operator contains a specific piece of information about that operator. For example, an HDA definition index file has a section called "DialogScript" that defines the parameters of the operator. The fact that the operator definition file is itself an index file is the reason why HDAs can be so flexible: each HDA can decide which sections it should add to its definition file. If an HDA function is defined by a network, such an HDA will have a "Contents" section, but if an HDA function is defined by a Python script it will have a "PythonModule" section, and if an HDA is a GLSL operator, it will have both "GlslVertex" and "GlslFragment" sections.
As a side note, the file path to the HDA section that defines, for example, help will look like "oplib:/Object/myoperator?Object/myoperator?Help", which means go the OTL index file that defines an operator 'Object/myoperator' and find a section named 'Object/myoperator', which is an index file, so go and find in it a section named 'Help'.
Coming back to the members of OP_OTLDefinition class, in addition to OP_OTLDefinition::getPath() described above, there are also two methods related to an index file, OP_OTLDefinition::getIndexPath() and OP_OTLDefinition::getIndexFile(). These method correspond to the script operators (such as SHOPsurface) that are defined in external index files rather than in OTLS and they are separate and unrelated to the OP_OTLDefinition::getPath().
As an example of using the OP_OTLDefinition class, here is a fragment of code that prints the help text for all the operators it contains:
OP_OTLDefinition def; UT_String section_separator; UT_String section_name; // create an instance representing the OTL library OP_OTLLibrary lib( "myOTLlibrary.otl", "" ); // print out name and help for each definition cout << "Operators and their help:" << endl; for( int i = 0; i < lib.getNumDefinitions(); i ++ ) { // obtain the definition object lib.getDefinition( i, def ); // write out a header, including the op name and label cout << "=======================================" << endl; cout << def.getLabel() << " (" << def.getName() << "):" << endl; cout << "-----------------" << endl; // get the index section name corresponding to the operator section_separator = def.getPath().lastChar( FS_SECTION_SEPARATOR ); if( ! section_separator.isstring() ) { cout << "ERROR: no section separator found." << endl; continue; } section_name = section_separator.buffer() + 1; // skip past the '?' // create and index file instance representing the HDA operator; // Note, the 'lib' is an instance of OP_OTLLibrary that derives from // FS_IndexFile, so we can ask for its section reader. FS_IndexFile hda_file( lib.getSectionReader( section_name )); // check if there is a help section if( ! hda_file.hasSection( OTL_HELP_SECTION ) ) { cout << "ERROR: HDA has no help section." << endl; continue; } // get the section length int buflen = hda_file.getSectionDataSize(OTL_HELP_SECTION); if( buflen <= 0 ) { cout << "ERROR: HDA help section is empty." << endl; continue; } // read in the help section and print it out char * buffer = new char [buflen + 1]; hda_file.readSection( OTL_HELP_SECTION, buffer ); buffer[ buflen ] = 0; // add string terminator cout << buffer << endl; delete[] buffer; }
The above code accesses the HDA's FS_IndexFile by obtaining a section reader using library's method FS_IndexFile::getSectionReader(). However, when working in Houdini HDK, it is often possible to use a simpler way of accessing operator's index file, using OP_Operator::getIndexFile(). This method is recommended for obtaining an index file from an operator, because it does some extra processing, such as making sure the correct OTL file is used and that all necessary sections have been properly initialized.
OP_OperatorTable * op_table = OP_Network::getOperatorTable( OBJ_TABLE_NAME ); OP_Operator * op = op_table->getOperator( "myopname" ); FS_IndexFile * hda_file = op ? op->getOTLIndexFile() : NULL;
The FS_IndexFile pointer returned by OP_OPerator::getOTLIndexFile() should not be deleted because it is owned by the OP_Operator class. This class deletes the pointer when you call OP_Operator::clearOTLIndexFile() or in the destructor. Invoking OP_Operator::clearOTLIndexFile() forces the operator to clear the cached information and reload it from the OTL. This is done, for example, when OTL has changed and there is a new HDA definition for this operator. Usually, though, there is not need for you to invoke it explicitly.
To rehash the relationship described so far, OP_OTLManager has a list of OP_OTLLibraries, and OP_OTLLibrary, in turn, has a list of OP_OTLDefinitions. When an OP_OTLDefinition is current, it corresponds directly to OP_Operator. The OP_Operator is owned by an OP_OperatorTable, which you can obtain from the global list of tables using OP_Network::getOperatorTable() given the table type. Each table corresponds to the network context, so there is a table for object, sops, pops, etc. Please, refer to OP_Node.h file for a complete list of the table type names that you can use. The OP_Operator, in turn defines all nodes of its type that exist in the current Houdini session, which you can obtain with OP_Operator::getNumActiveNodes() and OP_Operator::getActiveNode(). The following code makes use of these methods to calculate the total number of nodes created from the definitions contained in a given OTL library:
UT_String otl_name( "myassets.otl" ); OP_OTLManager & mgr = OPgetDirector()->getOTLManager(); int index = mgr.findLibrary( otl_name ); OP_Library * library = (index >= 0) ? mgr.getLibrary( index ) : NULL; UT_String table_name, op_name; OP_OperatorTable * table; OP_Operator * op; int total_node_count, node_count; int total_unsync_count, unsync_count; if( library ) { total_node_count = 0; total_unsync_count = 0; for( int i = 0; i < library->getNumDefinitions(); i++ ) { // get the table name, this is a shortcut for obtaining // the OP_OTLDefinition and invoking getOpTableName() on it table_name = library->getDefinitionOpTableName( i ); table = OP_Network::getOperatorTable( table_name ); // get the opertor name, this is a shortcut for obtaining // the OP_OTLDefinition and invoking getName() on it op_name = library->getDefinitionName( i ); op = table ? table->getOperator( op_name ) : NULL; // check if the library does indeed define this op; // it may be that some other library that does. if( op && library != op->getOTLLibrary() ) { cout << "There is an operator " << op_name << " but " << otl_name << " does not define it, thus not including it in the count." << endl; continue; } // get the node count node_count = op ? op->getNumActiveNodes() : 0; // get unlocked (unsynch) count unsync_count = 0; if( op ) for( int j = 0; j < node_count; j ++ ) if( !op->getActiveNode(j)->getMatchesOTLDefinition() ) unsync_count++; // print status cout << "There are " << node_count << " nodes of type " << op_name << ", and " << unsync_count << " of them are unlocked. " << endl; // add to the total total_node_count += node_count; total_unsync_count += unsync_count; } // print the total cout << "There are " << total_node_count << " nodes defined by the HDAs in the 'myassets.otl'" << " library, and " << total_unsync_count << " of them are unlocked." << endl; }
To go the other direction and to obtain the operator from a node, you can use OP_Parameters::getOperator() method, remembering that OP_Node derives from OP_Parameters. Then, you can use OP_Operator::getLibrary() and OP_Operator::getOTLDefinition() to obtain the information about the OTL and the HDA. The code below demonstrates how to use these methods. It checks which library defines the given node:
// first, obtain a pointer to a node with path "/obj/mynode" OP_Node * node = OPgetDirector()->findNode( "/obj/mynode" ); // get the operator that defines that node OP_Operator * op = node ? node->getOperator() : NULL; // get the library for that operator OP_OTLLibrary * otl = op ? op->getOTLLibrary() : NULL; // and finally, get the obtain the OTL file if( otl ) cout << "The node " << node->getName() << " is defined by OTL " << otl->getSource() << endl; else cout << "Could not find a node, operator, or the operator is " << "not defined by any OTL." << endl;
A node in Houdini can be synchronized or unsynchronized (locked or unlocked). A node is synchronized (locked) when it should exactly match the HDA definition, and it is unsynchronized (unlocked) when it can deviate from the current HDA definition. The unsynchronized nodes are used for modifying HDA defnition, by editing the network of the unsynchronized node first and then saving it as the contents section of the HDA. You can check the status of a node with OP_Node::getMatchesOTLDefinition(), which returns true if the node is synchronized and false when it is not. It is worth mentioning, that it is meaningful to speak of node being synchronized or unsynchronized only for the operators that have contents section. That is, for the operators that return true in OP_Operator::hasContentsSection() to indicate that there is an HDA section that describes a node network defining what the HDA does (ie, its procedural operation). Otherwise, OP_Node::getMatchesOTLDefinition() is always false.
Note, however that the node network stored in the contents section is not equivalent to the OP_Operator::getDefiningNetwork(). This method refers to situations when an operator is defined by a non-HDA network in the current Houdini session, for example, that method would return a specific VOP network that defines some VOP operator.
In general terms, if OP_Operator::getOTLLibrary() returns non-NULL, that operator is defined in an OTL library. In addition, if OP_Operator::hasContentsSection() returns true, that operator is defined by a network that can be edited and saved. On the other hand, if OP_Operator::getScriptIsPython() is true, then the HDA is a custom Python operator, etc. GLSL shaders are flagged as VEX shaders, with OP_Operator::getScriptIsVex().
A newer, and now default, method is to use an environment variable to search for the OTLs themselves. The variable name is HOUDINI_OTLSCAN_PATH and by default is equal to HOUDINI_PATH with "/otls" appended to each directory in that path. In Operator Type Manager dialog, you can configure which of these two methods Houdini should use.
However, even if Houdini is to scan for OTL files, the OPlibrary is still indirectly useful for manually loading HDAs into Houdini session, which is then saved in .hip files. In File > Install Digital Asset Library... dialog, the user specifies whether the OTL is to be installed to the current hip file. And if so, Houdini will write an .OPlibrary CPIO packet that lists the OTL file being installed. Then, when Houdini reopens the hip file in the future, it knows which additional OTL files to load, because, the hip file most likely contains instances of these manually loaded HDAs.
These sources that tell Houdini which OTL files to load are called metasources in the HDK. They usually represent a concrete OPlibrarary files, but there are a few special metasources, listed in OP_OTLLibrary.h. As mentioned, OTL_INTERNAL_META refers to the OPlibrary saved within a .hip file. Additionally, there is OTL_OTLSCAN_META which is an abstract metasource referring to the fact that the OTL library was loaded because it appeared in the specified scan path. Finally, OTL_FALLBACK is an abstract metasource for HDAs whose OTL files could not be located, for which only the dialog script is available because it was saved inside a .hip file.
All that has relevance to the OP_OTLLibrary::getMetaSource() method, which returns the OPlibrary, or an abstract metasrource, that describes how the library got loaded into Houdini. To get a descriptive name of a metasource for a particular library, you can use OP_OTLManager::getMetaSourceName() methods. Removing a metasrouce with OP_OTLManager::removeMetaSource() will uninstall the libraries it was responsible for.
You can also find out the library that has the newest modification time. You can do that with OP_OTLManager::getLatestLibrary(), that takes an OP_OTLLibrary pointer that gets returned if it turns out to be the latest library among a few other contendors that have the exactly same modification time.
The following example lists all the libraries that define an operator:
OP_OTLManager & mgr = OPgetDirector()->getOTLManager(); OP_OTLLibrary * current_lib = mgr.getPreferredLibrary( "Object", "myhda" ); OP_OTLLibrary * lib = current_lib; do { if( lib ) cout << "defined by: " << lib->getSource() << endl; lib = mgr.getNextLibrary( "Object", "myhda", lib ); } while( lib != current_lib )
Schematic representation of an index file.
FS_Reader reader( "my_directory/my_index_file.idx?my_data" ); UT_IStream * stream = reader.isGood ? reader.getStream() : NULL; if( stream ) UTcopyStreamToStream( *stream, cout );
The OP_OTLLibrary class actually derives from the FS_IndexFile. Each section of that file contains the data for a single HDA. There is also a special section called "INDEX_SECTION" that contains some basic info about the HDAs in that library. This avoids the need to search and parse the HDA section to find out this information.
Schematic representation of an OTL file.
"table_name/operator_name". Since both OTL and HDA are saved as index files, we are dealing with nesting, and to get to an HDA section, we need to specify two sections, one for the OTL and the other for the HDA, for example, "directory/file.otl?table_name/operator_name?HelpUrl".You can easily obtain the index file associated with OTL or an HDA by specifying the file path:
OP_OTLLibrary* library; OP_OTLDefinition hda_definition; // ... get library and definition ... FS_IndexFile otl_index_file( library.getSource() ); FS_IndexFile hda_index_file( hda_definition.getPath() );
OP_OperatorTable * op_table = OP_Network::getOperatorTable( OBJ_TABLE_NAME ); OP_Operator * op = op_table->getOperator( "myopname" ); FS_IndexFile * hda_file = op ? op->getOTLIndexFile() : NULL; if( hda_file ) for( int i = 0; i < hda_file->getNumSections(); i ++ ) cout << "Section " << i << ": " << hda_file->getSectionName(i) << endl;
// get the index file for a given HDA OP_Operator * op = OP_Network::getOperatorTable( OBJ_TABLE_NAME )-> getOperator( "myopname" ); FS_IndexFile * hda_file = op ? op->getOTLIndexFile() : NULL; // add geometry file to the asset; after that, this geo file can be referred // to as 'opdef:/Object/myopname?geodata.geo' in SOP File, for example. if( hda_file ) hda_file->addSection( "geodata.geo", "mygeos/box.geo" ); // remove data in section "someimage" that is no longer needed if( hda_file ) hda_file->removeSection( "someimage" ); // now, write out the changes to disk OP_OTLManager & mgr = OPgetDirector()->getOTLManager(); OP_OTLLibrary * lib = op->getLibrary(); OP_OTLDefinition hda_def; lib->getDefinition( lib->getDefinitionIndex( "Object", "myopname" ), hda_def ); mgr.addToLibrary( lib, hda_def, hda_file ); /* or alternatively: FS_Writer writer( hda_def.getPath() ); mgr.writeOutDefinitionContents( hda_def, *hda_file, *writer.getStream() ); */
A corresponding method, OP_OTLManager::removeFromLibrary() removes an HDA definition from the library. And another, OP_OTLManager::mergeLibraries(), combines two libraries into one, and deletes the secondary library instance.
void listLibrary( const char *libfile ) { OP_OTLManager & mgr = OPgetDirector()->getOTLManager(); OP_OTLLibrary lib(libfile, ""); OP_OTLDefinition def; UT_String section_name; UT_IntArray nc_assets; int i, j; // get the indices of non-commercial assets mgr.getNCAssets( & lib, nc_assets ); // print the library information cout << "Operator type library " << libfile << " contains:" << endl; for( i = 0; i < lib.getNumDefinitions(); i++ ) { UT_WorkBuffer buffer; // print the HDA information lib.getDefinition( i, def ); def.writeTextDescription( buffer ); cout << buffer.buffer() << endl; // make a note if its a non-commercial asset if( nc_assets.find( i ) >= 0 ) cout << "NOTE: this is a non-commercial asset." << endl; // list the HDA setions cout << "Sections:" << endl; def.getTableAndName( section_name ); { FS_IndexFile indexfile( lib.getSectionReader( section_name )); for( j = 0; j < indexfile.getNumSections(); j++ ) cout << " " << indexfile.getSectionName(j) << endl; } cout << endl; } }
void embedUsedHDAs() { OP_OTLManager & mgr = OPgetDirector()->getOTLManager(); OP_OperatorTableList tables; OP_OperatorList ops; OP_Operator * op; OP_OTLLibrary * tmp_lib; OP_OTLLibrary * internal_lib; OP_OTLLibrary * op_lib; int internal_index, hda_index; UT_String op_name, table_name, section_name; OP_OTLDefinition hda_def; int i, j; // create a temporary library tmp_lib = new OP_OTLLibrary(); tmp_lib->setDescription("HIP Internal Operator Definitions"); // find an internal library internal_index = mgr.findLibrary( OTL_INTERNAL, OTL_INTERNAL_META ); internal_lib = (internal_index >= 0) ? mgr.getLibrary(internal_index) : 0; // iterate through tables and obtain operators from each of them OP_OperatorTable::getAllOperatorTables( tables ); for( i = 0; i < tables.entries(); i ++ ) { // find out the table name table_name = tables(i)->getName(); // get operators from the table ops.resize(0); tables(i)->getOperators( ops ); // iterate through the operators and add their HDA to temp library for( j = 0; j < ops.entries(); j ++ ) { // get the op and its op = ops(j); op_name = op->getName(); // if this operator has not instanced any nodes in the session, then // hip file does not need it if( op->getNumActiveNodes() <= 0 ) continue; // get the library that contains an HDA for this op; if there is no // library then there is no explict HDA definition to save op_lib = op->getOTLLibrary(); if( !op_lib ) continue; // get the index of this operator's HDA in the library hda_index = op_lib->getDefinitionIndex( table_name, op_name ); if( hda_index < 0 ) continue; // if the HDA for this op is already in the embedded library, // then there is no need to re-add it if( internal_lib && internal_lib->getDefinitionIndex( table_name, op_name ) >= 0 ) { continue; } // no need to save a dummy definitions if ( op->getIsDummyDefinition() ) continue; // get the reader of the HDA data op->getTableAndName( section_name ); FS_Reader * reader = op_lib->getSectionReader( section_name ); if( ! reader ) continue; // create a new index file for the HDA back-up FS_IndexFile * hda_def_file = new FS_IndexFile( reader ); if( !hda_def_file ) continue; // Write out the HDA contents to a stream that can be used // for adding a definition to the library UT_OStrStream defos( hda_def_file->guessStreamSize() ); hda_def_file->writeFile(defos); // get the HDA definition from the library op_lib->getDefinition( hda_index, hda_def ); // Add the definition to the library. tmp_lib->addDefinition( hda_def, hda_def.getModTime() ); tmp_lib->addSection( section_name, defos ); // clean up delete hda_def_file; } } // finally, add the collected HDAs to the internal library; the following // call also deletes the passed in tmp_lib. if( tmp_lib->getNumDefinitions() > 0 ) mgr.addToInternalLibrary( tmp_lib ); else delete tmp_lib; }
1.5.9