|On this page|
This node will create a work item for each row in the input CSV file (but see the Limit rows parameter). It will extract data from certain columns you specify into attributes on the work items, or all columns automatically.
This can be useful to allow users to specify a list of work to do.
The opposite of this node (write attributes on work items into a CSV file) is the CSV Output node.
By default, if this node has input files, it creates a work item for each incoming row, but it does not extract any data. You need to decide how to provide it with CSV data, and specify which columns to extract.
Specify where to get the CSV data. Do one of the following:
If you the input work items have CSV files in their result data, select Upstream Result Files.
For example, if you get a list of files using a File Pattern node, you should add a Wait for All after the file pattern, and then wire the Wait for All into this node. Assuming the File Pattern node only matches
*.csvfiles, set it to tag its output files as
file/csv. Finally, on the CSV Input node, select Upstream Result Files.
If you want to statically read in one CSV file, select Custom File Path and set the File Path to the path of the static CSV file.
Choose what to extract. Do one of the following:
If you want to extract data from individual columns, use the Columns to Extract multiparm to specify the columns to extract and the attribute names to use. If your CSV file has a header row, turn on Has Header Row to tell the node to ignore it instead of using it as data.
If you want to extract all columns, turn on Extract all. If your CSV file has a header row, turn on Has Header Row to tell the node to use the header row to name the attributes. If the CSV file has no header, the node will make up attribute names like
column_1, and so on.
Using Extract all extracts all values as strings. If you want to properly type numbers in the CSV file, you must extract columns manually instead.
You can try generating the items in the node and check their attributes.
If there are multiple input files (from upstream item result paths), this node creates separate work items for each row of each file (however set Limit Rows).
To ensure you can cook the network on a render farm, make sure the input CSV files are in the shared network filesystem.
The index of the row in the CSV file that the work item is created from. This is 0-based, and does not include the header row if there is one.
An array of the names of attributes that were extracted from the CSV row. This attribute is optionally created, and can be used by CSV Output node.
How do decide what file(s) to delete:
Custom File Path
Specify the path of a single CSV file to load from (this field does not support file patterns).
Upstream Result Files
output file paths of the incoming work items as the file(s) to load. If you specify a file tag (usually
file/csv), the node will only use files with that tag.
When Custom File Path is selected, specify the path of a single CSV file to load from (this field does not support file patterns).
When File is "Upstream Result Files", delete paths in the
output attributes of incoming work items if they have this file tag.
Has Header Row
Turn this on if your CSV file(s) has/have a header row (containing column names). If you extract individual rows, the node uses this to ignore the header row (otherwise it will take it as data). If you select Extract all, the node will use the header to name the attributes.
Artificially limit the number of work items this node creates. The node reads the input rows sequentially as usual but simply stops at the given number of work items. This might be useful if you're worried about huge CSV files might be produced accidentally by a runaway process, or if CSV files are generated by humans and there’s a convention that they may not contain more than a certain number of items. However, you should remember that this setting can make the node silently discard legitimate data.
Work Item Index
Select how work item indexes are set when generating from an upstream item.
Use the CSV row index as the work item index.
Use the upstream item’s index.
Turn this on to explicitly set the column-separation character. If this is not turned on, the node will try guess the delimiter character from the first few rows of the file.
Add Columns Attribute
Turn this on to specify the name of a string array attribute that will be created to store the names of the columns that are extracted from the CSV. The order of the names will match the columsn in the CSV. This can be used in CSV Output node
Select the method of specifying column extractions.
By Column Index
You will specify the 0-based index of columns to be extracted in Columns to Extract.
When this is on, the node copies all columns into work item attributes. The attributes created are always string attributes. If you want to properly type numeric data, you should use another mode and extract columns "manually". If Has header row is on, the node uses the column headers to name the attributes. If Has Header Row is off, the node will make up attribute names like
column_1, and so on.
By Column Name
You will specify the header name of the columns to be extracted in Columns to Extract.
You will supply the name of an attribute to read column names or indexes from in Input Columns Attribute.
Input Columns Attribute
When Extraction is set to By Attribute, specify the name of an attribute to read column names or index from. This attribute can be a string or integer array. Column names must match the header in the CSV file, and indexes are 0-based from left to right.
Columns to Extract
When Extraction is set to By Column Index or By Column Name, use this multiparm to set up mappings between input column names or indexes to and attribute names/types.
CSV Column Index
The column number to read the data from. The first column is number
0, the second column is number
1, and so on.
CSV Column Name
The column header name from the CSV file to read data from.
The name of the attribute to assign the value to.
Attributes can have multiple values (to accomodate vectors and lists). You can set this to control at what position the value is inserted in the attribute value list.
The data type of the extracted value (string, integer, or float).
Work Item Generation
Whether this node generates static or dynamic work items. You should generally leave this set to "Automatic" unless you know the node’s work items can be computed statically, or that they need to be generated dynamically.
This node always creates dynamic work items: it waits until the upstream work items are known, and generates new work items from the upstream work items.
This node always creates static work items: it creates the number of work items it thinks it needs based on the parameters (and any upstream static items) before the network runs.
If the input is static (a static processor, or a partitioner with only static inputs, or a mapper), this node generates static work items, otherwise it generates dynamic work items.
This example demonstrates how to read in data from CSV files. The CSV Input node will read CSV files and create 1 workitem per row.
The following examples include this node.