|On this page|
This node creates work items that download content from a static list of URLs into files on disk. You can choose to batch all downloads into one work item (for small downloads), or launch separate processes for each download.
You can use a
file:// URL to copy the contents of one file to another. (See also the File Copy node).
The downloads are done in-process but can be during generation or cook.
List of urls to be downloaded.
1 to turn on SSL cert verification, 0 to disable.
Work Item Generation
Whether this node generates static or dynamic work items. You should generally leave this set to "Automatic" unless you know the node’s work items can be computed statically, or that they need to be generated dynamically.
This node always creates dynamic work items: it waits until the upstream work items are known, and generates new work items from the upstream work items.
This node always creates static work items: it creates the number of work items it thinks it needs based on the parameters (and any upstream static items) before the network runs.
If the input is static (a static processor, or a partitioner with only static inputs, or a mapper), this node generates static work items, otherwise it generates dynamic work items.
How the processor node handles work items that report expected file results.
If the expected result file exists on disk, the work item is marked as cooked without being scheduled. If the file does not exist, the item is scheduled as normal.
If the expected result file exists on disk, the work item is marked as cooked without being scheduled. Otherwise the work item is marked as failed.
Work items are always scheduled and the excepted result file is ignored, even if it exists on disk.
When to download the files.
The download will happen during the work item generation stage. When this is on, the file is downloaded whenever the node generates items (when the network starts cooking, or when you explicitly ask to generate static items).
The download will happen during the cook and within the graph cooking process.
Split Downloads Into Separate Work Items
Create separate work items for each URL to download them in parallel. When this is off, the node creates one work item to download the URLs serially. You may want to turn this on if you are downloading very large files.
Use this multiparm to set up the URLs to download.
The URL of the file to download.
The file tag to apply to the output files.
The file path to download the content into. If you turn off the checkbox, the node will pick a name based on the URL’s base name and place it in the working directory.