|On this page|
This node creates work items that download content from a static list of URLs into files on disk. You can choose to batch all downloads into one work item (for small downloads), or launch separate processes for each download.
You can use a
file:// URL to copy the contents of one file to another. (See also the File Copy node).
You will usually want to override this node to run on a local scheduler, and leave Split downloads into separate work items turn off, since in general, the overhead of scheduling and launching separate processes dwarf the download time unless the files are very large. However, you can try testing with the split checkbox on and off.
Work Item Generation
Whether this node generates static or dynamic work items. You should generally leave this set to "Automatic" unless you know the node’s work items can be computed statically, or that they need to be generated dynamically.
This node always creates dynamic work items: it waits until the upstream work items are known, and generates new work items from the upstream work items.
This node always creates static work items: it creates the number of work items it thinks it needs based on the parameters (and any upstream static items) before the network runs.
If the input is static (a static processor, or a partitioner with only static inputs, or a mapper), this node generates static work items, otherwise it generates dynamic work items.
Split Downloads Into Separate Work Items
Create separate work items for each URL to download them in parallel. When this is off, the node creates one work item to download the URLs serially. You may want to turn this on if you are downloading very large files.
Use this multiparm to set up the URLs to download.
The URL of the file to download.
The file tag to apply to the output files.
The file path to download the content into. If you turn off the checkbox, the node will pick a name based on the URL’s base name and place it in the working directory.
This example demonstrates how to download a file using the downloadfile node.
The following examples include this node.