Please read the HQueue Cloud Computing User Guide to learn how to get started using Houdini in the cloud.
If you have questions or are having issues using the cloud tools, feel free to post a message on the HQueue Cloud Public Beta forum. You may want to search the previous messages first before posting as your question or issue may have already been solved.
For artists, the large up front costs needed to set up a render farm are often prohibitive. With access to cloud computing, you don't need to purchase, rack, power, cool, upgrade, and maintain lots of computers. This means that you are left with more time to focus on your art.
For studios, the costs of maintaining a local render farm are a part of doing business. But anticipating when peak usage with occur and being prepared to scale up during these times can be a challenge. By rendering to the cloud, you can avoid the fixed costs of a render farm big enough to handle peak usage. Instead, you can scale your costs based on your actual usage then use the cloud when you get busy.
Rendering to the cloud from Houdini is super easy. Simply click a button and Houdini starts computers in the cloud, uploads your input files, renders them in parallel, and downloads the results.
In Houdini, you identify the input files needed by your render project and you choose the number of machines you want to render with. Houdini starts up machines in the cloud, uploads your project to them, processes your files, then downloads the resulting files. You can monitor the render progress using a dedicated web interface, and you can shut down the cloud machines at any time using Amazon's web console. Once the render is finished, you can continue to use the cloud machines or you can shut them down.
All Houdini users can use the cloud: commercial users, Apprentice HD users, and Apprentice users. The same conditions apply in the cloud as they do normally: Apprentice users have size restrictions and a watermark on renders and Apprentice HD users have a size restriction for image sequences.
All of them, but the cloud will be most beneficial to small studios and Apprentice HD/Apprentice users, since big studios have complex pipelines, environments, custom HDK operators and firewalls. Small studios and Apprentice users likely don't have access to a farm, which makes the cloud appealing.
The HQueue Cloud Computing tools are designed to protect the security of your data. All data you send into and out of the cloud is encrypted and transferred directly to and from Amazon, not Side Effects. Inside the cloud, the machines you are renting are completely isolated from other cloud users. From outside the cloud, only your IP address may connect to the machines you're renting, and only ssh and HQueue connections are allowed from that IP address.
On Amazon's EC2 Instance Types page you can find detailed information about a wide range of computers. After testing each type of computer, we have chosen to support only those machines that offer the best value when using Houdini. We want you to get the best value in terms of processing power per hourly price, so we have removed support for the machine types that don't provide as much value.
The Houdini prices include both the Amazon fees and licensing fees for the use of Houdini Batch and Mantra in the cloud. By paying for this licensing separately, your local Houdini licenses remain available for content creation while the cloud processes your scene files.
Houdini 10 and up will support the cloud. If you are using Houdini 10, it is best to use the current production release because this is the release that will be used in the cloud.
Amazon currently limits each customer to a maximum of 20 running machines (instances) at a time. However, you can request more machines from Amazon.
Yes. If your first job finished uploading and has started rendering, your next job will start uploading immediately and will automatically render after the first one ends. If your first job is still uploading, your next job is scheduled for upload immediately after.
No, not currently.
No, not currently, but we may support them in the future. You can do a simulation as part of the render, or use a Geometry or Dynamics [DOP] render operator to cache out the simulation. However, the simulation portion will only run on one machine in the cloud.
Mantra, Dynamics [DOP], Geometry, and Composite [COP] ROPs can be used in the cloud.
No, not currently.
No. Even if you own licenses for another renderer, using them in the cloud would likely be in violation of your user agreement for that renderer.
Yes, but you will be billed the standard Houdini cloud rates. Your one-time reserved instance payment to Amazon does not include the cost of running the Houdini instance.
No. Your Houdini license agreement does not permit this.
No. All files must reside in the cloud to work properly. To make sure that files are uploaded correctly and that their file reference path will be understood in the cloud, you can either pre-flight the scene file or at render time use the File Dependencies window to manage your files.
Files will be automatically uploaded every time you press render. Any changed files will be uploaded. If the cloud machine has shut down then all the files will be uploaded.
Use the Amazon AWS Web Console to see if any cloud machines (instances) are running. You can easily shut them down from the web console by pressing the Terminate button.
It can take Amazon up to 10 minutes to find a free machine in the cloud and wait for it to boot. Usually, though, machines boot in a few minutes. When launching a job in the cloud, Houdini has to first wait for the server machine to start before it can start the rest of the machines. You can check the Amazon Web Console to verify that your cloud machines are being started.
To save you money, cloud machines shut themselves down when inactive for too long. Amazon bills by the hour for machine usage and rounds up, so cloud machines shut themselves down if they are inactive and they approach a one hour boundary. You can also terminate them manually to make sure that they are shut down properly. You can easily start up a new set of machines for your next render job.
If you have an Annual Upgrade plan then you may contact support by email. Otherwise you should post your concerns to the Houdini forums.
cloudsubmit.log file contains log entries from the process that starts the cloud machines, uploads files to the cloud, and downloads the results. If something went wrong, you may find an error entry there. This file is located in your temporary directory (e.g.
/tmp on Linux).
No. When using the cloud you are paying for the use of the cloud computers and not for the results. You must manage your projects to make them successful in the cloud.
No. You are paying for the use of the cloud computers and not for the results. It is important to test your files with the production build to avoid issues. If you are using the 32-bit machine you may want to use the 64-bit machine instead because it has more memory.
It is difficult to say because this depends on the size of your files and the speed of your internet connection. The Amazon side is quite fast and should not be a bottleneck for you.
When you terminate a machine and shut it down, your files are removed to make room for another Amazon EC2 customer.
No. In the HQueue render node, the hourly rate is tallied depending on the machine you use and the number of nodes you want. The final money spent will also depend on how long your shot takes to process. If you need to keep under a set budget then you will need to manually keep tabs on your render times. Test rendering beforehand can help you optimize your shots.
The File Dependencies Window (available via the pre-flight menu option) is one way to make sure your files are set up properly. Test renderings both locally and in the cloud are the best way to understand what will happen before committing to a big rendering.
To ssh to a cloud machine, you need to know the path to your ssh key and the machine's name. Amazon uses passwordless ssh keys which are stored in your
$HOME/houdiniX.Y/aws directory (
$HOME/Library/Preferences/Houdini/X.Y on the Mac), where
X.Y is the Houdini version. The key is named
userid-ssh-key.rsa and corresponds to the instance keypair name that appears in the Amazon AWS Web Console beside the instance. You can find the cloud server machine name by looking at the Public DNS field in the Amazon Web Console. To ssh in using the key, run:
ssh -i $HOME/houdiniX.Y/aws/userid-ssh-key.rsa root@ec2-XXX-XXX-XXX-XXX.compute-1.amazonaws.com.