Spider-Man: Into the Spider-Verse | Sony Pictures Imageworks

With Into the Spider-Verse, the filmmakers were looking to directly reference comic book panels in telling their story. In some ways, this was the antitheses of much of what Imageworks had been crafting on previous shows, both in terms of animated films and in visual effects, which was photorealism or at least material photorealism. Here they needed to move away from that concept.

That did not mean effects simulations would not be used in the film; rather, notes Grochola, “it was clear we couldn’t count on simulation algorithms alone to be the design. Furthermore, photorealistic simulations simply wouldn’t fit the language of the film. Houdini gave us the freedom to easily manipulate our effects to ensure we nailed the design.”

“Houdini has a toolbox approach to problem solving,” adds Grochola. “If the tool doesn’t exist to solve a technical challenge, Houdini provides all the building blocks necessary to easily build that tool.” 

For that reason, Imageworks look to craft unconventional effects still through Houdini, and Into the Spider-Verse certainly had highly unconventional ones. These included ‘glitching’ characters and buildings, a ‘multi-verse’ world, heavily stylised rigid body dynamics, custom sprite systems, 2D-inspired designs, and a need to highlight individual character ‘lines’ on 3D modelled characters.

Explosions and particle effects 

There are several moments in the film that involve explosions or particle effects, all of which have this said mix of a 3D and 2D feel. Inspiration for this effects work came, says Imageworks FX supervisor Ian Farnsworth, from 2D hand-drawn anime. “We had initially tried doing some smoke or explosions using some simple particles and spheres, and then having them erode or evolve in interesting ways using solvers and shaders. While the results looked really cool, they never really hit the mark, and definitely never felt stylised or hand-drawn enough. They always had this element of ‘CG’ to them, and while we feel given enough time we could potentially push it a lot farther, we decided to look for a 2D FX artist to help us out.”

That artist was Alex Redfish, whose Vimeo demos exemplified a style that would fit the desired language of the film. He provided Imageworks with a small re-usable library of 2D FX, starting with an explosion. “We had him provide as many layers as he could for us, so we could mix and match as needed, and also place things in depth for the 3D stereo side of things, too,” details Farnsworth. “He was able to provide us with a few explosion variations with multiple layers each, gun muzzle flash shapes, spark hit shapes, smoke, steam and flame elements.”

At Imageworks, an HDA in Houdini was created that would allow artists to select images from the library and then view the animation in real-time in the viewport. The same HDA also output a separate geometry for render. “Being able to view the animations in real-time was really important to us,” says Farnsworth, “so we created sprite sheets for all the animations we were going to use. This is a common technique used in games, where instead of loading a specific frame of animation for every card, you only have to load a single image that has all the frames on it already and you modulate the UVs to switch frames.”

With that setup in place, the FX artists could use simple particle systems or hand-place these elements as needed, and focus on the timing and composition of the elements, without having to worry too much about rendering or the final look, since the render would match the viewport almost exactly. “The render side was nice and simple,” notes Farnsworth, “as we simply stored attributes for the hi-res image path, and simple things like age or alpha attributes so we could control a bit in the shader as well.”

“Using 2D animation as inspiration,” adds Farnsworth, “we also did some fun timing experiments on rigid bodies, outputting the geometry on 2’s, and sometimes switching from 1’s to 2’s or vice versa to get us a unique ‘stepped’ look that matched what the character animation team was doing. We used that in combination with some minor re-timing to really create a nice ‘punch’ and accentuate the action as desired. We also had to create a few custom boolean-based tools for creating stylised patterns and cuts. This is just simply not possible in another program other than Houdini.”

The film’s final battle sequence also utilised Houdini to fill up a volume of space with ‘Kirby dots’ - inspired by the Marvel artist Jack Kirby. Says Farnsworth: “These sequences were a big practice in composition and timing more than FX setups, however, Houdini made the actual process of iteration really fast. The setup used allowed the artists to create and place these ‘streams’ of particles, and then tweak and modify the internal motion / bubbling using just a few nodes. Once we had one shot nailed we could easily copy over some of the settings from that shot to other similar shots.”

Glitch effects

The development of a particle accelerator by the character Wilson Fisk opens up parallel universes (and brings in alternate versions of Spider-Man) into our universe. But it also causes almost catastrophic glitching on buildings and in several characters.  

Inklines

A key ingredient to the comic book look were line drawings on the characters, also known as ‘inklines.’ Early in the development process, Imageworks tried out a number of different solutions for creating inklines, but soon discovered that any approach that created inklines based on ‘rules’ - for example, toon shaders - didn’t look hand-drawn.