There are two parts to the problem, the shading technique (have a look at toon shading on the web and be prepared to have a tsunami of new concepts) and then the geometry that by when using the shader produces that look in a coherent way from frame to frame (lines not appearing and disappearing)
It is not trivial and I would recommend to have a look at Softimage toon shader who is amongst the best out there. (or should I say was)
In any case, it is not a small project to achieve something to organic as that reference.
As Jordi mentioned a toon shader could move you in the right direction?
Please find enclosed a very rough setup for how to to get you started - note I just used the default billowy smoke simulation setup - you will want to drive your volume in a way to generate those whirls which are a key feature of this look. as you will notice my volume is way too nosey and has features that are far too high frequency
maybe have a look at vorticles?
if I have a bit more time I will try to have another look for you laters it's quite a cool look to try
Thanks for the responses guys, I've managed to achieve something similar to glassman3d, which is my most successful so far. I was thinking that I could try using sprites that rotated/scaled depending that were then overlaid my original shading could work to give the impression of swirls.