Now for my opinion on using variations of the Cornell Box as a real world render test. I wonder what the Cornell team would think of this thread (hint: probably not much at all).
First of all the Cornell Box “test” is to allow you to compare how accurately a Render engine can match up with a real scene with real objects and real lighting:
http://www.graphics.cornell.edu/online/box/ [
graphics.cornell.edu]
Very valid 5+ years ago. These days? Arguably not as critical although still a worthy test if you want to test physical accuracy of the BSDF models more so and the light transport of a render engine.
By just creating your own box and comparing render engines in a subjective matter is just that, subjective. It has no real purpose other than to argue the “subjective quality” or appeal of the various image variations. If you were to create an identical cg scene to match the actual cornell box with real surface properties, then compare the rendered image to the actual images, then that I buy.
Today, you can get very very close to the actual Cornell Box with Mantra, Arnold, VRay, RenderMan and MentalRay and many other amazing render engines with proper surface shaders and lighting.
If you want to see how fast you can render the cornell box by adding caching schemes and acceleration methods, which by the way are all lossy and approximations, and you can get close, fantastic. But they are lossy and prone to artefacts in certain situations. You are manufacturing light information by using various inspection techniques to augment a render engine's own internal light transport mechanisms of which Final Gathering is one such approximation technique.
With 64bit hardware and operating systems along with healthy amounts of memory and very fast processors and threaded render engines, you can get decent render times by brute force raytracing such as Arnold, Mantra, latest RenderMan and other physically based render engines. VRay has a lot of various shema as well and is also right up there quality wise. The render landscape is changing rapidly all in a positive way. With Houdini you have Mantra. Fantastic amount of choice.
As for the render tests above from Korny, I hate to say but compared to the real cornell box, it isn't even close and don't argue that as it is objective. Just the hard pencil like lines on the corners are not physically plausible unless you took the equivalent of a marker and painted that on the wall intersections on the real cornell box, of which the profs would not be very happy as you defaced a quite famous artefact. There are other questionable artefacts as well.
Again it is all subjective and if you meet your client's quality requirements, fantastic! Everyone's happy, you get paid and move on. This is the real argument and there isn't an argument to be had imho. If everyone is happy with what they are getting, great. Just remember that the goal posts are indeed moving in the render world and clients, even those that aren't as sophisticated will notice and will eventually request quality improvements.
These days if you are comparing render engines, you need to present a variety of different scenarios involving indoor and outdoor lighting. Objects with lots of fine detail. Instancing vs. everything unique, etc.
In film it is not unheard of to have upwards of 1000 to 100k props in your scene. Some props may have as many as 10-40+ 8k to 12k texture maps being referenced by the shaders, with and without displacements with DOF and Motion Blur all in camera. That new normal is also being hit in high end commercial shots more often these days.
Another slowly emerging trend is the necessity to render everything in a single pass including large amounts of geometry, volumetric data and more. Light bleeding in to volumes which in turn scatter light on to surrounding geometry with a physically based render engine in the hands of a good lighter are phenomenal. More and more approvals are happening in Lighting but it is still early days. Eventually it will come to pass that the majority of approvals will be in the Lighting phase.
Interesting days ahead for sure. Although not as controversial, it reminds me of the off-line vs on-line wars of the late 1990's post houses where now you have: render passes or all-together, Comp up to beauty vs beauty minus adjustments, and everything in-between.
Stay flexible and learn as many render engines as you can, including Mantra.
And don't forget to pad the budget a bit to pay for the next bit of kit to take advantage of the various renderers and their vastly improving architectures and capabilities. Take that for what you will.
