Sometimes, you only have to develop a strong sense of humor to improve how you manage things. Not a cynical behavior, no, but you have to accept that ideas are just like dices: sometimes they show you the 1dot side, and when you don’t expect it, they show you the 6 dots. Let me explain that with the Sponza demo for BabylonJS.
It was not the right time
You know that. In every project, the major issue is to keep everyone focused. After the Mansion project, (http://www.babylonjs.com/Demos/MANSION/), I was facing a choice: Should I carry on with the BabylonJS team, and have another round with another demo, or should I try something new and try to improve myself with new side projects. I was thinking, at that time, of a serious Unity3d or UDK demo, with a very ambitious stack of assets to create, and real time shadowing for AAA gaming purposes. Got a lot of work during my daytime too (and less time at night for webgl). On the other side, the babylonJS team, was still very pushy, with a lot of new functionalities to take into account and the same passion. It was a bit hard to manage all those options, and on my side, decisions had to be made. And the dices needed to roll.
The “can you help me with that?” strategy
I was helping Julien Moreau Mathis (@luaacro) with the Actions Builder for BabylonJS, a unique way to create behaviors without a single line of code. In fact, I was creating just a simple scene with switches, in order to test the different workflows of interactions (http://www.babylonjs.com/Demos/ActionBuilder/). The same day, Julien asked for another advice and switched to the “Can you help me to understand how to light up a scene with 3dsMax” context. As the lighting process is not really a piece of cake, I suggested to work on a sample. This is where the Sponza scene by Crytek came in (http://www.crytek.com/cryengine/cryengine3/downloads/). Julien downloaded it and I began to think a lot about it. It was the beginning of a new adventure.
The initial scene
That scene...well, it was clearly not intended to run on a JavaScript engine…The purpose here was to demonstrate on an AAA engine the benefits of really high end features, such as indirect illumination and real-time radiosity. Not the kind of stuff you expect to run on a smartphone (which is the main target of a webGl engine like BabylonJS). Hooo, there will be optimization, here. Look at that: the flags, and all the fabric pieces represent more than 130 k triangles, just in the middle of the viewport (and in red in the image below).
If you agree that we had choices to make and a nice performances/visual quality to achieve, I simply choose to delete all those fabric objects and to concentrate on the visual quality of the palace itself. I just kept the upper flags with a strong face reduction in order for red photons to jump around and illuminate the walls.
Another issue there was the by-default rendering engine used by 3ds Max (yes, the file came in Max format). Mental Ray is great, but it’s absolutely not compatible with BabylonJS format. Considering that, I had to re-create all the materials in the scene and get rid of the original lighting. Good bye!
Back with the team
As a designer and artist, I always work with a context: Should the lighting be realistic? Yes, of course. Should it be real-time, or baked (rendered to textures?): why not both? Morning lighting? Evening mood ? Well, I’d like the idea of the Sponza Palace, after a calm night, with dying flames inside the fireplaces. Ok, let’s play it that way. One more night, setting up the rendering engine with my entire dedication, lighting, and environment and boom, here we are, the first rendered image.
The reaction of the team was awesome. They were very (VERY) exited by that project and have been very encouraging. At first, to me, that was a kind of joke to work on a lighning exercise that could become a master babylonJS demo scene…
Why a baked version, one more time??
Why a baked version, one more time??
Because, for the target we’re aiming, and for the quality I’m searching for, the only way to achieve good and quick results is to bake all the lighting information in the scene textures. We’ve got really convenient tools for that in 3Ds max, so it’s a very comfortable workflow. Even if it’s not very dynamic, we can also achieve some really fine results on specific topics like “blurred shadows” or “bouncing photons”. Those kind of effects are very, very hard to obtain with a real time engine (when available…).
So, basically, render the lighting to a texture is a mix of two processes. The first one is to create the convenient UV channel to support the lighting information. This is often called the UV channel flattening (ironing a 3D object so that all the mesh is constrained to a 2D surface). That could turn into a nightmare when you want the result to be very accurate, fast and relevant.
Then, using the new illumination I worked on previously, it’s quite simple: All you’ve got to do is to make serialized rendering for each object of the scene, using the high capabilities of a radiosity engine like Mental Ray, or Vray. These renderings will become textures that I add to a multiplying pipeline.
No magic here. I add as many channels as the artistic direction requires to create the material I need (a material is basically, from the artist perspective, a stack of combined textures ). The genius lies in the engine created by David Catuhe (@deltakosh) and by David Rousset (@davrous),which can combine for me those textures inside a powerful shader (a material from the engine point of view).
The material and shader creation is complete when all the objects have received their final aspect. And sometimes, you’ve got to search for a while and tweak or cheat to obtain the result you’re looking for. I added two extra lights in the scene to achieve the specular shading of the lion’s head, for instance.
When you add all these textures, you will sooner or later reach the performance wall. Too much textures, too much draw calls, download time too lazy, … It’s optimization time. Finally, old recipes are best recipes: Trying to keep multiples of height size, and remember your target: you don’t need a full sized 2048x2048 pixels texture, when your rendering target is only 1920x1080 pixels. You will hardly need more than 512x512 pixels textures in fact, often less than that. The smallest texture used in that scene is 16x16 pixels.
By the way, I also tried to give a chance to a low-end version of Sponza, reducing again and again the amount of textures by material, but that was not very convincing, as the frame rate only raised by an average 5 frames per seconds.
Another consideration was to add animations in that scene. So I add bones to deform the flags in the upper part of the scene, as they float slowly in the (supposed) wind. I also created five additional cameras for further purposes. We had then our 3D assets and job done.
Why a real-time lighting one?
Because we can! David Catuhe had once again done a amazing job, allowing me to give to the baked version a little sister, all in real-time lighting, and performing so well!!! I only tested the engine once or twice (with the very fast exportation process), added two lights in 3dsMax, tweaked them a little (lighting distance attenuation), animated one, of course removed all the baked lighting textures, and let the process roll. We had our real-time lighting demo scene in no time… And that’s amazingly fast. More to read there on that matter: https://blogs.msdn.microsoft.com/eternalcoding/2016/02/09/going-further-with-babylon-js-2-3-advanced-features/
Finalizing and polishing
But we were not fully satisfied yet. David Rousset and I were thinking about a specific launch screen, that could perfectly fit the demo. So I immediately began to storyboard screens and finally realized that mockup.
David Rousset expertise in HML and CSS was here a very precious weapon to integrate and realize the perfect responsive loading screen we were targeting. All that part of the story (and much more, like the Windows 10 tile…) is narrated here :
Should I say, that Mr. Rousset did an awesome job with the special score and music of the Sponza scene. Another great exemple of what polishing means….
David Catuhe, on his side, worked on a tool that allows to manage the cameras. The looping cameras transition inside the scene comes from that tooling. I bet that such a useful option will get integrated someway soon…
The “One more thing” chapter…
Did you notice the particles effects inside some fireplaces? I just want to let you know that this part of the job (and a bit of debugging) was done by me with an amazing tool made by Julien. No doubt that you will have news about this very soon, if you’re a BabylonJS addict… Stay tuned!
…and the conclusion goes there.
We love to plan projects, we have the habit to organize our work in tasks, our ambitions into schedules and one day, the young apprentice walks by and says that he needs an explanation about lighting. And so began the story of our most relayed demo scene so far. Isn’t that funny? That project, well, I’m so proud of it. Because it is probably the most team-played project we had within the babylonJS team, and ended up with some beautiful achievements. Rolling dices finally showed the 6 dots face.
http://www.babylonjs.com/Demos/Sponza/