By Julien Blervaque and Yannick Castaing, Ubisoft Film, and Television- Paris
Between January 2020 and March 2021, we were quite busy at Ubisoft Animation Studio (in-house studio of Ubisoft Film & Television) working on a 70min long special episode of our TV series Rabbids Invasion. This episode is named Blender and the Rabbids Invasion: Mission to Mars and it will be released at the end of the summer on France Television and on Netflix worldwide in 2022.
We handled the storyboard, previz, and editing stages internally while the following steps were done by partners studios: Supamonks, which worked mostly on Blender, Anima for a great part of the animation, and Miroslav Pilon studio for the sounds and voices. To facilitate the production the movie was divided into 3 acts of about 23’ each. We chose to use version 2.83 of Blender because it was an LTS and we needed stability on the project.
The challenge for us – apart from using Blender in this kind of big production for the first time, which was pretty exciting – was to offer to our movie director a very flexible previz environment so that he and the artists could concentrate on the quality and unleash their creativity.

Yannick Castaing (VFX Supervisor):

Before talking about this special Episode of Blender and the Rabbids Invasion I want to talk about the incubator team of Ubisoft Film & Television: we have used Blender for many tests, trailers, and small exercises that were made only to convince key people inside our studio, so we had to focused on the creativity more than on the technique.
We had to onboard different people from different backgrounds, and after a few days of training, it was easy, even with people less technical, to use Blender for their job. I do believe the best asset of Blender is to be able to break the big wall between 2D and 3D artists and make them work together easily (even if they are using the same tools differently). Furthermore, as proof that Blender is stronger than ever, the quality of what we made was on the exact same level as other DCCs.
On this Rabbids episode, we were a small team in production, mixing skills like storyboarding, layout, and look dev: 3 artists were coming from the usual storyboard department and 2 others had a 3D background. The result was so impressive that our broadcaster was amazed when they saw the previz, considering that it was much more immersive than a classic animatic.
But the consequences were bigger than aesthetic and storytelling, because Supamonks, our CG partner, had all the right data about assets, lighting, and space. Even the color scripter told us that they had their best work experience because they had low definition assets with pre-lighting giving them a solid base to iterate!
Nevertheless, this organic team suffered from the distance (because of the pandemic), and we had to find solutions in order to work and communicate well. Blender 2.8x in production was a first for some of them and created some aftermath in other steps. This version of Blender changed so many things in the workflow that people had to shift their habits and mindset to embrace it well. The price of mixing 2D and 3D can also be heavy for storyboarders, as everyone asks them to work quickly and effectively and we sometimes lost the flexibility of drawings but gain some consistency because of the 3D assets.
Our partner Supamonks was also very interested in using Blender on their side, and with my help, they went from zero to a solid pipeline in a few months which allowed them to make final assets final lighting and rendering on this 70 minutes episode. As a first experience, it was a tough challenge that they successfully overcome!

Julien Blervaque (Sr. Tools TD, Head of UX):

I was in charge of the graphic workflow and to do so I took inspiration in my experience of live-action filmmaking and stages. Thanks to our TD team we designed and developed several tools that quickly became the backbone of our pipeline.

The good news is several of these tools will be released soon to the community!!! 

As you know in production it is mandatory to keep track of the context and origin of every rendered image in order to closely follow the advancement of the project. Blender already has a feature to write metadata on pictures but we needed more flexibility and additional information, such as the frame index of the image in various time systems (3D, edit…). We also wanted to have a more customizable layout. I then started to write my first add-on for Blender 😊
I found the Python API very open and powerful. This allowed me to import Pillow, a common graphic library, to create an image with all the specified information, an image that I could combine afterward with the rendered picture thanks to a process based on the VSE and on the generation of a temporary scene. This got integrated pretty well in the pipeline, it can be used as a standalone tool or be called by the script as is the case with Shot Manager.
However, the add-on has an important limitation that we couldn’t overcome when in production. During a rendering script execution, the whole UI of Blender is frozen and there is no easy way to display the rendered images, update a progress bar nor interrupt the process (in case the images appear invalid for example).

It quickly appeared to me that in order to let the director experiment and concretize his vision in a short amount of time it was crucial to iterate fast in the way sequences are shot and cut.
To do so I developed an add-on named Shot Manager that introduces a true shot entity in Blender scenes (by a shot I mean not only a point of view, which is provided by a camera but also a recording time) and a wide and powerful set of tools to build and edit sequences in real-time directly in the 3D context.
The key idea lies in the analogy with shooting a continuous live action from several real cameras. That’s why our sequence scenes contain an action with a plausible timing, we then introduce several cameras and the tool defines when the cameras are recording and in which order they appear in the edit. It also provides a play of the shots directly in the viewport, supporting ellipses and jumping back in time.
In practice we were able to reach the fast iterations by working on several axes:
Quick overview of the features of Shot Manager:
This add-on received a lot of attention regarding its UX and UI in order to make it as simple to use as possible in spite of the number of features. It will come with documentation and tutorials.

Having a video editing tool integrated inside the work environment is really appreciated in production. It considerably extends what can be done in the 3D viewport, providing sound support for instance. It is also of great help for many casual pipeline-related actions such as video previews. The Video Sequence Editor appeared a bit limited for our needs though and we thought having a more classical approach to the tracks in terms of UI would make it more efficient for the artists. We then developed this add-on named Video Tracks to introduce a set of track headers and various tools to facilitate the manipulation of time, markers, and clips. We used the VSE quite extensively during the second part of the production to compare edits and batch export shot videos with the final voices.
Some of the limitations we bumped into are: no separation between audio and video tracks, the number of tracks is limited to 32, slowdowns to navigate between markers when there are too many, sampling mode of the viewport when the image is zoomed in Nearest, invalid time duration for long MP3… We also regretted that it was not possible to use the output of a VSE as a texture in another scene.
Once this add-on is released I believe it could be used as a good basis for discussions about improvements that could be introduced natively in the VSE.

Our TD department developed several production-specific tools in order to populate the sequence files with the 3D assets of the project. One of them ensured that the assets contained in these sequence files matched with the casting specified in our in-house production tracking tool called Gattaca. Another one, named Asset Bank, was used to link or import new assets from the bank “on the fly”. It is simple and efficient, and although probably obsolete soon with the asset manager that is about to come natively in Blender it helped us a lot at the time. We will share it as an archive as we believe it could still be of some interest to the community.

And of course, there is Mixer, our tool for real-time collaboration, to which we gave a lot of attention and which was recently released in version 1.0, with documentation and tutorials. Although not used in practice as extensively as we initially thought because it was not yet production-ready, Mixer helped us to validate the concept that work sessions including artists and the director around the same sequence lead to quick and original results.

Yannick:

Switching to another software is critical, and many studios are afraid to discover something new and different because it costs a lot of time and money. Blender is sometimes very different from other DCCs in its mindset and its workflow and as a result people often do not use it at its full potential.
I then personally think that one element can be improved to help onboarding studios. We have so many tutorials by artists for artists (and it’s awesome!), and even in the Blender Cloud, you have a lot of resources about creation or how to use Blender as an artist. However, there are almost no specific tutorials and examples about workflows and how it is used in studios, nor best practices regarding links and collections. These are topics Blender Cloud may propose.

This production on Blender was a great experience for us. We were able to integrate the application into our in-house pipeline and connect it to the other applications quite easily thanks to the strength of the Python API. In spite of some features that would need some maturity, we had the confirmation that Blender is production-ready and comes with tools that can really push up our way of working in terms of creativity.

source