Timing May Be Right for Collaborative Content Production

Content Insider Special – New World

By Miles Weston

“We Have No Idea What Threats Are Out There. We Can’t Do This Alone. We Need You.” Nick Fury, “Captain Marvel,” Walt Disney Pictures/Marvel Studios, 2019

Sometimes
timing is everything.

Bob
Igor put the finishing touches on his $71B Disney deal for 21st
Century Fox just as Jensen Huang, CEO of Nvidia, kicked off the firm’s annual
GTC conference to announce Omniverse which could streamline the toughest and
most expensive part of the M&E industry … production.    

Disney took possession of 21st Century Fox which includes Twentieth Century Fox, Fox Searchlight Pictures, Fox 2000 Pictures, Fox Family, and Fox Animation; Fox’s television creative units, Twentieth Century Fox Television, FX Productions, and Fox21; FX Networks; National Geographic Partners; Fox Networks Group International; Star India; and Fox’s interests in Hulu, Tata Sky, and Endemol Shine Group that includes franchises like The Simpsons, Marvel, Star Wars and Avatar under the same roof as the Mouse’s family.

And
with the upcoming roll-out of Disney + streaming services you know he’s going
to want to get that huge (and valuable) library of content prepped to stream to
folks around the globe as quickly as possible. 

BAM!! Nvidia unveils Omniverse.

Strong Backup – Jensen Huang had some strong backup when he unveiled Omniverse with ultra realistic game characters that could help take out the competition in a heartbeat.

Okay,
the importance of Omniverse got a little buried in Huang’s three-hour
technology heaped keynote at San Jose Convention Center. During the keynote, he
spelled out what the company was doing to progress accelerating the sprawling
data centers that power the world’s most dynamic industries.

He
rolled out a spate of new technologies expanding on the company’s work in AI, deep
learning, and ray tracing, as well as graphics enhancement that were adopted by
industry leaders and incorporated into offerings from the world’s biggest name
service providers like Microsoft Azure. 

The
Omniverse announcement was important to Igor and the rest of the content
production industry because it could dramatically change and speed the production
of movies and shows streaming viewers want to view on their screens.

M&E
may be just entertainment to you; but to the people in the industry, it’s a big
deal.

According
to PwC’s most recent report, the industry is projected to reach $792.3B by 2022
and the MPAA (Motion Picture Association of America) says it will be created by
more than 972,000 independent shooters, production and distribution folks
worldwide. 

A
number of Huang’s announcements will impact the quality and speed of video content
people get to enjoy, but it’s Omniverse – and the stuff in it – that could
change darn near everything about the creative process. 

Landing Space – Nvidia’s Geforce RTX and advanced ray tracing technology looked as powerful as it is in real workflow.  

The
solution is based on Nvidia’s Turing GPU (graphics processing unit)
architecture which enables video production people to create, refine and
enhance/improve content. Goose (the cat) has razor-sharp claws, Carol Danvers
can shoot death rays from her eyes, and the Skrulls have remarkable and
realistic-looking facial features.

The
RTX real-time ray tracing technology is what has put Nvidia at the top of the
graphics marketplace and the combination that production/post folks turn to to
get the job done right the first time.

To
show off ray tracing, Nvidia reworked Quake (a 1990s first-person shooter) to
render and change lighting effects on a visual scene to simulate the physical behavior
of light in almost real-time.  Then, Huang
showed a 2019 BMW 8 Series Coupe beside a real-time rendered car and challenged
the audience to tell the difference.

Most
production people we know, especially those in VR and volumetric content production,
know the technology and rendering techniques much better than we do; so we
checked with them following the event and will cover those points later in this
column.

But
to find out more than we really wanted to know about it, we turned to our pixel-chasing
friend Dr. Jon Peddie, Jon Peddie Research. 

The
first thing he said was that it’s big, huge even.

Simply
stated, it’s the physically accurate, photorealistic representation of objects
film and video people use to create breathtaking animation (like the creative
works from Pixelworks) and the creative work from ILM (Industrial Light &
Magic) as well as the hundreds of post production and special effects studios
around the globe. 

Coupled
with 3D modeling, it uses extensive and complex libraries and has become an
integral part of the content creation process that keeps people glued to their
screens.

Nose to Nose – The advanced capabilities of Nvidia’s graphics technology enables video production teams to quickly produce film elements that look impressive–even up close. 

Thanks
to ray tracing (and a lot of talented folks), the apes looked really real in The War of the Planet of the Apes) and
Kong looked more like himself in the 2005 film as opposed to his first
appearance in 1933.

Peddie estimates there are about 2.1M 3D rendering software users; and with
every studio and distributor pushing for more and more good content, he estimates
it could be a quarter of billion-dollar market in 2023. 

He noted that ray tracing is the most essential general-purpose rendering
technique available. It’s not the only rendering technique, nor is it the
fastest; but it is the most accurate and can be the most photorealistic. It has
become an indispensable tool in the movie/series industry.  If you want to know more, they have a report
on the topic at –
or  

In
other words, the technology is great for content production and it is even
better than great in the hands of a highly skilled editor, special effects
craftsperson or production/post person.

Global Production – Film editors, special effects, audio and other creative post production people are no longer tied to a specific studio but are spread across the planet. Film producers work with a large number of creative talents to get the best creative work possible no matter where they are located. 

However,
the high-powered workstations and software tools were (and are) in post
facilities located where the professionals want to live/work rather than next
door to the studio. 

Previously,
since no executive producer, show runner or project investor wanted to entrust
the success or failure to the best creative content available, hard disks of
precious creative work were shipped to the facility to do their magic.

Post Delivery – To get content to the best possible post production teams – regardless of where they’re located – producers have historically relied on overnight shipping and delivery. 

And
the content was securely shipped back and perhaps then on to another member of
the production team, adding time and financial investment to the production
schedule.  For the majority of projects,
time was their most critical/scarce investment.

While
the game, IT and other solution providers didn’t go bananas when Huang unveiled
the firm’s global studio collaboration, we could see that it just might be the
most important development/introduction to the film/series industry since celluloid
cinema film.

New Collaboration – At Nvidia’s GTC, Jensen Huang introduced the firm’s cloud-based global studio collaboration solution that puts tools in the hands of creative production people and enables near-real-time communications and interactive workflow

Huang
noted that accelerated creative content production wasn’t just about chips but
the collaboration, codesign, continuous optimization of chips, system,
algorithms and applications.

The
Nvidia ecosystem includes more than 1.2M developers, 125 GPU-powered systems
among the world’s 500 fastest supercomputers and more than 600 applications
powered by platforms including those by Adobe, Avid, Dolby, Autodesk, Dassault
Systèmes, Pixar, Siemens, Unity, Unreal and Weta Digital – the latest version
of every tool a production/post person would need to optimize his/her creative
work, reduce production time and virtually eliminate nonproductive time.

Global Creativity – During the GTC keynote, Jensen Huang showed a live demonstration of exactly how a number of creative people could work collaboratively on specific frames and film segments to more quickly complete projects.  Source – Nvidia

To
illustrate that Omniverse is ready for film/series production today, Huang
showed a live demonstration of an actual project using multiple applications to
create and share scenes across different teams in different locations.  

One
artist using Maya with a portal to Omniverse collaborated with another creative
using UE4 and both could see live updates of each other’s changes in their
application.  

Jeezz!

It
could cut days, weeks, maybe even months off the project’s production schedule.

Huang
noted, “We wanted to make a tool that makes it possible for studios all around the
world to collaborate. Omniverse basically connects up all the designers in the
studios, it works with every tool.”

Sure,
it worked beautifully for Hans Zimmer and his Remote Control Productions team
when they worked on Captain Marvel;
but cripes, Disney (now the owner) is on its way to racking up $1B so the production
budget probably wasn’t a major concern.

But
how about normal folks, Indie filmmakers?

We
asked a few:

Lewis Smithingham –
30 Ninjas

Lewis
ran the shooting/production – https://preview.tinyurl.com/y49s8xck
– of Invisible
( created/directed
by Doug Liman (The Bourne Identity, Edge
of Tomorrow).  It was the first major
scripted VR mini-series designed specifically for 360-degree virtual reality.

“We
had teams working on the production on both coasts and Doug was working in parallel
on other projects and needed to constantly preview the work,” Lewis
recalled.  “In addition, this
was a pioneering project.  We had to
develop/refine our production tools/techniques to produce Invisible. 
Combine the two and you’re faced with some enormous hurdles. 

Blocking Scene – 30 Ninjas’ Lewis Smithingham (center) goes over the set-up of a scene for Invisible with producer Don Limon (back to camera). 

“Team members would work on a few frames or segments, package them up and overnight them to other members of the team or to Doug,” he continued. “Shipping was one of our major budget items for the series.  But the time lag of not being able to work together in realtime or near-realtime, having to develop/duplicate production tools on both coasts was a major challenge. 

“But
the biggest obstacle any creative team has to overcome is the lack of fast,
precise communications,” he added.  “Personal communications is probably
more important than all of the new tools filmmakers have at their disposal
today.” 

Cirina Catania –
The Catania Group

Cirina Catania has worked on more
than 130 film, TV and new media projects around the globe; and over the past
two years, she has been focusing on meaningful documentaries.  She is in
the process of finalizing a doc entitled Kionte’s Storey, highlighting
the dramatic journey of a Marine corporal who lost his lower right leg to an
IED (improvised explosive device) in Afghanistan.

Remote Post – With many of her projects, Cirina Catania has found that fast, reliable Internet isn’t available to her in places such as in the high desert (l) and on the Amazon (r), so she has to resort to the multiple storage solutions she takes with her on her ventures to work on production when she has some free time.

 “Being able to work in the cloud in real time with
production people on opposite sides of the world can produce huge savings for
filmmakers,” she said, “but sometimes that’s impossible, because global
connectivity isn’t always feasible.”

 
She has shot projects in Nevada’s high desert where the closest access to half-way
decent Internet connectivity was hundreds of miles away and in the Peruvian
Amazon where even satellite communication was iffy.
 
“If the pipe is too narrow, too iffy, you just have to fall back on the tried
and true and make an insurance copy and two backups,” she explained,. “Then,
start post in the field or wait until you get back to ‘civilization’ and hope
the film gods are on your side.” 

Andrew MacDonald –
Cream Productions

Headquartered
in Toronto, Cream has built a strong reputation producing documentaries as well
as VR/volumetric series and projects for Hulu, HBO and major clients like Lexus
and Microsoft.

Sneakernet – When production work is being done in-house at Cream Productions, creative director Andrew MacDonald (l) and other team members often resort to sneakernet technology – moving storage systems from one workstation to another – to keep the project on schedule and meet deadlines.  At other times, they need specialized high-performance systems (r) to handle the GPU-intensive work. 

 “With a good creative team here, we’re able to
do much of our work in-house,” Andrew MacDonald commented. “But there are
always those instances where you need a very unique tool for a specific task,
or you need someone with a special talent who can work on content better and
faster than you can.  Even in a town like
Toronto that has great internet connectivity, getting the work to a specific
talent is difficult and driving the segment to the individual is a ridiculous
waste of talent and time.  Being able to
go online and share ideas in real-time in a collaborative cloud lets you get
more done, more efficiently and more cost effectively.”

Increasingly,
there are projects that require monstrous computing power that a studio doesn’t
have in-house such as a recent project shot in 8K with lots of layered heavy
effects, spatialized music and more.  For
that heavy load, Cream contracted with SIRT (Screen Industry Research and
Training Centre) for a 48-node render system that required 14 days (round the clock)
to render a 7-1/2-minute project. 

“It
would have been great to put the project up on a big, robust collaborative system
had it been available,” said MacDonald.  “We
wouldn’t have had to live with the noise level of a jet engine running
continuously.”

Aaron Semmel – BoomBoomBooya

With
a film project, one of the toughest jobs is that of the EP (executive producer)
who not only has to help get the funding for the project but also manage every
expense to bring the film or show on or – preferably – below budget.  To get some input from the loneliest person
on the project, we touched base with Aaron Semmel, an executive producer we met
a year ago at the HPA (Hollywood Professional Association) conference.

Budgetmeister – Executive producers like Aaron Semmel (l) have one of the more thankless jobs in the production of a film or series project, satisfying the needs of investors and the audience by meeting the team’s critical goals – time, money, budget.

For
the past 15 years, Semmel has been responsible for a wide range of co-production
projects with Critical Content, Big Fish Entertainment, GRB Entertainment, Pilgrim
Studios and others. His company, BoomBoomBooya, is currently working on several
television and feature film projects that are in various stages of development
and production.  

“The
bigger the project, the bigger the budget and the more line items you have to
juggle, manage and control,” he noted.  “The
largest portion of a project budget is the above-the-line costs – talent, crew,
distribution, marketing – and these are fairly fixed. The area that keeps us
awake at night is the below-the-line costs – production, post.”

With
the rapid shift of studios/production facilities combined with the unprecedented
demand for films and shows he said budgets still aren’t infinite and if
anything, have become tighter.

“There
are only so many quality line/supervisor producers, directors and production people
available; and time is critical,” he explained. “Being able to tap the right editor,
special effects or audio people who know how to use the tools to produce results
simply puts more money on the bottom line to satisfy our fickle critics –
investors and viewers.” 

Yeah But…

The
only problem is I’m a storage guy … you know, like to get up in the morning, go
to the office and pat the box that holds all of our work; and we think everyone
else should do the same.

Igor
and Huang just stared at us.

Carol
Danvers straightened it out for us though when she said, “I’ve been fighting with
one arm behind my back.  What happens
when I’m finally set free?”

Nuff said!