Great Video Requires Talent and Lots of Fast, Reliable Storage
Content Insider #603 – Protection
by Miles Weston
Now, my point is that the lack of a silver spoon has set you on a certain path that you needn’t stay on. If you’re prepared to adapt and learn, you can transform. – Harry Hart – “Kingsmen: The Secret Service,” 20th Century Fox, 2014
After catching the glitz and beauty of TIFF (Toronto International Film Festival), we visited some friends at Cream Productions. Great way to remind ourselves that behind the glamour on the screen it requires a lot of work to create great film, ad, series projects.
You know, that clean, organized green screen studio set-up and orderly post areas folks like to portray.
Silent Set – Members
of the Cream Productions crew set up their gear and walk through the shooting
segments for the VR Hulu series, ‘Curious Minds.’ The series
earned them the Gold VR/360-360 Video
Award at this year’s W3 Awards.
They only exist in movies.
Or in the polished marketing pitches at events like IBC, NAB and SMPTE.
Whether you’re an EP (executive producer), showrunner, DIT (digital imaging tech) or Indie filmmaker, the time/money budget is tighter than a pair of Lululemon yoga pants.
The content (every frame) is critical, expensive. That’s why it’s important to back up every step of the production/post process and back that up at least once, preferably twice.
At SMPTE, there was a lot of discussion about the speed, flexibility, scalability, ease and savings of cloud production.
A number of session speakers emphasized that the fast-shifting workflow, resource and collaboration requirements and up-front costs were tailor made for cloud-based compute and storage services.
The problem is the cloud wasn’t designed for most content creation folks/teams.
To put the production market in perspective, Coughlin Associates breaks down the industry’s output by production budget:
- High-end projects $50M plus budget or about 15 percent of the creative content shown
- Medium-budget projects $10-$50M or about 35 percent of the video stories
- Low-budget work is that produced with a budget below $10M (probably most in the range of $1-$5M) represents about 50 percent of the films/shows people view
Workflow – Whether it’s a lone filmmaker project or a minimum crew effort, nearly all content productions follow a typical workflow schedule with direct attached storage in the field or on the set which is then used for post-production work. Increasingly, filmmakers are moving to more expandable network attached storage to have access to and protect all of their projects.
Cloud folks like to point to their successes with tentpole projects produced by high-end facilities (estimated 800 worldwide) as well as the creative work of mid-range production houses (about 36,800).
Citing their system and storage expertise, they present sound reasons why creative work should be done in the cloud.
Of course, these operations have a full-time IT department or person to keep things running smoothly.
The high- to mid-range facilities leave the workflow management task to the boss or the person who stepped out of the meeting for a moment.
More Us than Them – While every filmmaker dreams and works to do a well-budgeted film ($5-%50M plus), the majority of video storytelling projects are produced by individual to small crew projects. And there are 10s of thousands of filmmakers around the globe turning out outstanding “economical” films and shows.
Low-end budget production teams and Indie filmmakers (roughly 840,000) tend to prefer to keep their content storage closer to home.
With filmmakers working in 4K, 8K, VR, HDR formats to futureproof their creative efforts; are focused on delivering today and keeping content where they can see it for tomorrow.
Without constant attention, network limitations and stability make it almost impossible to do a complete project in the cloud … no matter what marketing/sales folks tell you.
The cloud is great for moving small files like proxies or a backup for the backup of the backup…the hail Mary move they have to make when everything goes to …
Whether it’s a high-end, mid-range or low-end project or the transport and management of content production, the impact of IP is still limited when speed, performance and security are major concerns.
Growing Numbers – With the growing demand for quality content for streaming and the increased use of digital, IP based systems/solutions; filmmakers at every level are shooting more content that needs to be reliably, securely stored.
4K, 8K resolution films can easily produce up 1.5PB of content that must be stored while a TV drama sucks up to 400TB of storage.
Even projects like Cirina Catania’s “The Kionti Story” documentary produce 2TB of data (and counting) for of The Catania Group,
Lewis Smithingham, of 30 ninjas, noted that for Doug Liman’s six-part episodic VR series (the world’s first broadcast-quality 360-degree production and winner of a number of industry nominations and awards) the firm racked up 135TB of data for the highly acclaimed immersive project.
Filmmakers agree that because of timeline demands as well as the size/number of dailies/rushes required; it is better, easier and safer to retain all of the data on site.
At SMPTE,
Blackmagic Design’s President Dan May said DaVinci Resolve 15 is ready for projects
of all sizes.
“Deadlines are even more critical today,” he commented, “which
is why we’ve been enhancing its collaboration tools and feature set. Now, editors,
assistants, colorists, VFX artists and sound designers can work on a project at
the same time.”
“Adding open source RAW support (images as seen by the cameras’ sensors) means
organizations can download the solution to their network and not be subject to
Internet quirks or security issues,” he explained.
While IP-based production solutions have grown dramatically because IP is ubiquitous,
it’s also more easily hacked.
Fast, Economic Post – Blackmagic Design has been instrumental in putting high-quality post production tools in the hands of today’s filmmakers and post production crews.
Blackmagic RAW is not just a video data container, the files include metadata with information about the sensor and its color science as well as the camera settings, so all of the workflow data are instantly available.
Keeping all of that data from the shoot makes it fast and easy for MAM (media asset management) tools to manage/track all of the media components – video, audio, metadata – during the editing/post process.
It beats the heck out of the old paper/pen data logging!
High-speed Internet connectivity is still an integral to the production process.
The major issue that exists for production workflow is that project activities can be severely degraded or disappear completely. The key parameters for filmmakers are:
- Bandwidth
- Latency
- Packet loss
OTT services like Netflix, Hulu, Sky and Tencent as well as service providers like Akamai and Ooyala solve these challenges by installing high-capacity, high-performance storage and thousands of servers around the globe to keep the films/shows close to the user and improve streaming performance.
These cloud issues also become major issues for filmmakers working with color, FX and other specialist on other sides of the country or half way around the globe.
“Only after the high-resolution content has been safely, reliably stored locally and backed up a couple of times do we move content to expert partners,” Smithingham emphasized. “A segment or film file is a terrible thing to lose.”
Back in Post – Seasoned filmmakers like Andrew MacDonald, of Cream Productions, use fast, reliable, low-cost direct attached storage on the set but immediately transfer all of the material to centrally located network attached storage, so the entire production staff can quickly access specific segments to work on. The advent of high-performance media asset management systems also ensures nothing is lost or misplaced.
“By acting as
your own data center with your production facility, you have more control over
who has keys to content and who comes/goes,”said Andrew
MacDonald, creative director at Toronto-based Cream Productions.
The organization’s creative technologists produce 4K TV specials, series and
ads as well as immersive VR projects like “A Curious Mind” for Hulu that received
the Gold VR/360-360 Video Award at this year’s W3 Awards.
“You know what your crown jewels are,” MacDonald added, “and know how the cost of
losing them will impact you, so you store accordingly.”
While it is often an afterthought for content creation, production and
distribution; data storage is a key element in today’s video-driven world.
Even your best, most casual video projects require a safe, reliable place to sleep so they’re ready to be used or viewed.
It becomes more critical because of increasingly large file sizes for high-resolution, multi-camera images and RAW content that are 2-3 times larger than logarithmic encoded images.
While the cloud holds promise for filmmakers who need to meet the fast-shifting demand for resources and to speed collaboration, the challenges of moving large video files and reliability/security encourage most storage to remain locally.
Home Sweet Home – While cloud services like to extoll the savings and reliability of storing all of your content in the cloud, most filmmakers and production teams prefer to know all of their content is safely, reliably and securely stored at home.
Most showrunners and DITs use DAS (direct attached storage) RAID (Redundant Array of Independent Disks) on set because it is a fast, economic, reliable means of moving content from lower-capacity flash cards to high-capacity (12-50TB plus) devices.
High-speed interfaces such as Thunderbolt 3 or USB C provide data transfer rates up to 40 Gbps.
While most filmmakers (especially Indies) continue to buy production storage based on the projected capacity needs for each project, there is a growing trend to move to expandable capacity NAS (network attached storage) solutions.
NAS makes it easier and faster to carry out simultaneous collaborative work and provides fast access to present and past work.
In addition, expanding capacity is as simple as adding more high-capacity HDs or SSDs rather than buying a complete, fully populated DAS unit.
And a lot less expensive considering how many projects you will produce during your career.
Shifting Trend – Initially, filmmakers thought network attached storage was only a realistic investment for large studios; but increasingly, they are understanding that the early investment of the NAS infrastructure also becomes less expensive as time passes and more projects are all stored on the rack rather than spread across 10-20 direct attached units, somewhere in the office.
So, what do you do with the project DAS storage?
Use it as your first copy, failsafe backup for each project and keep your years of work online, readily available and easy to go back in time, if necessary.
The biggest question most post operations have in moving to NAS is copper Cat 6 or fiber optic cable. Performance is about the same (salespeople will argue the point), fiber cable is slightly more expensive, but the biggest difference is distance – 300 ft for Cat 6, 1-6000 ft. for fiber.
The kind of show people want to watch on one of your screens changes with age, time of the day; and you never know when someone will want you to refresh the project you did years ago when viewers agree with Harry Hart, “Nowadays, they’re all a little serious for my taste. But the old ones … marvelous. Give me a far-fetched theatrical plot any day.”
Every filmmaker wants his/her insurance copy in addition to the mandatory insurance copy!
Nice to know you have all of the pieces on your production storage instead of in someone’s cloud.