AI Is Nice but It’s Storage That Really Matters

Content Insider #957 – Treasure

By Andy Marken – andy@markencommunications.com

“People had more than they needed. We had no idea what was precious and what wasn’t.” – Eli, “The Book of Eli, “Warner Bros., 2010

You ever have one of those lightbulb “Well yeah!” moments when you finally realized why
you like something even when you don’t like it? 

Ours happened recently during one of our weekly Saturday AM discussions with two super smart friends (yeah, they include us out of pity) on how AI is either: A – a really valuable and time-saving productivity tool or B – something that will slowly turn your mind to mush and eventually will kill off inferior humans.

We had one of those hot flash epiphanies that AI was the best thing to happen for an industry we’ve been around for years … storage.

AI can’t do anything without data … a lotta lot of data.

The more data the better because it has an insatiable appetite and it in turn creates more … data.

Poor Jensen Huang and his Nvidia team have to be constantly investing time, energy and money (lots of money) to make bigger and better chips to gobble up, chew, digest, and spit out more data. And so, the cycle continues.

But storage (tape, optical, hard drives, flash) hasn’t/doesn’t change that dramatically over time except for adding more capacity, better reliability and occasionally getting a little smaller.

It was 60 plus years ago (1956) that IBM proudly shipped the first 5MB hard drive.

It weighed a ton, had 50 massive 24-in platters, was finicky and leased for $750/mo. (about $9K in today’s dollars).

Cripes, our first computer had a whopping 10MB hard drive and we couldn’t imagine that we’d ever run out of storage space; but of course, it had floppy discs for backup … just in case.

Today, there are 2TB SSD cards that filmmakers swear by because they are rugged, ultra-compact, very reliable and cost less than $300.

Filmmakers can shoot anything and everything; and if they’re getting close to filling one in their Blackmagic or RED camera cards, they just swap it out with another and they’re ready to capture even more action.

Heck, even our daughter’s iPhone has 512GB of storage because … well she needs it for all her texts, photos and videos.

Yeah, she produces a lot of data but she’s far from alone.

The world generates tons of data.

According to estimates by IDC, we (everyone, everything) generate about 402.74M TB.

That includes everyone’s social media, online transactions, emails, photos/videos, systems talking to each other/doing stuff; and now, AI creating data out of data, copies of data.

That sounds like a lot of data, and it is but what’s even better/worse is that 90 percent of the data was created in the last two years.

Think that’s mindboggling?  See what happened in the last 60 seconds!

‘

We know you’ve been working your butt off to meet that project deadline, but there’s someone out there stirring the pot to make more data that needs to eventually rest/sleep somewhere and that’s what storage is for.

In the meantime, other folks are doing 59M Google searches, sending 251.1M emails, watching 138.9M YouTube/TikTok videos, messaging, streaming hours of movies/shows, playing serious video games and stuff.

Yes, we store the data on our personal devices (smartphones, computers and tablets as well as on our office computers and business data centers. Heck, we store data anywhere we can find to put it–even if we can’t remember where we put it.

Then, along came Nvidia’s high-performance chips, AI and more powerful systems and things got demanding.

Of course, everyone got excited because folks could see the value of processing, using and storing your data, their data and ours.

BAM, the big boys (Amazon, Google, Microsoft, IBM, Oracle, Alibaba, Huawei, Tencent, Baidu and other storage firms around the globe) rushed to build big data centers everywhere they could find enough land, power (it runs the centers) and water (it cools the centers operations).

Cloudscene and Statista estimated that there are over 11,800 data centers worldwide with more than 5,400 of them located in the US and more everywhere ramping up to go online and store everyone’s stuff.

There’s so much need for big data center storage that a few companies are working hard to figure out how to build and operate data centers in space.

No, they won’t need IT folks to run/monitor them up there … that’s what AI will have to do.

There is no end to the demand in sight and no, we won’t get into the environmental impact on the globe right now but there will be hell to pay.

Even tomorrow’s autonomous vehicles that transportation people keep promising will need a constant stream of data for and from sensors, black boxes, maps/navigation, infotainment, OS and a bunch of other applications – 2 plus TB just to get out of the garage and go somewhere.  

So yeah, being in the storage business is better than being in the bleeding-edge business of processors, AI tech and systems because storage hasn’t changed radically over the years.

Tape, optical, hard drives and flash haven’t changed much since the technologies were first introduced.

Sure, they get more reliable, capacity storage increases are made, speed is tweaked, performance is better, and each has gotten less expensive over time.

But they’re all reliable businesses because you (and they) keep creating, processing, reprocessing, storing and using data.

In fact, the better/less expensive the storage the more you use it because you want to store more stuff … just in case.

Don’t get us wrong, the storage industry doesn’t just sit back, rake in the big bucks and profit from the huge profits.

Every one of them is working hard to develop newer, better solutions so they can get a lead on the next guy/gal and be ready to deliver the biggest/baddest storage solution to wow data center owners,

Companies are racing to develop even better, higher-density storage technology including DNA, holographic technology (check Wikipedia or do a search for the work … online) as well as constantly enhancing today’s technologies.  

Trust us, DNA storage is a cool idea. We know because we’ve been following it for years and it’s going to be dynamite when it’s finally stable enough and inexpensive enough to go into full production.

The whole DNA storage concept is mindboggling – encoding 1s and 0s into DNA’s four base molecules (A,T,G,C) and then synthesizing DNA  strands to store data in a molecular form producing a solution that will have extremely high density, long-term stability and the potential of soring all of the world’s data in a small space for thousands of years.

But there’s still “a little” work to be done.

As always, it’s all about getting a competitive edge on the next guy and giving you more, better, cheaper storage so you can store more stuff.

Yeah, it’s all about you and the data you use/work with and just have to have … just in case.

You might have noticed in the 60-second chart that folks were watching 43 years of streaming content every minute.

And it’s true, the streaming industry has increased the volume of video content that is created, produced, post produced and watched everywhere on the globe.

During the first quarter of this year, about 2,300 scripted and unscripted projects were developed. That’s a growth of 34 percent and the demand just continue to grow.

And that doesn’t include the old films/shows people still enjoy watching … if they’re available.

For years, movies were created, copied and stored on celluloid film.

Directors who can dictate to studios like Christopher Nolan, Quentin Tarantino, Martin Scorsese and others still prefer to shoot on film because of its aesthetic qualities, limitations and discipline the filmmaker has to have to create a project.

They, and a lot of seasoned directors, just feel the visual quality including warm, organic halation that to be truthful just can’t be achieved with today’s digital production.

Oh yes, we forgot to add that celluloid is very flammable.

Millions of hours of past productions are stored in humidity, temperature and ventilation-controlled vaults because nitrate film is basically unstable and ages/decays over time. 

Despite all the precautions, there have been major fires causing the loss of irreplaceable historical and valuable films.

In 1978, a fire in the National Archives destroyed millions of feet of historical newsreel footage; and in 1937, decaying film led to the destruction of most of the silent films produced by Fox before 1932.

In 1965, an electrical short in the MGM vault caused an explosion/fire that destroyed hundreds of their early films and as recently as 2008, a fire in Universal Studio’s archives destroyed thousands of films and original audio masters.

A number of years ago, we were in Atlanta for a series of meetings at the Turner Broadcasting Techwood Campus and CNN.

While at CNN, we were shown one of their many physical tape library facilities which looked like a football field of tape systems with people still mounting and refreshing tapes. 

We were told it was a never-ending process.  By the time they finished with one row of drives, they’d start the process over and over to protect and preserve billions of news clips.

Fortunately, they’ve since transitioned to digitally storing the clips in local libraries and the cloud (someone else’s storage systems), saving millions of dollars in staff time and making it possible to retrieve a single clip in minutes instead of days or …

A little side note here. In the early days of television studios, network and local stations often captured the shows on videotape for later showing but since the tape was so “expensive” they often recycled the tapes overwriting new shows over old segments until they realized ****, that segment is gone … forever.

Yep, since George Gobel’s, Red Skelton’s, Burns and Allen, Abbott and Costello and other earlier shows were before our time, most of those variety and comedy shows no longer exist.

It wasn’t until Jackie Gleason added in his contract that he would retain the rights to every show, and every tape that executives realized those tapes were valuable for reruns and they kept the reels until they ultimately converted to digital.

Enriched Capabilities – The switch to digital throughout the film industry put all of the hardware, tools and services in the price range that made it possible for indie filmmakers like Cirina Catania (back to camera) to pursue passion projects like her Kionte film.

Despite the protestations of some purists, digital did change the film/show industry for the better.

Lower cost, higher-quality gear and systems have put the creative tools in the hands of many more filmmakers (including a lot of our indie shooter/producer friends) and have opened the industry to tens of thousands of dreamers around the globe.

Digital technology– and the internet – have made it possible for project owners, producers and directors to work with the best postproduction people possible, no matter where they are located.

Dispersed Workload – With the transition to digital film production, creative teams don’t have to be located next to each other or even across town.  For example, the technology and internet make it possible for people like Andrew McDonald (l) in Toronto to edit and polish video content while Woody Woodward, Allied Post Audio in Santa Monica, can improve the music and audio portion. 

Production crews can be in London, Toronto, Atlanta, Saigon, Buenos Aires, Dubai or anywhere and can send their dailies to their preferred postproduction team wherever they are located.

Similarly, in many instances, postproduction work can be done by professionals spread around the globe.

Increasingly, postproduction supervisors, editors, VFX specialists, colorists, foley artists, sound engineers/editors and other team members live and work next door, across the country or wherever they prefer.

While editors and specialists can do their work in the cloud, most prefer to have the dailies and project on their own high-capacity storage systems to store the RAW and production work saved and backed up locally.

In many instances, one specialist will work on the dailies and send his/her work product to the next member of the team in another part of the world and the project can be completed better and faster because the team is working ng on a global 24-hour clock.

Today, the movie/show content industry uses a multi-tiered storage approach, including high-speed SSD and NVMe drives on the set and post-production editing with centralized NAS (network-attached storage) and RAID redundancy.

People on the production/post teams also have their own RAID NAS and high-capacity hard drives to do their artistry on the project, passing the work product to the centralized storage system and on to the next person on the team to do his/her part of the project while keeping a copy locally … just in case.

Yes, while many phases of the video content creation industry are benefitting or arguing with their AI-enabled tools, the one common denominator that keeps the film/show industry churning out as many great projects as they do is … storage. Lots and lots of high-capacity, reliable and ultimately expensive storage.

So, AI may be the “hot topic” using/creating tons of data, but just in case you forget what you were looking for or where you put it, at least you know it’s someplace in storage.

And as Eli said in The Book of Eli, “It means that you know something even if you don’t know something.”

All you have to do is figure out where in your reliable storage it’s located.

Hey, storage folks can’t do everything for you!

Andy Marken – andy@markencom.com – is an author of more than 900 articles on management, marketing, communications, industry trends in media & entertainment, consumer electronics, software and applications. He is an internationally recognized marketing/communications consultant with a broad range of technical and industry expertise, especially in the storage, storage management and film/video production fields. He also has an extended range of relationships with business, industry trade press, online media, and industry analysts/consultants.