James Tucci is the CTO of Archion Technologies (http://www.archion.com), the Los Angeles-based provider of intelligent networked storage solutions designed specifically for post production and graphic professionals. Here, he looks at storage in the foreseeable future.
The one thing we all know about the future of storage in post production is that it’s just going to keep expanding. But this expansion is no longer just in storage capacity — there is also speed, latency and throughput. These attributes create the balance that allow 4K, 8K and beyond to become possible. And there is no top-end in sight. But where are we going with all this storage?
One: Expanding Capacity
Let’s face it, 4K is just a beginning. We’re going to be capturing 8K, 16K, and stereoscopic VR within the next year to two. Even with compression, we will still see digital storage increase its capacity by five to six times what it is today.
Using compressed formats like HEVC (H.265) for 4K will not slow the expansion of digital storage, so capacity is going to be key. HDDs will have a long life span going forward in spite of what the SSD and NVMe (Non-Volatile Memory) manufactures claim. Being able to handle drives with densities up to 10TBs per drive is a challenge. Having a 10TB drive go down in any RAID system is going to take a long time to rebuild. Manufacturers like HGST are putting in software that helps speed up the rebuild. New technology — like erasure coding — will speed it up even more.
Two: Speed
So this is great for storage manufacturers who make big high-density boxes able to hold all of the 4K and 8K media ready to connect up to nonlinear editing system for post production. Currently, 4K workflows use DPX sequential files. Each frame is a file of 48 megs in size. These frames must be played back individually, up to 120 frames a second. This is a challenge for the storage.
But how do you get those DPX high-resolution files to play smoothly? Future storage is going to have to increase bandwidth exponentially. The days of 300MB/sec. working are gone. And stacking small RAID systems to get high bandwidth is a losing sum game.
We at Archion use our intellectual property to jumpstart bandwidth within the RAID controller. Using sophisticated algorithms to minimize latency and maximize bandwidth within our Velo, Omni and Omni Hybrid systems, we are able to get 4,000MB/sec. in a 24-bay 4RU.
Three: Latency — The Quicker The Better
Not only do you need to have throughput, but you also need to have the ability to get to the piece of material that you want to play. The time it takes to get to any piece of material is called latency. The time it takes for the heads on a traditional HDD depends on the position of the head and is distance to the data. SSDs or solid-state storage has a fixed latency — it’s very low.
With RAID systems built with SSDs, where the new NVMe PCI cards are really fast, the latency is fixed and the bandwidth is high as well as the price per terabyte. SSDs offer a limited number of writes, so you can run out of free blocks and then the drive can go down. They are fast systems, but they have a finite lifespan.
Looking ahead
Storage for digital video will certainly increase at least 6x over the next four years. While HDDs will still be the backbone of storage for the foreseeable future, systems such as our Omni and Omni Hybrid will provide the high bandwidth and low latency needed for DPX and future 8K workflows.
Storage technology companies who wish to stay relevant should plan to keep developing new high-speed technologies with the new SSD and NVMe systems. These new solutions will provide the highest bandwidth and lowest latency to support the next generation of requirements and demands for media & entertainment in 2017 and beyond.