By Claudia Kienzle
Issue: June 1, 2003

STORAGE AND NETWORKING FOR NLE'S

When it comes to storage and networking, few applications are more demanding than nonlinear editing (NLE). Besides handling huge files, NLEs need data flowing in at sustained levels, without any glitches, hiccups or data loss. Storage failures compromise productivity and creative energy.

Nowhere is the need for speed more insatiable than the high end where editing high definition (HD) video is pushing the limits of network bandwidth and storage. There's a growing demand to network resolution-independent finishing systems (by such companies as Discreet and Quantel) to form "collaborative workgroups" where multiple editors have simultaneous, realtime access to near-film resolution video in central repositories.

The conventional wisdom is that only Fibre Channel (now 2Gbps), configured as a SAN (Storage Area Network), is up to this daunting challenge - making up to 400MBps of data available to each network client without bottlenecks.

But because Fibre Channel is such a huge capital expense, it's often deployed judiciously, only to those workstations that require ultra-high-bandwidth for the most demanding realtime editorial.

Unable to cost-justify Fibre Channel, the desktop NLE market - especially those working with non-realtime or compressed video - is finding Gigabit Ethernet (1Gbps) to be a better fit. At one-fifth the cost of Fibre Channel, "Gig-E" has reached a point where its price performance makes it a compelling alternative for a tight budget.

And, what about facilities that have say four high-end finishing stations and maybe six low-end desktop systems, all of which need to share the same data, but Fibre Channel is too costly to connect them all? Well, an interesting option has emerged called iSCSI, a newly adopted standard (not to be confused with SCSI or Ultra-SCSI) that will play a significant role in a growing trend called "IP SANs." IP (Internet Protocol) relates to the TCP/IP protocol central to standard networks like Gigabit Ethernet.

iSCSI routers, which have a Fibre Channel front-end and a Gigabit Ethernet back-end, are a hybrid solution bridging the Fibre Channel SAN with the Gigabit Ethernet workgroup to form one network on which anyone can share the same data, albeit at different data rates.

So, the controversy over whether to network NLEs with Fibre Channel or Gigabit Ethernet - or a combination of both with iSCSI - is only going to grow since hardware prices are falling while performance is rising dramatically. The vendors we canvassed have divergent views as to which approach is best suited to collaborative workgroups, from HD to DV, now and tomorrow.

Studio Network Solutions

"Only Fibre Channel guarantees high performance to every workstation on the network no matter how many users access the data at the same time," says Gary Holladay, chief systems design engineer for Studio Network Solutions (SNS) in St Louis.

"While Gigabit Ethernet is a switched network, it's a shared bandwidth environment," explains Holladay. "So, the more users ‘hitting the switch' at once, the lower the bandwidth available to each client on the network. Because Gigabit Ethernet is hampered by the overhead of TCP/IP processing, it's not sufficient to support several workstations requiring 170MBps for high-end video editing, especially with realtime effects.

"When several editors need to work on the same HD content simultaneously, the only way to guarantee that throughput to every client on the network is by using 2Gbps Fibre Channel. Below that HD benchmark, 1Gbps Fibre Channel is ideal for the demands of standard 601 editing," he adds.

"At SNS, we push the Fibre Channel hardware even further by optimizing the [Seagate] drives, and customizing the HBAs [Host Bus Adapters] for more streamlined, powerful and reliable SAN environments."

At NAB, SNS (www.studionetworksolutions.com) introduced SANmp Management and Sharing Software, which facilitates greater interoperability when sharing media between Windows and Mac OS platforms. When used with SNS's A/V SAN Pro, a SAN that scales from three to over 64 seats with up to 400MBps sustained throughput, users derive a multi-platform environment. SNS also offers A/V SAN, a two-user configuration - which, in addition to the A/V SAN Pro, was used by Grammy Award-winning music engineer David Gleeson in the editing, mixing, and recording of the music score for The Lord of the Rings: The Two Towers.

"Our customer base has grown largely by word of mouth. By seeing what we've done for others, people recognize that our integrated solution will add value to their SAN by simplifying installation and ensuring reliable operation," says Holladay. "Every business wants to save money, but cost cutting should not be done at the expense of workflow efficiency or the quality of the end product."

Huge Systems

"If you understand the fine points of Fibre Channel and Gigabit Ethernet, it becomes clear that Gigabit Ethernet is the better solution to people's file sharing problems," says Mike Anderson, chief engineer for Huge Systems (www.hugesystems.com) in Agoura Hills, CA. "When Gigabit Ethernet is employed for file sharing on a network, it qualifies as a ‘SAN.'" Anderson says that widely-held perceptions about Gigabit Ethernet are no longer true.

According to Anderson, in just the last year, many of the Gigabit Ethernet cards being built - by companies such as Intel, Q-Logic and Adaptec - now have something called "TOE" (TCP Off-load Engine) right on the silicon. "So now, instead of having the TCP/IP processing handled by software, the packets are now processed by dedicated hardware, which improves the efficiency and the speed of the network."

He notes what he calls a key problem with Fibre Channel: "it supports a very primitive protocol that's block-oriented with respect to the disks. But there's nothing in the protocol that will resolve contention to the same block on the drives - meaning that one user can write over another's data," says Anderson. "So, if two Windows servers are ‘talking' to the same Fibre Channel disk drive, both would be able to write to the same block without restriction, rendering that disk unusable." People get around this shortcoming by adding expensive SAN management software.

"But Ethernet inherently supports a whole set of protocols intended for sharing devices and already has built-in mechanisms to coordinate file sharing, so no SAN management software is needed," Anderson reports. "And since the majority of content creation facilities are cross platform, people should know that ‘Gig-E' is very compatible with many file sharing protocols and OS, including Microsoft's SMB, Apple-Talk and Linux, and many PCs, Macs and servers already have a built-in Gigabit Ethernet port to plug into."

Huge Systems provides the rest of what's needed in its integrated product, SAN Stream. It's a hardware disk array that combines a network switch, server, 24 Gigabit Ethernet ports, RAID-3 protection and (low-cost ATA drive-based) storage, all from one vendor.

"Since the market for Gigabit Ethernet is 100 times bigger than that of Fibre Channel, prices are as much as 80 percent lower. The speed is sufficient for video editing and very soon, 10Gbps Ethernet will increase that speed ten-fold," says Anderson. "Soon after our solution was introduced at NAB, we began taking orders for it. When folks first hear about it, they say, ‘It'll never work. But if it does, I want one.'"

RAID INC.

"As the industry moves up to HD resolution, many post houses are making a sink-or-swim decision to purchase the necessary storage and networking infrastructure to support HD," says Matt Goulet, sales and marketing manager for the AV division of RAID Inc. (www.raidinc.com) in Lawrence, MA. (RAID Inc.'s AV division was created with the 2002 acquisition of Boston-based JEMS Data). "While the HD market will grow over the next five years, the dilemma is that there's not yet enough HD business to justify the capital expense. But if they don't make the upgrade, they won't be positioned to attract any HD business.

"What our new X2i iSCSI networked storage system will enable them to do is to deploy 2Gbps Fibre Channel between HD editing workstations where performance matters, then join that SAN to their SD editing workgroup connected with Gigabit Ethernet so that both can share the same data cost effectively," says Goulet. "It's the best of both worlds - the high performance of a 2Gbps Fibre Channel SAN with the economy of a Gigabit Ethernet infrastructure in one iSCSI solution."

In terms of front-end host connectivity, "the X2i accommodates 10 2Gbps full-duplex Fibre Channel ports and two Gigabit Ethernet ports. On the back end, it allows two 2Gbps storage processors, each of which has a 200MBps Fibre Channel port. Two redundant loops provide up to 800MBps. Multiple iSCSI routers can be joined for bigger installations," reports Greg Morris, sales engineer for RAID Inc.

"Leveraging Cisco System's 5428 iSCSI router, the X2i uses the SCSI protocol in 2Gbps Fibre Channel and converts it - the size of the blocks, frames and packets, etc. - into a form compatible with TCP/IP for data transfer over Gigabit Ethernet." he adds. RAID Inc. also offers the X2A/V 2-Gbps Fibre Channel storage solution which can be used as centralized storage on a SAN or as Network Attached Storage (NAS). The modular three-unit rack-mount design scales up to 18TB of storage and is fully redundant. Also, RAID Inc. offers StorageWatch, a service that allows customers with qualified products to connect to RAID, Inc.'s redundant Network Operations Center so the vendor can remotely monitor and administer their systems.

Medea

"Our customers want an affordable solution that allows them to collaborate within a shared storage environment," says Roger Mabon, VP of channel marketing for Medea (www.medea.com) in Calabasas, CA.

"That is the market we're focusing on - both with our current products and ones in development," he continues. "We are evaluating cutting-edge SAN software, in the context of popular hardware configurations, with the goal of recommending highly-reliable, affordable shared media environments that make financial sense for the desktop video segment."

Medea has two next-gen disk arrays - VideoRaid FCR2 and VideoRaid FCRX2 - which are SAN-ready; offer fail-safe RAID protection; dual, redundant load-sharing power supplies; and take advantage of 2Gbps Fibre Channel speeds with data rates of 200MBps per channel and 400MBps per enclosure.

"These new disk arrays are ideal for building high-bandwidth, low-cost SANs for demanding digital content creation applications," Mabon says.

VideoRaid FCR2 disk arrays are housed in a compact five-drive desktop tower, which can be configured for 720GB for under $4,899. And, VideoRaid FCRX2 features a three-unit, rack-mountable design that can be populated with five or 10 disk drive modules, with capacities up to 1440GB per rack, starting at $5,599. Also new from Medea is the VideoRaid RT3, the first Ultra-DMA or IDE-based Ultra-320 SCSI disk array. "At NAB, we had a single five-drive unit playing 1080i HD in realtime from a Pinnacle CineWave HD system, and people were amazed that that level of performance could be sustained by such a compact, affordable array, with just five IDE disk drives," says Mabon. While the five-drive array offers 140MBps sustained, it's possible to get 250MBps, for near-film resolution, by striping across two arrays.

For DV, Medea now offers FireRaid, an innovative 500GB FireWire storage solution (for $1,199) that allows users to pull one of the system's two modules from the base unit to take on the road for DV production.

ADIC

ADIC's StorNext File System is a software solution that allows users working with different applications on multiple OS platforms to access and share the same disk volume from centralized SAN storage at Fibre Channel speeds. "Considering that one second of 2K film resolution imagery can exceed 1GB, and that such projects typically involve over 100,000 frames of 2K imagery, copying this volume of data and moving it over standard networks takes an inordinate amount of time - time during which editors and artists sit idle waiting to start the work," says Steve Rovarino, business development manager for rich media initiatives at ADIC (www.adic.com) in Redmond, WA.

Effective SAN management, using StorNext File System, is an important component for any post environment because it shortens timelines, improves productivity, cuts down on storage inefficiency and eliminates file transfer bottlenecks," he adds. "Because StorNext Management Suite uses a ‘kernel attached' architecture, our file system can co-exist with the file systems on IRIX, Linux, Windows and other OS with performance matching that of the native OS. Users with our software on their workstations can see all the files stored on the SAN and access them in their native file formats at Fibre Channel speeds." The original data is preserved and new versions can be tracked.

Digital FilmWorks (DFW), a producer of motion picture digital effects in Hollywood, installed ADIC's StorNext File System in 1998 when DFW realized too much time was being spent copying huge files and moving massive amounts of data on tape between workstations, film scanners and recorders. Today, DFW president Peter W. Moyer says, "We have eliminated storage servers and duplicate files, and each workstation directly accesses the same data for true data sharing from workstations running different OS. And, when revisions are needed, we can often make the changes while the client is in the studio, something we could never have done with the old system." StorNext File System was also recently installed by Rainmaker in Vancouver to provide a cross platform environment sharing 2K images across SGI and Windows.

EMC

EMC Corporation acquired the Clariion brand of networked storage devices from Data General in 1999 and continues to market and grow the product line. "We feel that Data General had created a storage system that is very complementary to EMC's own broad product line," says Jay Krone, director of Clariion product marketing for EMC (www.emc.com) in Hopkinton, MA. "While we've preserved the original product architecture, last August we launched a new version of Clariion called the CX Series, which we feel is the product to beat."

Besides the system's (2Gbps) Fibre Channel disk drives, "this past March we added the option to use lower cost ATA drives, and put them in a ‘rack-n-stack' arrangement behind the same controller, dropping the price of the individual disk drive by a factor of four and the price of the whole system by a factor of two," Krone explains. "While costs are lowered, bandwidth and storage have been improved. We can put 50TB in a single Clariion CX storage array that consumes only 1.5 six-foot racks of space."

The Clariion CX Series, including CX200, CX400 and CX600, is fault-tolerant and directly supports Windows, Linux, Sun Solaris, SGI IRIX and HP operating systems. EMC also offers a suite of Clariion software that simplifies every aspect of SAN management.

CIPRICO

With its DiMeda 2400 storage appliance, Ciprico combined the performance of a SAN with the reduced cost and ease of use of a NAS (Network Attached Storage). To do this, DiMeda delivers a high QoS (Quality of Service) to any networked client requesting access to shared centralized storage, without the complexity and expense of Fibre Channel, because it uses optimized Gigabit Ethernet for data transfer.

"DiMeda is a NAS with software optimization that provides Bandwidth Management, a QoS feature allowing customers to provision bandwidth on the storage device per the needs of a particular application. Depending upon each client's priority, the software allocates the shared resources so the most important needs are met," says Mohan Mysore, broadcast and entertainment market manager for Ciprico (www. ciprico.com) in Plymouth, MN. With roughly 100MBps available on Gig-E, the system can give seven clients 14MBps each. But should an eighth user try to access the storage, exceeding the network's aggregate bandwidth, the software re-apportions the bandwidth according to which application is more critical and time-sensitive; e.g., digitization would take precedence over browsing.

For 2Gbps Fibre Channel-based SANs, Ciprico also offers the FibreStore family of SAN-ready products, including the FibreStore 2210, with performance up to 400MBps per enclosure and 1.46TB per array.

DATA DIRECT NETWORKS

"What we're seeing now is growing demand for high-performance storage systems, such as our Silicon Storage Appliance, because of HD requirements," says Paul Bloch, president/co-founder of Data Direct Networks (www.datadirectnet.com) in Chatsworth, CA. "We're seeing much more opportunity in post production this year than ever before," he continues. "We can help our customers with HD, 2K and even 4K, all building on the same architecture." With Data Direct's Silicon Storage Appliance (S2A), users have flexibility and scalability from a handful of disks to 130TB of data on over 1,000 disk drives on the (S2A A8000) system. "We also offer intelligent advanced drive management techniques, multi-dimensional RAID protection, cross-platform access and other capabilities that help customers get the most from their SANs," adds Bloch.

Data Direct has two models of its Silicon Storage Appliance particularly well suited to high-end NLEs: the S2A 8000 and the S2A 3000. The flagship S2A 8000 exceeds 1GBps (bytes, not bits), a speed that is sustained in both read and write applications when Fibre Channel disks are used. The S2A 8000 can be configured as a single two-unit system with four 2Gbps ports or as a dual four-unit system with eight 2Gbps ports. Bloch says, "One of the Fibre Channel ports can be used for a NAS, allowing Macs or PCs on the standard network to access the data stored on the SAN."

While the S2A 8000 starts at about $75,000, there's now an entry-level storage system, the S2A 3000, starting at $40,000. This system delivers 600- to 700MBps sustained throughput and scales up to 200 disk drives, or over 30TB. Both systems can be configured using lower-cost Serial ATA drives, but Bloch says, "Only Fibre Channel can guarantee the high-quality of service required for HD and other high-end post applications."

EXAVIO

Recognizing that digital media is inherently different from transactional or other types of data, Exavio designed its ExaVault storage sub-system to reduce the cost of a high-performance network, while optimizing it for the exacting requirements of digital media content creation and delivery. The key distinction with digital media is that large, valuable data blocks must be accessed and read many times in their entirety, resulting in uncompromised throughput, storage and security requirements.

ExaVault addresses these needs through its architecture - a Linux server, ExaVault configuration software (for RAIDs level 0, 1, and 5) and storage arrays scale up to 3TB within a single rack-unit frame, and 120TB when multiple units are joined for massive storage. But, rather than using Fibre Channel disk drives, ExaVault uses lower-cost IDE/ATA disk drives, a FireWire-based back plane (transparent to the end user) and Gig-E and IP connectivity across the network.

"We believe that Gigabit Ethernet is a very competent solution for SD video applications, and it provides a very cost-effective alternative to Fibre Channel," says says Ji Zhang, president/CEO of Exavio, Inc. (www.exavio.com) in San Jose, CA. "For uncompressed HDTV, which exceeds 1Gbps, there is no way for Gigabit Ethernet to support that right now. However, this will change in about one year because 10Gbps Ethernet, with 10 times the throughput of Gigabit Ethernet, is just around the corner, and in the long term we believe that it will offer the best cost-efficiency for the future."

While the hardware to support 10Gbps Ethernet is now available, Zhang anticipates that, with the dramatic price drops typical of open standards network solutions, next year the prices for 10Gbps Ethernet will converge with today's 2Gbps Fibre Channel and eventually surpass the price performance.

"The cost of a 1Gbps Ethernet adapter has dropped from $1,000 to $50 dollars, or $50 per gigabit; compared to a 2Gbps Fibre Channel adapter, which is now $1,200 or $600 per gigabit. Also, a Fibre Channel switch is about $1,000 per port while it's only $100 per port on the Gigabit Ethernet side, or one-tenth the expense," explains Zhang.

According to Gary Law, Exavio's VP of marketing, "Until now, Fibre Channel has been the only means of offering the high performance needed to network multiple NLEs for digital content creation. But Gigabit Ethernet is rapidly catching up. Soon, post professionals will be able to have 10Gbps Ethernet installed for demanding realtime HD applications, Gigabit Ethernet deployed for multiple SD, DV or MPEG compressed NLE, and all of it compatible with IP-based wide area networks, which don't suffer from the geographical limitations of Fibre Channel. IP technology has won as the dominant networking protocol out there, so it just makes sense for post professionals to build on one flexible, scalable network that can meet all their needs very cost effectively."

Lastly, when asked if Gig-E could erode Fibre Channel's foothold in post production, especially for HD, proponents of Fibre Channel point out that a 10Gbps Fibre Channel protocol is on the horizon and that prices are falling. While it's difficult to determine which networking approach will be the most powerful, cost effective for post, the one that delivers will be extremely appealing, especially to the growing base of Apple Final Cut Pro users. Since Final Cut Pro systems can be fully-configured for HD for under $25,000, this is a very price sensitive market segment - but one which is showing strong demand for file sharing and creative collaboration if there's a high-performance solution that fits their budgets.

OTHER OFFERINGS

Obviously, due to space constraints, we couldn't interview all storage companies. Here's a short list of some others that are offering product for editors…

BlueArc (www.bluearc.com), with its optimized Si8000 SiliconServer NAS System, combines the ease of management and configuration typical of a NAS with the scalability of a SAN. It scales from 500GB to 200TB, with the ability to add disks and controllers, and monitoring via the Web. The Si8000 takes advantage of 10Gbps Ethernet networks but can also benefit from a Fibre Channel SAN at the back-end.

Dot Hill (www.dothill.com) offers the SANnet II Fibre Channel 12-drive RAID storage system. Designed for applications such as image processing and media streaming, it stores up to 1.75TB in a two-unit enclosure, expandable to 28TB. Its two 2Gbps Fibre Channel host connections provide sustained data transfer rates up to 1,400MB per sec. and up to 160,000 I/Os per sec. for high performance and data availability.

Glyph (www.glyphtech.com) offers Trip2, a hot-swap FireWire storage product designed for video and audio post. It offers hot-swappability of two to six drives, which may be SCSI/FireWire, AIT backup, SCSI and/or FireWire DVD-R/RW drives.

nStor (www.nstor.com) offers the 4320F 2Gbps Fibre Channel disk array storage system, which supports four independent host connections, built-in volume sharing, dynamic cache allocation, and more. The 4320F is based on nStor's new dual-active 2Gb Fibre Channel RAID controller. NStor also offers StorView Web-based storage management software for an intuitive visual user interface and remote network support capabilities.

Rorke Data (www.rorke.com) offers a broad array of storage solutions, including ImageSAN OS X for Mac-based SANs. ImageSAN OS X enables efficient SAN workflow by enabling high-speed file sharing and creative collaboration, even for HD resolution video. ImageSAN OS X is targeted at Apple Final Cut Pro users who wish to share files using SAN technology. Rorke also offers ImageSAN for the PC, including the Windows 2000, XP and NT platforms, which includes multi-thread access to files.

Rorke also offers the Galaxy 60 2Gbps Fibre Channel RAID system that delivers performance ranging from 200MBps to 400MBps. There's also the Galaxy 60 JBOD 2Gbps Fibre Channel which has throughput speeds from 200MBps to 800MBps.