For years, there has been demand for a new way to manage and automate data movement from on-premises production environments to the cloud. It has also been in the interest of media and entertainment companies to gain actionable insights into the operating expenses related to video storage and migration through cloud-based data movement services.
The emergence of COVID brought this all to forefront, as the trend has been to reduce video content management costs as budgets tighten. Customers are looking for deeper intelligence into what is driving content management expenditures and where can they be controlled. The ability to more closely track overall cloud spend, so that users know their cloud capacities, and what is stored where will increase in priority over the next several years.
According to recent research by Technavio, the cloud migration services market is expected to grow by USD $7.1 billion during 2020-2024. The growing adoption of cloud-based migration is encouraging the development of a wide range of innovative solutions specifically targeted to businesses and other organizations as they move to reduce expenditures. This has been one of the primary factors driving global cloud migration services’ market growth.
The acceleration of digitized media is creating large amounts of data, which has to be processed, managed and migrated to enhance revenue-generating potential. After digital content has been reviewed and analyzed, it must be stored for long-term accessibility. Producers today are looking toward cloud infrastructure for the storage and administration of rich media to complement and possibly supplant portions of traditional in-house applications and systems.
Unfamiliar with the pricing schedule and caveats found with many public cloud provider storage and processing services, many customers find that the removal or transition of data to and from the cloud to be not only complex, but very expensive in comparison to local production operations. Being able to weigh the pros and cons of performance versus the cost of using these services compared to on-premise operations is an important consideration for many customers. Until now, there have been too few tools available to help IT solution providers and production house customers find the right mix of solutions.
According to Tom Coughlin, president of Coughlin Associates, "Data is the life-blood of modern organizations and moving that data around is critical to the organization’s operations. For that reason, data mover software that controls the migration of data, as well as its movement between clouds and on-premises operations, is a vital operational tool. This is especially important during the COVID-19 pandemic, when most work is being done remotely. These tools must be able to help organizations balance the capital costs for on-premises storage and processing against the operating expenses (OpEx) of storage and processing in the public cloud."
The requirements of organizations today call for solutions that predict the cost and speed of data transfer to and from the cloud, and save time and money. A new breed of data management software is putting users in control of their budgets and providing insights for making smarter storage decisions. Developers of these solutions have designed products that work with all types of storage — on premise, hybrid and cloud. Users can define detailed policies or manually transfer data. Add to this the ability to automatically monitor and track every job, generate reports on demand for departmental chargebacks and client billing, and flexible integration with third-party applications to bring alignment with a variety of workflows, and organizations are compelled to act. Next generation solutions even go so far as to offer unlimited data movement and subscription-based pricing to support a broader range of customers.
"When using remote cloud-based operations, movement of data is required,” says Coughlin. “Much of this locally-generated data is unstructured and needs to be moved to a private data center (private cloud) or to a public data center for analysis and perhaps for longer term storage (including disaster recovery, data protection and archiving). Data movers may also be used to move data out of the cloud or between various remote data centers (multi-cloud operations). The costs for moving data often depend upon the required performance levels but are ultimately gated by the organization’s network performance."
Select systems allow for test-runs that deliver predictive analytics so that customers are able to model the cost and duration of proposed data migrations to and from the cloud. In these cases, the producer's content can be moved automatically and efficiently, according to a policy-based migration. The more effective data movers oversee cost and data migration resource visibility. AV production may use this information to help determine which of the available cloud environments are most appropriate, not only in terms of cost, but also in services and performance levels that meet customer service level agreements (SLAs).
Post production professionals can benefit immensely from new data-mover solutions. Most production has only been possible during the COVID-19 pandemic because projects have been orchestrated remotely, often using cloud-enabled tools. These environments are using video proxies and collaborative workflow solutions to facilitate workforces nationally, and often internationally. When shifting workloads to and from the cloud, the industry's newest data-movement tools allow users to set up a cloud transfer budget in order to monitor how individual file transfers affect costs with a realtime dashboard. A budget limit can be set to prevent excess spending. Data transfer jobs are tracked and itemized so that chargeback routines can be implemented.
Because of various performance allocated storage environments and data transfer performance rates involved in video migration, as well as the increased use of multiple clouds for various operations, the process of calculating costs for different options can be complex and expensive. At the same time, most organizations are sensitive about the cost of operations and, for a large organization with multiple cost centers, being able to allocate fees to the appropriate cost center is critical for sustainable operations.
Cloud data services charge differently for various types of storage, depending on the speed available to access the data and for large movements of data. They also charge different fees for bulk data movement, standard data movement and expedited data movement. Realizing the options available among different services and organizations is critical to vetting the preferred vendor. Data movement solutions must integrate with cloud-native APIs to move past the need for constant integrations and management of the process. In one example of an organization that relies heavily on video content captured from games, practices and interviews, their legacy storage infrastructure could not support the move to 4K and 8K video, nor the performance and storage requirements needed. The group needed a simple and cost-effective way to migrate off their legacy storage system. Using next-generation data mover solutions, they were able to tackle their Tier 1 and cloud migration, leveraging the elasticity of the cloud. They even brought their archive online. The data migration software was installed and configured on a VM environment. It enabled content migration from legacy storage to a new high-performance SAN.
The data-mover solution was integrated with a media asset manager (MAM) and provided with automated, policy-based data movement from on premise to the public cloud. Furthermore, a dry run feature provided insight into cloud storage and retrieval costs. As a consequence, the organization’s active content is now searchable and available to users, and the automated migration reduced required IT resources. This not only saved time and money, but minimized on premise storage costs by using elastic cloud storage coupled with realtime analytics to control the cloud budget and spending.
Electronic sports — or esports using video games — is a rapidly expanding entertainment industry. An active electronic sports company has multiple publicly-viewed games and leagues that are inspired by card games, board games and role-playing fantasy (RPF) games. The media workflows include videos of competitions with players, in-game footage and commentary. Media is also needed to create in-game video content. This company had disparate islands of data, with cloud data stored in proprietary formats and workflows. Content could not be searched quickly and easily, and the company needed more insight into cloud costs.
A video management and automation tool was implemented with its data migration platform in a hybrid cloud deployment, with archive on ingest into a cloud-based archive. The data mover transitioned data to and from the cloud within the integrated tool. Video editing is now done using lightweight video proxies, which are streamed to its video editor. As a consequence, its gaming videos, legacy game material, artwork and promos are now searchable and available to multiple groups. Editors can now work from home (WFH), edit via proxy and conform on-premise, increasing productivity. The realtime cost analytics help control the cloud budget and spending.
"Today’s data growth and the availability of fast networks is enabling organizations to combine on-premises storage and processing with cloud and multi-cloud elastic storage and processing resources,” notes Coughlin. “The right data migration platform should provide easy-to-use data movement that allows clients to understand and control the costs of data movement to and from the cloud."
Greg Holick is vice president of product management for IMT Software (https://cloudsoda.io). He oversees all aspects of shaping product direction and the roadmap for IMT’s software division and inaugural product, SoDA. Holick works closely with channel and workflow application partners to bring innovative software tools and applications to customers that automate and simplify business operations, workflows and hybrid cloud data management.