Outlook: Overcoming latency for realtime collaboration
Rebekah Wilson
Issue: November/December 2020

Outlook: Overcoming latency for realtime collaboration

As developers, parents and teachers, we engage daily with audio professionals, musicians and students who have transitioned to collaborating and learning online, which in so many ways is lacking. We are driven by a fierce optimism that we will see in the coming years developments on the most profound, fundamental topic that affects remote collaboration for musicians: latency. We would love to see latency driven down, if not eliminated. This would allow the synchronization of musical events when we are distant from each other. However, as a blow to this aspiration, the laws of physics do not allow us to surpass the speed of light: even with intercontinental transmissions, we face latencies that are too high for playing in time together. 

We experience events as being heard at the same time only when they arrive to our ears with less than 30ms delay between them; even less for percussive sounds: our ears can detect delays as low as 9ms. Starting for 20ms speed-of-light latency in a vacuum between London and New York, add in the fact that consumer fibre optic speeds are at least 30 percent slower than the speed of light, in addition to routing and hop latencies and hardware and software media processing time, it is absolute that without a breakthrough in fundamental physics, latency is an unavoidable property of performing together when at a distance. Even when we have ultra-high-speed cables to our door, and ultra-low-latency hardware, such as those used in fintech: how can we perform on high-latency mobile networks, or across distant oceans, or even with a colony on Mars — where latency is up to 20 minutes back to Earth? 



As artists and audiences, we will embrace the inherent latency of remote collaboration and build rich applications on top of high-quality audio, video and data streams that know at all stages the position and trajectory of data as it traverses the network. We will share that information to synchronize our instruments and tools so we can play and move in time together; we will feed that information back into our performances. We foresee an sudden explosion of asynchronous time-based art-forms: pattern-based forms that change over time, advances in song-writing that embrace syncopation phrase-based performance, such as those inspired by the free jazz and improv music worlds. We'll see new influences of non-western musical forms such as Hindustani classical music, that have different concepts of time and synchronization. We will learn that playing together isn't necessarily about being in time together, as much as about starting and ending together. Yes, we'll always prefer to play together in the same room: acoustically, physically, emotionally — that experience can't be reproduced online. But as advances in networked music applications improve, performing and making music online together will become an increasingly-satisfying way of being together, even when we're apart.

Rebekah Wilson is the Co-Founder & CEO of Source Elements (www.source-elements.com), a company that helps connect creative people from around the world to collaborate in realtime on the production, approval and delivery of professional-quality media. She resides in Western Europe.