Michael Nikonov is the Founder/Chief Technology Officer of iPi Soft (http://ipisoft.com), the Moscow, Russia-based developer of markerless motion capture software. Here, he talks about the challenges mocap technology developers face in terms of accuracy and realtime performance.
Over the past seven years when we first entered the market with iPi Motion Capture, the markerless motion capture industry has moved quickly from academic experiments, to affordable, accurate and easy-to-use production tools that are unconstrained by sensor suits and greenscreen stages. Increasingly, large film and game studios, as well as independent content creators, have adopted our image-based markerless motion capture systems for animating human characters, finding it especially helpful in previs and simulation of large crowd scenes.
We are encouraged that our markerless motion capture technology continues to be leveraged effectively by high-powered post houses in 2016 on a range of projects, including Iloura’s Emmy Award-winning “Battle of the Bastards” episode on HBO’s Game Of Thrones. It also proved central to both VR productions and YouTube content creators who cited the software’s simplicity and flexibility to track multiple people without the need for expensive suits or marker placement, as well as the ability for one artist to handle the entire capture process — from performance and data management to processing. However, for the majority of entertainment content creators on the high-end, a realtime solution remains mission critical.
While realtime capture exists in some markerless motion capture systems (the realtime version of iPi Motion Capture is scheduled for release in 2017), for more sophisticated production pipelines, the barriers to deliver an affordable realtime, markerless solution are constrained in part by the mismatch between huge computing power requirements of markerless motion capture and the actual capabilities of modern personal computers.
As software developers, we are constantly aiming to improve speed and accuracy. This year, we were able to deliver processing speeds four times faster than previous algorithms at 15fps, very close to realtime, for customers using standard, off-the-shelf Kinect devices, with twice the accuracy. Others in the markerless mocap market have had their own successes as well. Although realtime solves the problem with the speed, there remains the issue with accuracy, and for that we must rely on companies like Intel and Nvidia to continue advancing faster processors and video cards, respectively.
We are also encouraged to see game engine companies like Unity and Unreal that enable game developers to incorporate markerless motion capture into the creative pipeline to produce amazing 3D animated games with limited investment. I expect that will continue in the New Year.
While significant challenges remain before us, we are hopeful 2017 will bring new hardware and optimizations of software. This will in-turn allow us to release our affordable solution with realtime speeds and immediate feedback for mocap actors, resulting in an overall improved and reliable markerless mocap workflow for creative users across the production spectrum.