Before Dave Brailsford became involved in British cycling in 2002, Great Britain and Northern Ireland were nonentities in the cycling world. In the history of the modern Olympics, only 14 Brits had ever won a gold medal in any cycling event. And on the road, in 65 years competing, no Briton had ever won the Tour de France. Fast forward to today, and Team GB cycling has won a total of 38 Olympic gold medals and the most medals of all colors of any nation, and the Union Jack has now been raised six times at the end of the Tour de France.
Most pin this success on Brailsford applying a simple principle to every aspect of preparation by his team: the accumulation of marginal gains. That is, to break down everything they could think of that goes into competing on a bike, and then improve each element by 1%, adding up to an overall net gain in performance.
Many of these measures have become part of the fabric of professional cycling, and the marginal gains ethos is now entrenched in almost every sport played on the planet. Today, sports scientists use video and data to make ever more fine-grained marginal gains in almost every aspect of professional sport, including but not limited to player recruitment, nutrition, training, performance, and rehabilitation. And becoming key to the marginal gains blueprint is optical motion capture.
Motion capture, or MoCap, in sport has a lineage that stretches back to Eadweard Muybridge’s famous 1878 experiments using a series of 12 cameras to record the gallop of a horse and settle what was then a bone of contention—whether all four hooves leave the ground at once. Modern MoCap is significantly more advanced, showing movement in 3D space, from any angle, in micro detail.
The gold standard is marker-based MoCap—offered by companies including Vicon and OptiTrack—which exploits cameras to track the motion of reflective markers attached to specific locations of the subject’s body. “Marker-based tracking evolved into what it is today after decades of refinement,” says OptiTrack Director of Software Anthony Lazzaro. “It’s hyper-optimized to provide submillimeter accuracy with minimal latency,” he adds, making it the go-to choice for anyone wanting to capture precision movements in almost real time, from Hollywood studios working on the next Marvel movie to veterinarians assessing whether a limping dog is in pain.
Marker-based systems can be active or passive. Both take advantage of the infrared portion of the electromagnetic spectrum to minimize interference and improve accuracy. Passive MoCap attaches retroreflective markers not unlike the reflectors on a child’s bicycle, and uses high-frame rate (up to 2,000 frames per second), high-resolution (up to 26 MP) cameras surrounded by an array of LEDs that strobe infrared light. This light is reflected by the markers and detected by the cameras, from which in-motion analysis can be performed with specialist software.
For example, a passive T40 Vicon eight-camera motion capture system was used in a 2017 study to better understand the complex set of movements that go into serving the ball in tennis. The aim was for these insights to be used to improve athlete performance and reduce overload injuries. Ten ranked players had 43 markers affixed to them and five additional markers were affixed to their rackets. The players proceeded to serve multiple times. Results exposed how top players attempt to maximize external rotation of the shoulder before hitting the ball, suggesting this should be an area of focus for exercise and training.
Active MoCap is similar to passive, just with one key change: Markers have a built-in light source, typically an infrared LED, whose light is tracked by purpose-built cameras. The main visible difference compared to passive MoCap is that athletes typically wear special Lycra suits that hide the wires connecting and powering the LEDs. Because each marker can be tuned to have a unique ID, active MoCap systems are the least susceptible to errors and regarded as the most accurate.
This is why active MoCap is the go-to system for sports with rapid, complex movements. For example, a growing number of Major League Baseball’s pitchers suffer elbow injuries. This is why the Chicago Cubs decided to team up with Japanese sports science firm Next Base, whose biomechanics insights had been preventing pitcher injuries in the Nippon Professional Baseball leagues for a decade. Captured by 14 high-performance one-thousand-frames-per-second cameras, 48 markers are placed across the athlete’s body—including numerous markers attached to the fingers from the joints all the way up to the tips of the fingernails—for measuring movement. This level of detail offers unprecedented insight into athletes’ body motion, enabling Next Base to suggest safe and effective training protocols tailored to each athlete’s individual needs.
As these examples show, marker- based optical MoCap systems have made significant inroads in the sports world. But they suffer from two critical weaknesses. “An athlete wants to be in their natural element to perform their best and not be distracted, and lots of sports require special uniforms, pads, or other types of clothing,” shares Lazzaro. “Wearing a motion capture suit [or markers] just isn’t feasible for these use cases other than for specialized analysis.”
OptiTrack’s Motive software processes thousands of data points instantly, creating highly detailed 3D motion reconstructions. Photo credit: OptiTrack.
Moreover, professional-grade optical MoCap systems can cost several hundred thousand dollars, far out of reach of junior and lower-level teams and athletes and, in fact, many sports. Is there a cheaper way to find those marginal gains for less well-funded sports or athletes striving to step up to join the elite?
In the past five years, technological progress in AI and deep learning has brought motion capture full circle. Standard video cameras are the only hardware needed when AI trained on massive datasets of human motion is analyzing footage to accurately reconstruct 3D motion. With no Lycra suits, no markers, and no infrared cameras necessary, markerless MoCap is not only significantly cheaper—usually far less than $100,000—it captures the natural motion of the athlete(s) on the pitch, in the velodrome, or at the pool with no physical constraints or inhibitions.
“Athletes and coaches do not have time to put all the markers on and set up all the cameras,” confirms Dario Cazzola, associate professor in computational biomechanics at the University of Bath. “You need invisible monitoring: It’s the jackpot when you can get the data very easily, and they can get the analysis, the insight, and use that insight quickly.”
Though budget markerless MoCap apps have emerged that solely require a single smartphone camera for data capture, their accuracy and validity largely remain questionable, and their faculty to be used to improve sports performance in anything more than very basic ways is still unclear. Multicamera markerless MoCap systems such as Theia3D, Simi Shape 3D, and KinaTrax, hold far more promise in delivering actionable insights for athletes, coaches, and teams.
For a sport like college football, where budgets—though large—are not limitless, more than 100 players might need to be assessed quickly, and many in-game movements need to be analyzed in the natural environment of the practice or football field, these markerless systems are ideal. Virginia Tech and the University of Oregon have publicly documented their use of Theia3D technology, while the likes of the University of Kansas, University of Florida, and others have declared that they have adopted markerless systems, too.
ForceTeck, a spin-out of the University of Bath, has not worked with American football teams, but has developed their markerless offering in collaboration with the Bristol Bears rugby team. “I went to Bristol Bears with a PowerPoint of ideas and a very initial research software prototype,” recalls Cazzola, who is also ForceTeck’s founder and CEO. At the time, the Bears were collecting sprint data across the whole team twice a week and struggling to analyze this video quickly. Cazzola set himself the challenge to provide that data and analysis on the pitch in five minutes. Fast forward to today, and the Bears are now one of ForceTeck’s prime customers.
ForceTeck’s technology consists of three types of models. The first is a computer vision model to extract motion from videos and track joint positions and movements. These video recordings can come from any camera with sufficient spatial and temporal resolution for the application. Cazzola says his team can even use iPhones and iPads: “They have a reasonable number of pixels, and you can get up to 240 fps, which is acceptable for many applications.”
Data from the first vision model is then fed into a unique physics-informed machine learning model. This is where the technology goes beyond motion tracking, estimating, for example, the ground reaction force an accelerating sprinter generates, or the collision forces involved in a rugby tackle, all to deliver a better understanding of the physical demands placed on athletes, and for coaches to thereby help improve performance and avoid athlete injuries.
The final model is a generative one to create realistic simulations of how players might perform under different conditions, helping coaches test training strategies and assess injury risks in a safe virtual environment.
How do this and other markerless systems stack up against traditional marker-based systems in terms of precision and accuracy? Though marker-based systems are widely regarded as the gold standard, and often used to bench test new markerless systems, Cazzola sees things differently. “From a biomechanics perspective, if I have an X-ray, and I can see the bones in your body while you’re walking, that’s the ground truth,” he argues. “We want to track the joint position, but we are not rigid bodies and if you put the marker on your muscle, the marker will wobble a lot—so estimation is actually probably better with a markerless system than a marker-based one.”
Though incumbents such as Vicon and OptiTrack would strongly disagree with Cazzola, all this progress in markerless technology has not gone unnoticed by them. Both are introducing markerless offerings. Vicon’s markerless system captures data through the purpose-built Vanguard camera, and offers real-time full body tracking via proprietary machine learning algorithms. And OptiTrack recently launched a hybrid system: Duplex Mode.
A golfer’s swing captured in real time with OptiTrack motion capture. Photo credit: OptiTrack.
Duplex Mode aligns the high-precision marker-based tracking capabilities of OptiTrack’s X series cameras with software company Captury’s real-time markerless tracking technology driven by machine learning. “The system at its core is pretty simple,” says Lazzaro. “It allows the camera to both transmit object data [for traditional marker-based tracking] and video data [for markerless tracking] at the same time from the same camera—you’re able to maximize the benefits of markerless MoCap and also track anything you can put markers on.”
This means a baseball coach, for example, can use a single system to both help improve a batter’s swing by leveraging the submillimeter accuracy and minimal latency of marker-based MoCap in a controlled environment, and then move the cameras pitch-side to perform quick, high-volume analysis of markerless data for in-game performance insights.
Where does all of this progress in markerless MoCap leave junior and amateur athletes and teams? Though such systems only require cheap hardware in contrast to their marker-based counterparts, employing them in a sports environment still comes with a cost in the tens of thousands of dollars bracket. Why? Cazzola says that the real cost of markerless technology comes from the expert biomechanical and coaching analysis that companies like his can provide, as well as all the computing costs associated with transforming video of an athlete performing their sport into data that can deliver actionable and valuable insights. He adds: “These costs need to go down a lot for markerless MoCap to become widespread.”
While the high cost of analysis keeps this technology largely in the hands of the pros for now, the trajectory is clear. As AI models become more efficient and capable and cloud computing costs drop, the gap between elite and amateur athletes will narrow. Soon, the same technology delivering marginal gains for your favorite sports team could be in the hands of any coach or aspiring athlete, helping everyone put their best foot forward.
Benjamin Skuse is a science and technology writer with a passion for physics and mathematics whose work has appeared in major popular science outlets.
