In the video above, Lambert shows how the CV tool identifies the balls’ spin axis with a neon green dot and the seam orientation in other neon hues.
“I think that’s probably the most applicable use of CV at this point is that… getting some of the metrics I can’t get from Hawk-Eye,” Lambert said. “I’m sure you can imagine if my high school brother in Cincinnati is throwing a bullpen and just doesn’t have a Trackman available, if we can get some footage, get some estimates for what’s going on, we can better like adjust that process from there.”
A computer vision system learns by analyzing thousands upon thousands of labeled images – sometimes even millions like in the case of something Tesla’s early self-driving efforts – using convolutional neural networks to then identify patterns and understand spatial hierarchies. This is deep learning.
Boddy and others at Driveline did tons of labeling, heavy lifting, to train the system – labeling seams, spin axis, and pitch types of thousands upon thousands of recorded offerings. The system is still learning, it’s still getting better.
Lambert made about 50 throws earlier this month, studying the ball flight and impact of each adjustment guided by feedback from Driveline’s real-world AI effort.
He wasn’t able to perfectly replicate the pitch in one bullpen, but he was able to mimic some of its characteristics after just one grip and release recommendation from our computer vision model and some tweaking.
“What I was able to recreate was I could get the high, arm-side run that he would throw,” Lambert said. “I could not kill the spin efficiency enough to get the gyro action. I found it easier to basically create a changeup profile with a supinated (release) than to create the true gyro version of his slider. That was most of the iteration process. It actually didn’t take that long to produce some pitches with the high, arm-side run.”
Imagine what actual pro and college pitchers and coaches might be able to do with the tool?
That is one application of a computer vision model: helping coaches and players understand how to begin with a pitch.
Driveline’s pitching director Connor White explains the other great benefit of deep-learning aided pitch design.
“The speed of analysis is one of the most exciting things,” White said. “We want to keep those pens game-like. So, if that’s having to stop after every pitch and look at a bunch of metrics and consult the video, and next thing you know it’s been a minute between pitches or more it really kind of breaks that flow… The computer vision allows you to look at the observed versus like spin-based (movement), getting the closer to the ball physics of what’s happening in real time.
“The speed at which these (advancements) can be applied is just so exciting.”
Shortening the feedback loop, the understanding of what a pitch is doing, is indeed exciting.
Our computer vision model is not a finished product, but it is already having results in our gyms.
Driveline pitching trainer Grayson Liebhardt says it’s already helping him as a coach.
“It’s a really helpful tool,” Liebhardt said. “It’s earlier in development but it’s helping us bridge the gap, and understand seam orientation without any access to the data that the pro organizations have… It gives us just more context on why a pitch may move a certain way, or, how to optimize seam orientation for certain movement profiles.
“Pitch physics isn’t completely solved. There’s a lot of stuff, non-Magnus wise, like seam-shifted wake, and possibly other variables that we may not even know about, that affect ball flight,” Liebhardt said.
For instance, Liebhardt notes we know how seam-shifted wake affects ball flight but we cannot quantify how much it affects movement alongside other variables, some he notes that “we may not even currently consider.”
We don’t know everything. And what’s so exciting is computer vision will lead to more understanding.
“These tools are super helpful for utilizing the information that we already have,” he said of CV, “as well as collecting more information to be able to learn more about pitch physics.”
What’s also exciting about real-world AI breakthroughs is they keep learning, they keep getting better.
“The cool part for me is being able to have an easier way to look at seam orientation and spin axis,” Liebhardt said. “That’s just something that, historically, you’d have (study an) Edgertronic camera and try and find it and guess where the spin axis would be.”
Now, Liebhardt has a tool that cuts out more of the guessing.
He shared this clip of another Imai-like mystery pitch, this one from Driveline athlete Tony Oreb.