Making of NYT’s Mariano Rivera’s pitches

The New York Times has been building interactives with audio-visual explanations for a while.

Last year we tried to take the techniques we were using a bit further with a piece on tennis great Roger Federer’s Footwork, which incorporated an animated 3D tennis player. (I don’t think I ever thanked Michael Agar for his very nice comments on that one.)

And earlier this year, we forged ahead with even more sophisticated interactives as part of our coverage of the Winter Olympics in Vancouver.

But none of them are quite like How Mariano Rivera Dominates Hitters, which accompanied a Times magazine profile of the Yankees closer last month.

promo

I asked the three graphics editors who put the piece together — Shan Carter, Graham Roberts and Joe Ward — explain how the piece came together and the decisions they made as they were building it.

The piece was remarkable, because as graphics director Steve Duenes puts it: “They managed to fully integrate a series of terrific data visualizations into a compelling, linear piece. It’s not something you see often because it’s difficult.”

Why Mariano Rivera?

Even if you know nothing about baseball you’ll get — and love — the graphic. I asked Joe Ward, the graphics editor for sports, what was so impressive about Rivera. (Although, it becomes obvious as you watch the graphic.)

Joe explains anyway: “What makes him most interesting is that he is one of the best relief pitchers in history. He has also been good for a very long time, and has relied on a single pitch, the cut fastball, or cutter. He has incredible accuracy with his pitches which was verified when we got a heat map of the locations of his pitches.”

“We contacted the guys at Complete Game Consulting who have been compiling that data. His cutter has significant lateral movement, which we were able to show with data from PitchF/X on the mlb.com site.”

A Processing app to understand the data

The data set contains XYZ coordinates for the nearly 1,300 pitches Rivera threw in 2009, which allows you to plot the curve of each and every pitch in three dimensions.

Here is a partial example of what the data looked like:

Picture 19

So, how can you make sense of all data? Shan Carter built an application in Processing that allowed him, Joe and Graham to navigate and investigate the data.

Yep, they built a graphic to know which graphic should they do. The visualization helped them decide which data to display and how they should show it. Here’s a screenshot:

app

“We built it to be able to explore the data and experiment with different interaction models,” Shan says. Although we all know that there’s a certain beauty in exploring the data by yourself, “in the end,” he continues, “we decided it would be more interesting to show users what was interesting rather than requiring them to find it themselves. We were trying to show what makes Rivera the best — allowing a user to browse all his pitches wasn’t the most effective way of doing that.”

The storyboard

Once the basic decisions on what to show were made, Graham put together a storyboard and Joe wrote the script based on that.

storyboard

I have talked long about the virtues of storyboarding. It helps organize your workflow, structure your story, try out the narrative … I repeat it in every workshop. And I have proven it to myself many times on my own graphics: the more complete and polished the storyboard, the easier and more coherent the storytelling will be in the end.

Advanced 3D for newspapers

It could be a wild guess, but I’d say you use a script to load 3D data from a database into Maya and work with the NYU’s Movement Lab only once in a blue moon.

To get all pitches into Maya, Shan wrote “a Python script that loaded the tab-delimited text file and programmatically created and animated the baseballs. Graham then lit, textured and rendered the Maya file.”

And it’s also the first graphic The Times has done using motion capture data instead of traditional character animation — I’d guess the first one any newspaper has done (comment below if I’m wrong and you know of other examples).

Last year, we visited NYU’s Movement Lab to learn a bit about the technology and ended up putting Graham Roberts in a mo-cap suit.

But the chances of getting Rivera in a mo-cap suit in a lab were pretty slim. So, the NYU researchers used another technique they’d been working on to reconstruct Rivera’s windup and delivery.

“We gave them videos of Mariano Rivera from several different angles — mostly slow motion videos,” Graham explains. “Those videos became then simulations of the different cameras they would use on a motion capture stage.”

“It is kind of working backwards. Normally a camera would shoot someone wearing a motion capture suit. In this case, they extrapolate a camera position from several videos, from different angles, to make up the stage, and bring each video into the same space.”

A short video from NYU’s Movement Lab shows a bit of the process used to build the animation.

A few of my favorite things

There’s many things I think are genius in this graphic, but two of the smartest choices I think Shan, Graham and Joe made were orienting the graphic around the batter’s perspective and freezing all his pitches in that massive ball cloud.

Joe explains the thinking behind those decisions: “It is tough to recognize the pitch in the time necessary for a batter to make his decisions. That is why we decided to freeze the pitches where we did, to show what the batters see. They usually recognize pitches by their spin. The spin on his cutter is nearly indistinguishable from his fastball.”

Simply brilliant.