How I got to play tennis with (a virtual reality) Maria Sharapova

Have you ever wondered what it would be like to play tennis with international star Maria Sharapova, a five-time Grand Slam title holder? Well, if you did, you probably did not expect your U.S. Open tennis superstar dreams to be realized while standing in the center of a beige-colored room, with a black stick in hand, and a virtual reality headset covering your face. Where are the fans cheering you on as you (attempt to) volley back Sharapova’s 100 mile-per-hour serve, you may ask? Well, instead of the throngs that usually gather at the USTA Billie Jean King National Tennis Center in New York, there were a few technicians and public relations people watching you while standing in a semi-circle around the perimeter of the room.

That at least was my experience playing Sharapova, as I tested out “You Vs. Sharapova,” an American Express-sponsored fan experience that will give attendees at this year’s U.S. Open tennis tournament the chance to play against a photo-realistic virtual reality version of one of the sport’s biggest names.

During the tournament, which runs Aug. 31 through Sept. 13, fans will step into one of three stations at the tennis center, where they will be able to wear a HTC Vive headsets - and play against a computer-generated Sharapova.

“We have a history of trying to push for new ways to immerse fans and enhance the fan experience, and for us this is the next generation of that approach, of making the experience come to life for fans,” Deb Curtis, American Express’s vice president of global partnerships and experimental marketing, told “The decision to go with a virtual reality experience was really inspired by an insight about the U.S. Open itself. There’s a uniqueness of this event living in the heart of New York City – there’s a competitive intensity of the game and the energy of New York combined. You bring that together and you give fans the chance to experience that intensity, to return the volley of a player like Sharapova.”

Well, this sports-challenged journalist did not return any volleys, but the experience was, quite simply, a lot of fun. The HTC Vive headset uses Steam VR virtual reality technology. Once the headset was placed over my eyes, after a brief delay, I was shocked to see the beige walls before me vanish, to be replaced by a beautiful sunshine-filled day at the tennis center. That black stick placed in my hand became a racket, and after a quick intro from a friendly, computer-generated guide, I was greeted by Maria – yes, we are on a first-name basis now.

Over the course of four minutes, I attempted to volley back some of the tennis player’s serves, but only made contact with the ball a few times. While I won’t be triumphing on the international tennis stage anytime soon, it was easy for me to buy into the illusion that I was really there, and that I was inelegantly missing real tennis balls.

How was the realistic environment created? That’s where Dale Carmen, founder of Reel FX, comes in. He and his team were responsible for creating the immersive virtual reality world. For this project, they used the same techniques that Hollywood special effects wizards use for films like “Lord of the Rings.”

Sharapova's body and movements were recreated thanks to motion-capture technology. Carmen told that there were three components behind the computer-animating process. First, Sharapova was filmed in a studio against a green screen. Wearing her standard tennis outfit, she was there to offer a visual, lighting, and motion point-of-reference. Then, she had to wear what is essentially a spandex leotard outfitted with motion capture sensors. This was necessary in order to create a computer-generated “skeleton,” which looks like an actual human skeleton. The final piece of the puzzle required the tennis tsar to be scanned by a 3D laser scanner.

On a computer screen, a wire-frame mesh would cover the animated bones of the skeleton, moving and stretching in the way flesh does. Textures, lighting, and shadows are applied to the mesh, creating a realistic, animated copy of Sharapova.

“I want people to really feel transported, I want to achieve this sort of ‘holy grail’ idea that I have a virtual reality that can create a presence, the sense that you are somewhere— that you’re in an actual place,” said Carmen. “You know, people take it off and then you have to re-acclimate to the real world.”

Carmen said that the most positive feedback he received from American Express was astonishment at how real the world seemed.  From the seats in the stadium to the blowing leaves on nearby trees, Carmen said those “simple and subtle elements are a little surreal. It’s surprising that you can feel like you are being physically transported from a standard room and then suddenly you are outdoors.”

The entire process of perfecting the virtual reality world took several months — from capturing Sharapova’s movements to testing it out on willing guinea pigs, like yours truly.

(American Express)

So, what does Sharapova herself think of the experience?

Carmen said that she has yet to see the finished product.

“I’m hoping she’ll be able to stop by when she gets a chance during the Open,” he said. “And then she can try and play herself.”

I think she’ll be much better than I was.