The line between video game and reality continues to blur.
A year ago, soccer leagues leaned on Electronic Arts’ soundtracks to recreate fan noise in stadiums emptied by COVID-19. This time around, the action is headed to the metaverse. Nay, the Netaverse.
The Brooklyn Nets and YES Network unveiled the latest bit of broadcast tech during a game earlier this month, using 100 cameras to capture gameplay and digitally recreate the players (and the ball, the refs, etc.) in a virtual world. The result is something that looks very much like NBA 2K action, with the point of view moving in three dimensions behind James Harden and Kevin Durant.
YES producer Frank DiGraci can ask for a view as if the camera were feet behind Harden as he drives to the lane. He can set up a point of view from under the basket as Durant rises for a dunk.
“Actually, we can go underneath the floor,” explained Ken Ito, a senior director at Canon U.S.A.’s Innovation Center, during a recent Zoom call. “Anything that’s on the court, we can basically capture.”
Canon connected with Brooklyn through the NBA about a year ago, offering a setup it has been developing for years. An earlier system was deployed at the 2019 Rugby World Cup in Japan.
“My mind started spinning with how we could use it and show different things on our broadcast,” DiGraci said. “For regular highlights, for teaching highlights, for telestrating, for seeing things of all 10 [players] that regular cameras can’t capture, we look forward to using it in all these different areas.” The biggest thing DiGraci wanted from the tech was giving fans the sense of watching the game as if they were standing on the court.
Canon’s free viewpoint system will be on display at this year’s All-Star Game, used for specific replays and possibly during the dunk contest.
“The most exciting thing about this is that we’re actually bringing the fans onto the court,” NBA Entertainment EVP of media operations and technology Steve Hellmuth said. “For the first time, we can ‘place’ cameras in places where physically a camera can’t be.” He likened the capabilities to having six independent skycams constantly zooming around the action.
In the future, the data could fuel a live virtual feed, or even potentially a hologram version of the game. “We think it’s going to transform the way our fans experience the game of basketball,” said John Abbamondi, the CEO of Nets parent company BSE Global.
As for the Nets and YES’ return on a year-plus of work, Abbamondi said “there’s been a lot of interest already” from potential sponsors. “We will have a partner, or partners, in addition to Canon, of course,” he said, “but we’re going to be thoughtful about it.”
The unique view will also serve as an anchor for the Nets’ continued metaverse explorations. “That is absolutely one of the things that we’re looking at down the road, as far as VR and how this can really become an immersive experience,” BSE Global chief information officer Travis Sampson said. Shawn Bryant, a sports technologist helping Canon bring its product to market in the U.S., mentioned the content’s potential uses across broadcast, in-venue and social.
“To tell you what it’s going to look like, exactly, five to 10 years from now is difficult to know,” Abbamondi said. What he is confident of is that the virtual renderings “ultimately will be indistinguishable from traditional video except for your ability to manipulate it and view it in ways you could never do with traditional video.”
“You know the pressures that other RSNs are under,” Abbamondi said, “so the fact that YES is investing time, energy, money and resources into improving the broadcast at a time when others perhaps are not speaks volumes about them.”
(The story has been updated to include a comment from the NBA’s Hellmuth.)