Google Cloud - AI Basketball

Together with the Google Demos & Experiments team, we created a highly sophisticated (and very glossy) AI-powered shooting environment for aspiring basketball players. From concepting to art direction to creative technology, our teams sprinted to architect and execute an experiential showcase that precisely mimicked the act of ‘shootaround’ practices that real NBA players do daily. 

The experience netted nearly 1,200 signups over the course of two and a half days, and was able to analyze roughly 10,000 shots by Next ‘25 attendees, including an unexpected cameo from former Golden State Warrior and Atlanta Hawk, Zaza Pachulia!

The key was built to scale by George P. Johnson’s team, inside of the Mandalay Bay Convention Center, and wrapped in real-time screens with sign-up, tech explainers, leaderboards, queues and form analysis screens. A truly gigantic undertaking that—despite an average throughput of under two minutes per shooter—managed to accrue a 90 minute wait list at one point. File that under ‘good problems to have,’ though we sprinted on-site to shorten the wait time and increase throughput anyway.

Beginning five months ahead of Cloud Next ‘25, we started refining the concept and experience with rounds of sketching and early art direction, factoring in everything from metrics and stats that we could capture with skeletal tracking to the ideal spatial dimensions and throughput of the experience. 

On a convention center floor containing 36k people, how could we ensure a rich, remarkable experience while moving shooters through the flow expediently? And how would the tech marry with the interfaces and UX to really celebrate the street hoops mentality? Digital was our purview, but it went beyond digital, too. Wayfinding, physical user flow, and dimensionality were all crafted in collaboration with the Google and GPJ teams.

As our developers worked to determine a fitting tech stack, we collectively began to imagine how experience and tech would meld. Multiple Pixel cameras tracked shooter form via video, and another two Pixel cameras provided a live-stream to spectators on the outside of the court, all of which streamed live through Google Cloud Storage. Google’s engineering team implemented the core human analytics, which we integrated into our work.

Shooters were face to face with a 20 foot tall LED screen positioned behind the hoop, which provided instructions and live reactions as their shots flew. They were encouraged to shoot as much as possible inside of the 30 second window, while Gemini, MediaPipe and AutoML on Vertex AI tracked detailed metrics such as elbow position, knee-toe alignment, make/miss rates, entry angle and more.

AI Basketball Coach was trained on pro coaching tips, which Gemini ingested and then interpreted to help improve shooters’ quality in real time. Firestone made it possible to analyze pose and shot data, and Gemini Live used this data to provide dynamic, contextual real-time coaching tips, spoken aloud to shooters via Gemini Native Audio. Just like having a real NBA coach on-hand for your shootaround.

While shooters and spectators gathered outside of the court, a gigantic wraparound ticker streamed the queue, leaderboard and branded graphics. Massive attract loop screens provided storytelling around the experience, and a learn and signup section let attendees drill deep into the experience, including their shots, others’ shots, and the tech powering the court. In addition to all of these screens, we also designed a 3D model that provided animated diagrams of shooters’ form and metrics. 

Customized post-experienced takeaways were provided to each shooter, and available for safe keeping. The takeaways contained video of their session, the pointers they were provided, and their overall metrics for easy social sharing and for posterity.

In addition to Zaza Pachulia, 635 others shot their shot with AI Basketball Coach, amounting to roughly 10,000 shots analyzed and measured in real-time. 

We designed, architected, configured and animated 12 screens, varying in size from ‘big-screen TV’ to ‘jumbotron’, all inside and around a 20x10 foot basketball key.