Google Cloud - Quick Build Game Show

Quick Build Game Show was a well-hewn concept by the time we began working on it with the Google Demos & Experiments team. Essentially, the aim was to showcase Gemini's ability to make nuanced and meaningful evaluations in real-time based on context provided in prompts. So we decided to test its visual understanding and live commentary capabilities in a gamified way—with the ubiquitous building block. QBGS became the dynamic, playfully competitive experience that anchored the ‘Build’ space at Next ‘25.

In ‘dark-mode’ Google branding, the QBGS had space for four players or teams to compete in rapid-fire building block challenges, all under the watchful eye of Gemini. Pixel cameras captured every brick placement in real-time, streaming directly to Cloud Storage where Gemini analyzed the ongoing construction. Throughout the two and a half days of the event, over 260 building competitions took place, with Gemini providing live commentary via Text-to-Speech for more than 730 unique players.

Gemini was a skilled host, facilitating a seamless game show experience where players were able to react to instantaneous feedback, encouragement, and sometimes playful critique from their AI host. As players raced against the clock, Gemini's commentary evolved in response to their progress, identifying key design elements and creative approaches before ultimately crowning a winner and explaining the reasoning behind its decision.

The key challenge we were tasked with solving was: how do we make Gemini's visual understanding capabilities tangible and entertaining while showcasing its ability to provide contextual, engaging commentary that enhanced rather than distracted from the experience. 

QBGS had to successfully demonstrate the practical applications of AI in creative assessment and problem solving. By having Gemini analyze construction techniques, identify design elements and evaluate creativity in real-time, we showcased how AI could provide meaningful feedback in scenarios ranging from classroom learning to professional skills development.

Our experience across all of these showcases and customer stories really gave us the opportunity to sharpen our prompting and agent-building axes. Each was so different in terms of the role that generative, live, and multimodal AI played. In the case of QBGS, we really needed to nail the recognition, judgment and commentary in order to count the experience as a success.

With a front-end powered by AppEngine, Quick Build Game Show appeared from the outside to be a legit ‘game show’. But behind the scenes, the moving parts were expansive.

Pixel cameras were strategically positioned to capture brick building progress every few seconds. These images were immediately streamed to Cloud Storage, creating a real-time visual record of each player's construction process. This approach allowed us to capture the evolution of each build rather than just the final product, giving Gemini insight into the players' techniques and problem-solving approaches.

Judging a completed build is an entirely different thing than evaluating the strategic evolution of a build over time, then combining it with the overall quality of the finished product.

Once in Cloud Storage, the gameplay data followed two distinct paths. First, the ongoing build progress images were analyzed by Gemini through Vertex AI, using its visual understanding capacity to recognize building techniques, design elements and creative choices. Simultaneously, the game state progress was tracked to maintain awareness of timing and competition parameters.

Gemini's analysis then powered two key outputs. The commentary generation system crafted contextually incisive and motivational observations, identifying clever solutions, potential aesthetic direction, and construction challenges in real-time. These commentaries were transformed into speech through Text-to-Speech technology, creating the voice of the AI game show host that players heard throughout their experience.

As builds neared completion, the final brick creations were captured and sent through Media Pipe to a dedicated AI judging system. This system employed both build evaluation and a scoring system to assess the final constructions against the original challenge parameters. Gemini evaluated technical execution, creativity and adherence to the theme, ultimately selecting a winner and generating explanatory commentary that highlighted what set the winning build apart.

The prompting architecture behind this system was particularly cool. It made it possible for Gemini to speak and show commentary that was encouraging, insightful and, sometimes, humorous—maintaining the energetic atmosphere of a game show while providing genuine feedback on players' creations.

Quick Build Game Show captivated Cloud Next '25 attendees, drawing consistent crowds as participants engaged in friendly building battles under Gemini's watchful eye. In the end, the experience successfully demonstrated how AI could enhance rather than replace human creativity, serving as both guide and appreciative audience.

Over the course of the event, the installation hosted more than 260 building competitions for over 730 individual players. Particularly impressive was how organically players adapted to the idea of an AI host, responding to its commentary and adjusting their building strategies based on its observations. 

The real-time nature of the commentary proved especially compelling, with many attendees expressing surprise at how accurately Gemini could identify specific building techniques and design elements as they emerged. "I see a house with a red roof. It looks good! You need to add more colored bricks," Gemini might observe, prompting immediate adjustments from players eager to improve their scores.

We heard over and over that its fluency—and even its sense of humor—was uncanny. The host’s bent for ribbing contestants was so realistic that we actually had to dial back its tone on day one to keep it from roasting the builders as they worked!

The judging phase also demonstrated Gemini's ability to make nuanced assessments, with commentary like "You presented a colorful, charming and well-constructed cottage with delightful details" that explained its decisions in accessible ways.

Watching the tech stack work, you could easily imagine applications in almost any field, like arts and design education, training simulations, product R&D, or even landscape architecture. Essentially, QBGS was a proving ground for any field where real-time object or composition analysis could trigger big-picture feedback that’s tethered to a strategic goal.

Side benefit: kids are obviously going to lose it over the web version we’re releasing after Cloud Next.