Dec 6, 2024
SIGGRAPH Asia 2024 Keynote: Co-Creating Worlds with Game Designers, Animators, and Filmmakers
Sony Group CTO Hiroaki Kitano took the stage at SIGGRAPH Asia 2024 to address attendees. Three creators from across Sony joined Kitano in Tokyo for the event, sharing with SIGGRAPH attendees how game designers, animators, and filmmakers are embracing technology to build intricate worlds with artists at the center of the creative process.
The presenters shared how technical breakthroughs are opening up new possibilities for storytelling, from PlayStation’s 30-year technical evolution to the convergence of cutting-edge production techniques in film and animation at Sony Pictures Entertainment.
CREATING TRANSPORTIVE WORLDS
Kitano opened the presentation with a reflection on the key elements that a creator must conjure to create a new world from the ground up: dynamic characters, vivid environments, a compelling story, and lastly, a meaningful way for the audience to engage or interact with the world.
As a creative entertainment company, Sony has a long history of helping people experience new worlds through its iconic electronics devices and technology. As a partner to the creator community, Sony delivers the tools that artists need to bring their wildest, most imaginative ideas to life. Kitano emphasized that ultimately, this is why Sony exists. We are here for creators. We are here to help creators build and transport people into new worlds.
Kitano’s fascination with world-creation began early in his career as a researcher in the 1980s developing natural language processing systems at Carnegie Mellon University. He was motivated by the potential for natural language processing to support large-scale real-time translations. Early in his career, he developed a text-based adventure game to explore how natural language processing could become a tool for storytelling.
While the game never made it to market, it served as the inspiration for one of his first projects at Sony: a 3D society for the internet called the “Virtual Society Project.” In this project, Kitano and his collaborators created a language for in-browser 3D graphics to help people build worlds and communicate online. It was a rewarding challenge, equal parts technical and creative.
PLAYSTATION: THREE DECADES OF TECHNOLOGY PROPELLING ART
Kitano shared that the history of PlayStation perfectly illustrates Sony’s spirit of collaboration between technologists and creators. He took a moment to acknowledge the PlayStation brand’s recent 30th Anniversary milestone and the impact that PlayStation has had on the gaming world.
From its 1994 launch to today, the PlayStation console has been a vehicle for developers to create vivid worlds on massive scales. The “Emotion Engine” CPU introduced in P2D helped developers more accurately simulate reality in 3D worlds. This worked together with the “Sony Graphics Synthesizer” GPU to deliver enhanced polygon rendering. PS3 brought developers the CELL processor and the RSX “Reality Synthesizer,” which enabled more cinematic types of storytelling in games. This included more advanced lighting models, pixel shaders, dynamic real-time shadowing, and advanced water simulation.
Kitano shared how PS4 gave developers an even richer palette for world-building with 4K resolution and HDR graphics. These advancements are particularly evident in character design. Polygon evolution brought iconic characters more realism and depth. The breakthroughs continued with real-time ray tracing introduced in PS5, giving players more vibrant environments to zoom around in. The debut of haptic feedback and 3D audio also gave developers powerful new dimensions to transport people, grounding players in the game and giving them the ability to hear footsteps or a nearby waterfall or crackling fire coming from different locations – just like in real life.
GUEST CREATORS FROM ACROSS SONY
Kitano welcomed technical trailblazers from across Sony to share their perspective on how the creative process evolves in tandem with technology.
A MESSAGE FROM THE CREATOR OF GRAN TURISMO
Kazunori Yamauchi and his team at Polyphony Digital have been developing for the PlayStation console since the beginning with their beloved GRAN TURISMO series. Kitano called Yamauchi to share his thoughts on 30 years of technological advancements on PlayStation.
Yamauchi reflected on Polyphony Digital’s original vision to replicate the real world using computers. His team has explored this by simulating the look and feel of the cars and racetracks in GRAN TURISMO, and the beautiful scenery surrounding the tracks. He shared how the expansion of computing power has enhanced the sense of realism in the game over the years and looked toward the future of Polyphony Digital. He shared how his team is exploring design-based modeling and simulation and data-driven neural networks to continue evolving GRAN TURISMO.
A LITTLE BIT OF TECHNO MAGIC: CREATING THE WORLD OF ASTRO BOT
Fresh off the launch of a new title - ASTRO BOT - Team Asobi Studio Director Nicolas Doucet joined Kitano to share how his team built the joyful and colorful world of the game. Based in Tokyo, Team Asobi’s unifying objective is to turn technology into magic, or “Techno Magic” as it’s referred to by the studio.
ASTRO BOT tells the story of Astro, a cute and fun robot that flies aboard the PS5 mothership. To make the game feel like a toy, Team Asobi built a highly reactive, physics-driven layer of play into Astro’s world, creating delightful interactions that are not tied to any rules or serve any purpose other than satisfaction and fun. Technology brings this infectious spirit of play to life.
Team Asobi also used the DualSense wireless controller vibration system to add powerful haptic expressions to the game. This technology unlocks a new dimension of the game and helps the player feel grounded even though Astro’s colorful, fantastical world is very different from our own. Team Asobi’s keen attention to these experiential details can make any player feel like a kid again.
VFX CHARACTER STUDIES FROM GHOSTBUSTERS: FROZEN EMPIRE
Imageworks DFX Supervisor Ben Aguillon took the stage to share how the VFX industry has transformed alongside emerging technologies. He explained how Sony Pictures Imageworks works collaboratively with filmmakers to solve creative challenges. Open-source tools are essential to the Imageworks production process, helping to bring many worlds to life, from HOTEL TRANSYLVANIA to THE AMAZING SIPDER-MAN, to MEN IN BLACK: INTERNATIONAL.
Aguillon went behind the scenes of VFX-driven visual development in GHOSTBUSTERS: FROZEN EMPIRE. For this project, Imageworks set out to bring the look of the classic 80s visual effects together with modern technology, resulting in a world that feels both nostalgic and fresh. The team took inspiration from the proton streams in the original film. During the final battle they introduced a new tri-beam proton stream, layered with caustic effects, liquid emissions, energy auras, and color shifts that synced with on-set lighting.
The elemental effects in the film brought the world to life - bending fire, shattering frozen characters, scattering thousands of icicles and layers of frost. The ominous cloud that symbolized the arrival of the film’s main villain – Garraka – was inspired by the water tank FX from the original film.
As Imageworks continues exploring worlds for their next projects, they look to continue advancing the state-of-the-art through collaborations with creators across the industry.
CREATING A PLAYGROUND FOR FILMMAKERS AT SONY PICTURES ENTERTAINMENT
Torchlight Pipeline Supervisor and FX Technical Director Rey Jarrell took the audience behind the scenes of Torchlight, an advanced visualization studio located at Sony Pictures in Culver City, California. Torchlight’s mission is to bring Sony’s cutting-edge technology directly to the fingertips of filmmakers and artists, helping them transform the incredible worlds they envision into cinematic reality. Filmmakers come to Torchlight during the pre-production process to experiment with a suite of custom virtual production tools, including tech from emerging R&D projects across Sony.
The Torchlight team can build out the entire 3D world of a film in Unreal Engine. Filmmakers can explore these world using a stage full of virtual camera rigs built to feel as close to a live-action set as possible. They can plan shots, conduct virtual location scouts, and develop the visual language of their film.
At the heart of Torchlight is the idea of “One Sony.” Torchlight collaborates with amazing teams across Sony to bring creatives’ ideas to life faster than ever. Ultimately, Torchlight strives to support creators with Sony’s groundbreaking technology, helping them reclaim time and focus on turning filmmakers’ dreams into cinematic reality.
THE FUTURE OF CREATION
Kitano returned to the stage to offer closing remarks on Sony’s vision for the future of creation. As tools like game engines converge across different genres of content creation, storytellers of many different mediums can share a common technical language.
With advancements in AI and deep learning, Kitano shared, it’s clear that we have reached an important inflection point in creative entertainment technology. During these big periods of transformation, collaboration between artists and technologists is essential. Kitano shared, “We are honored to be among peers who continually expand the boundaries of what’s possible for creators to achieve. We look forward to partnering with many of you to co-create worlds that continue to inspire and entertain people around the world.”
