Immersive Space Entertainment

Music visualizer & Cyber gym

How designers are shaping
the technologies for
"future experiences"

Sony's "Immersive Space Entertainment" concept ushers users into VR environments
without the need for any head- or body-mounted devices, transforming experiences of absorbing worlds.
We recently sat down with one of the Immersive Space Entertainment producers,
who explained how Sony takes designers' ideas for "future experiences" and "future media," channels those visions back into the development process, and then sets out to create the technologies that those innovative,
design-driven concepts call for.

The designs create the objectives

Tell us about how Immersive Space Entertainment took shape.

Nashida (producer) : I spend most of my time working on interface design. Basically, interface design involves figuring out what kinds of graphic user interfaces (GUI) would make it easier for people to use products with clear, pre-defined objectives and applications—BRAVIA™ products, for example. For the Immersive Space Entertainment project, though, we changed things up: Instead of creating designs for a specific objective, our job as designers was to create the actual objectives themselves.

With how powerful computer graphics are getting, we're at the point where we can use the available technology to make responsive, high-quality video—and that progress is culminating in the world of VR. It's a compelling scene, but the technology could still go further. When you look at the VR devices on the market, for example, the standard format is a head-mounted display (HMD). We wanted to explore new possibilities and create a different way for people to immerse themselves in VR and AR environments. That's where the Immersive Space Environment concept started coming together. As we got to work, we focused on one of the basic approaches to immersion—wrapping the user's entire field of view in a different "reality"—and came up with the idea of using a projector to cast images on a hemispherical dome around the user.

We wanted the visuals to dazzle, of course—Sony has obviously built a reputation for astounding video quality. The challenge was finding the right projector setup. A single-projector configuration wouldn't be able to cover the full pixel range. Using two 4K projectors would deliver high-definition performance, of course, but putting the units close together would create diffuse reflection. After exploring some different approaches, we worked out the optimal solution by piecing together several existing technologies: a reflection-reducing coating for the dome, a wide-angle conversion lens to expand the visual scope, and stitching technology to bring everything together seamlessly.

Three offerings, three target experiences

At SXSW 2017, Sony featured three programs for the Immersive Space Entertainment concept: 360 Movie, which gives users intuitive control over 360-degree video, Music Visualizer, which renders sound in a visual format, and Cyber Gym, which offers a new take on what entertainment can be. What were you aiming for with that contentand how did people respond to the experiences?

Nashida : When we started developing the first one, 360 Movie, the initial idea was to have the system detect the orientation of the user's face and transform the visuals accordingly. That ended up being too sensitive, we found. Linking the visuals with chair movements, though, made for a much smoother experience and gave the experience a more natural, immersive quality. Without prototyping different ideas first, we wouldn't have been able to come up with that kind of "sensory design." That's a big part of our development process. It's an offering that brings the awe-inspiring experience of enveloping, high-resolution immersion—the IMAX theater approach—into a small, intimate space. It really gave me a thrilling taste of what future media could be.

Music Visualizer, meanwhile, generates high-precision, real-time visualizations of audio input. It redefines the music experience, giving you a new, refreshing way of engaging with sound content. You don't just listen to music with your ears any more—with Music Visualizer, you listen to it with your eyes.

The third package we showcased was Cyber Gym, an interactive space that integrates a stationary bike with moving visual projections on a dome-shaped screen. As the user rides the bike, the system tracks the user's leaning motion, pedaling speed, and other movements and then uses that input to modify the video surrounding the rider. The technology was the most popular part of our showcase—some people said they couldn't wait to have their own Cyber Gym systems. The setup occupies quite a bit of space, so our first target segments will probably be gyms and other larger-scale facilities. Having seen Cyber Gym in action at SXSW, I think it would be cool to see the technology grow into a platform for entertainment-oriented exercise.

Immersive Space Entertainment has the potential to be more than just a big-screen TV—the products could revolutionize the concepts of space and time in virtual experience media, I think. To make that transformation as powerful and far-reaching as possible, we're going to focus on making the devices more compatible with home settings and giving users easier access to what the technology can do. We've still got a long way to go, though.

Photo of the equipments of the Immersive Space Entertainment Photo of a visitor enjoying the 360 Movie

The benefits of publicizing prototypes

The Immersive Space Entertainment made its debut at SXSW 2017. Isn't it unusual for Sony to make pipeline technologies public at the prototype stage?

Nashida : Yes, it is. Unveiling technologies that are still in development can be a risky venture, of course. When you're going after new designs, though, the process of starting with the design, using design ideas to locate the vital technologies for future experiences, feeding those findings back to engineering teams, and then developing the requisite elements makes a lot of sense.

Rapid prototyping is all about making—and opening doors to—things that people can experience. At SXSW, that's exactly what we did with Immersive Space Entertainment. Bringing people together with the technologies, even though they're still works in progress, gave us great insight into how different users engage with the experiences. When you're designing interactions, subjective tastes can have a big impact on whether users find their experiences fun, cool, or enjoyable. That's why the kind of direct input that we got at SXSW is so valuable.

Sony's media reach is getting broader, expanding from our traditional strengths in music and video media to the new sector of VR and other "experience media." When you peek into that world, you can see the thrilling possibilities: the full-sensory fascination of visual media coming together with elements like haptic technologies that augment tactile interaction, smells that fill the air, and wind sensations that add another atmospheric dimension to the experience.

During the testing phase, we played around with the idea of projecting a life-size person on the hemispherical dome surface. The results were astonishing—if you stood facing the screen, it looked like a real person was standing right there in front of you. If you made eye contact, it was like you were making a personal connection. Those eye-opening, emotional awakenings, driven by cutting-edge technology, are going to keep making Immersive Space Entertainment part of a paradigm shift: the creation of new media. To do that, we'll work to make the new design process—making, experiencing, and iterating that cycle—an even bigger part of our day-to-day efforts.

(L to R) Mikami (engineering designer), Nashida (producer), Yamazaki (engineer), Takahashi (engineer)