Interactive Content Solution

Designing a compelling moment

Our everyday life is filled with messages.
Billboards, posters, TV programs, newspapers, e-mails, social media, the list goes on.
How can we make an impression that stands out and remains in our memory?
Sony Design Center Europe is working on Interactive Content Solution,
a system consisting of Sony's display technology, sensing technology and content creation.
Interactivity, through sensing and content integration, makes the content more compelling,
creating a moment for people to stop and engage.
We sat down with the project members for the project to discuss the background and its future.

Moé Takemura
Creative Producer
Design Center Europe

Johan Hansson
Project Lead
Technology Promotion Department

Karl-Johan Dahlström
Technology Promotion Department

Stefan Andersson
Site Manager
Sony Lund Office, Sweden

Space revitalization
with interactive contents

The project started as a part of office space revitalization activity in Sweden. How did this start and what were you trying to achieve?

MoéWe were helping the facility management team for office space renewal and had a challenge to revitalize a plain grey 12-meter-long hallway space. At that time, we were also helping a team of system developers creating mini games for Sony Pictures movie promotion activities, utilizing projectors and sensors. The experience is interactive and immersive and has always been highly appreciated by the marketing team as well as audiences. We wondered if we could install a similar setup in the hallway space and make one of the walls interactive. The space could get a new life and we could also use it to test and further develop the technology as well as the user experience. We made a proposal to the site management team.

StefanThe proposal from Moé aligned with our vision. In Sony Sweden office, we have several teams developing smart solutions for B2B, and are utilizing our office as a testbed. As the hallway is by the entrance, it was going to be a face of a modern and smart workplace. We wanted the space to inspire us employees and our guests.

Movie promotion for SPIDER-MAN: INTO THE SPIDER-VERSE, 2018

Movie promotion for MEN IN BLACK: INTERNATIONAL, 2019

(Left) : Movie promotion for SPIDER-MAN: INTO THE SPIDER-VERSE, 2018,
(Right): Movie promotion for MEN IN BLACK: INTERNATIONAL, 2019

Flexible and easy framework for designers

What did you develop and how did the system materialize?

MoéOur challenge was how to create a flexible and easy-to-update platform. The entertainment experiences we create for Sony Pictures movie promotion is labor intensive. Every content is custom programmed by system developers. We couldn’ t afford to have a similar setup for the office space. Unlike movie promotion content, the audience see the content every day, and it gets old quickly. It was crucial to be able to update it in an efficient and cost-effective manner. This led us to building a system framework, which enables designers to create and update content without involving system developers. We sketched out several use-case scenarios, categorized them per interaction pattern and assessed the functional requirements of the system framework.

JohanIn the concept sketches, we could basically see two kinds of content for the hallway. One is to communicate a message, or information. Interactive digital signage as infotainment. The other one is something fun and entertaining for the audience. Considering that it is a corporate space, and there is a frequent need for PR and internal communication, we thought that a framework for interactive digital signage / infotainment would add great value as a start. We built a system framework using touch sensors so that people can touch the wall and explore the content.

Continuous framework development
with latest sensing and display technology

How did the hallway space get perceived, and what is the next step?

MoéThe hallway got a lot of attention internally and externally. It was very cool to see how simple interaction could catch people' s attention and make them stop. Soon our colleagues, working in new business development and with external collaborations, were demonstrating it for external partners to "show off" the space and utilize it as a conversation starter for business discussions.

We are constantly learning what kind of content and interaction resonates with people. Johan and I are often in the hallway testing content, and many people stop by, share their feedback and ideas.
Touch interaction is adding value, but it is not always enough. We need effortless interaction as well. The next step is to start adding motion interaction – you walk pass the space, and something happens so you don’t have to always touch the wall.

Karl-JohanThere are various ways of implementing motion interaction. It's a combination of sensing technology and machine learning from Sony R&D department in Tokyo and in Sweden, and we are continuously testing new sets of technologies. With the latest development, we can detect not only where you are but also how you are moving with accuracy. We can also follow multiple people's movement at the same time. The current focus is to filter the sensor input and investigate how to use the sensor input for interesting and meaningful user experiences.

We are also working on a solution for different display technologies. The hallway installation is for a long flat wall using Ultra short throw projectors, but the interactive content installation can be for curved walls, for floors, or for a set of walls, it works with any surface. The surface doesn’t need to be big either, it also works for smaller surfaces. In addition, if you want more crisp and vivid images, we can use display screens instead of projectors. We are now developing solutions that aim to do real time interactive experiences that can cover much larger screens and larger areas than what we have seen so far. Really looking forward to showing these capabilities in a near future.

Lunch time at the hallway space with lots of casual meetings

Work-in-progress joint detection development

(Left): Lunch time at the hallway space with lots of casual meetings
(Right) Work-in-progress joint detection development

Flexible display type from Ultra short throw projector
to Crystal LED Micro-LED display

Flexible interaction from touch, motion to gesture

Flexible format

(Left): Flexible display type from Ultra short throw projector to Crystal LED Micro-LED display
(Middle) Flexible interaction from touch, motion to gesture
(Right) Flexible format

Exploration of
various use case scenarios

Where are you planning to take this solution in the future?

MoéThe project started as an office revitalization, we saw the potential of turning this into a creative tool for many scenarios in the early stage, and started a dialogue with Sony's sales and marketing teams responsible for projectors and display products. Now, we are applying our creative tool and developing different use case scenarios for both internal and external partners. I have been leading the content creation for our corporate space and enjoyed pushing the boundaries and discovering the capabilities. We are looking to get input from other content owners as well as the audience, so we are closer to customers and people, reflecting the real world needs and feedback in our development process. I would love to learn what other kinds of creative ideas other potential users have.

Bringing Sony Pictures movies
to life with interactive content

We are using the interactive content created to promote upcoming Sony Pictures, and it has greatly enhanced the in-theatre consumer experience. We've had the technologies at business conferences where we're able to showcase the technology and entertainment to our most important exhibitor partners. The highlight for us is the consumer facing activations in movie theatres where they can come into their community movie theatre lobby and step into another world. Our customers get the experience of running through the jungle with The Rock, Kevin Hart and Jack Black in JUMANJI: THE NEXT LEVEL, shooting aliens in MEN IN BLACK: INTERNATIONAL, or have a comic book come to life in SPIDER-MAN: INTO THE SPIDER-VERSE. Everyone who has experienced these programs has been transported into our films and enjoyed the games and immediately connected to these properties.

(Left): Ann-Elizabeth Crotty — EVP, Global Exhibitor Partnerships & Global Marketing Operations,
Sony Pictures Entertainment

(Right): Vinele Grana — Vice-President,
Worldwide Exhibitor Partnerships, Sony Pictures Entertainment