Any Surface

New design concept
for IoT & Agent

A single point:
The origin of Sony’s
new-generation user interface

The relationships between humans and devices are changing. The conventional interface model, one where people would simply manipulate a standalone product directly, is giving way to an approach that unites actual users with AI-driven agents in the use of ubiquitous Internet-of-Things (IoT) devices. As these connections continue to evolve,
our research on user experiences (UX) and user interfaces (UI) are opening up new horizons.

A natural interface for optimal harmony

Given all the shifts reshaping our social realities and day-to-day lifestyles, we’ll soon be inhabiting a world where the IoT—a networked web of smartphone-controlled electronics ranging from audio devices and projectors to lighting, door locks, and other home appliances—will form our spheres of daily existence. Developments in artificial-intelligence (AI) technology, meanwhile, are quickly making devices more responsive and convenient than ever: Instead of relying on manual user input, devices are now able to perform a multitude of tasks automatically at the sound of a voice command.
Most home electronics obviously need to be within a certain size range, have access to power, and accommodate the other demands of regular domestic use. What it comes down to, then, is how products can blend simplicity, communication, and user-friendliness. As those needs started to come into view, Sony knew that it had to push its UI to the next level.
The goal was to create a UI that would let users focus exclusively on what they wanted to do and know without having to give any thought to the devices or apps they’re using. The ideal UI would play a non-intrusive role, only showing up when there was a clear need; it would do its job and then get quickly out of the way. To develop an “Any Surface” UI language capable of rendering a viable interface out of whatever the user was looking at, we set four core goals.

  • -Keep high visibility in environments covering various ranges of information
  • -Make the UI scalable to IoT devices that operate under capacity and power-consumption restrictions
  • -Establish a uniform process for ensuring natural conversation-based agent operations
  • -Enhance usability through smooth, logical screen transitions

The process of translating that vision into an actual UI design was a complicated effort, though. We needed to make sure that the focus on simplicity didn’t result in a cold, dull feel—the quality of the interface depended on interactivity that would give the design a kind of warmth, nestling the interface into the user space as harmoniously as possible.

Breathing new life—and new meaning—into basic elements

“A musician aims to play sounds that transcend silence, while a painter strives to paint beauty that transcends a white canvas.” Transcending the state of nothingness—that’s what art is. In the creative process, the moment that defines a work often comes at the initial encounter between the artist and the medium: The first measure of a composition or the first stroke of a painting often determines whether the work will succeed. At Sony, hardware design operates on a similar basis—the first basic line sets the course for the entire design. Searching for the best way to make that first, crucial mark, the UI design team eventually made a breakthrough that would give real shape to the Any Surface concept.
We decided to use the most basic, fundamental element of all: the single point.
A point, having no height or width, represents zero-dimensional space. In our day-to-day lives, points often signify locations, serve as markers, or indicate origins. Points have a deeper power, too: Like the first drop of rain hitting the ground or the center of a ripple spreading across the surface of a still pond, points also have a more symbolic, suggestive connotation hinting at something more to come.

We decided to build the Any Surface interactions around the idea of a “point”—a dot—appearing in empty space, a concept that would reflect the intelligence of the UI’s dialogical capabilities.
When the dot in the Any Surface UI moves, the interface gains the visual components of lines and motion. The lines define the width of the display area, preparing the user for oncoming information, and then immediately disappear to foreground the target information. The motion of the point, meanwhile, not only lends the interface a dynamic dimension but also shapes the Any Surface’s distinctive personality. To give that visual character an organic, welcoming identity, the team meticulously designed the motion to exude visual warmth and evoke a sense of comfort.
For the final stage of the information-display process in the Any Surface UI, we decided to use a “surface” concept. The rectangular surface visualization chimes with the gentle, intelligent personality of the entire UI, as well, featuring rounded corners during the transition to highlight the information within its borders and imply convergence.

Hardware design starts with a single line. We figured that we could build an interface around that same idea, using a single point as the origin of everything else. As we explored that concept, the new user interface idea started to come into view.

Iriya, Chief Art Director

Making user-agent interactions effortlessly natural

If two people are just standing together and talking at the same time, they’re not having a dialogue; there’s no back and forth, no real communication. That basic understanding—the need for both talking and listening—was vital in shaping the Any Surface framework, which includes numerous features that help the agent listen to the user and encourage the user to wait for responses.
First of all, the interface uses visual cues to indicate what the agent is doing. When the dot is on a small, stationary circle, the user knows that the agent is in standby mode and waiting for voice input. When the display shows larger, pulsating circles around the dot, the interface is telling the user to wait while the agent is thinking.
Some devices also feature illuminations and tactile feedback. To integrate those elements into the interface, the design team worked to make sure that the timing and intensity of any additional signals would seem as real and natural as possible during user-agent interactions.

Multilayered interaction:
Visual, aural, and tactile dialogue

The market is actually home to many products that do searches based on AI-based voice commands—but their interactive styles are still a long way away from being as natural as two people having a conversation. A lot of that awkwardness comes from how the AI handles situations where it misunderstands something or can’t make out what the user is saying. In those cases, the products tend to switch immediately into overly mechanical, impersonal responses or rely so much on actual audio input that they fall into annoyingly redundant patterns.
Unnatural interactions were the last thing we wanted. To make the interface as smooth and intuitive as possible, we decided to bring audio together with text and graphics for an optimal, communicative blend of interactive elements.
It’s no rare occurrence to mishear or misunderstand the person you’re talking to. The same goes for the Any Surface interface; there’s no guaranteed way to get around it. When a human-to-human dialogue runs into that kind of problem, the speakers often turn to text and pictures to streamline the communication experience. The Any Surface follows that same, natural pattern. The UI also presents choices to users in a text-based format and opts for graphics—maps, graphs, gauges, and more—when a visual representation would be more effective than voice-based communication. All together, the dialogue system is a true Sony original with unprecedented usability.
Another feature is the “modeless” process flow, which enables users to combine voice commands with tactile operations at any time and even delegate remaining operations to the agent if the need arises. Giving the user as much freedom as possible, the interface embraces a thoroughly user-centric design.
For adults who’ve grown accustomed to the traditional app style, the Any Surface might take a little getting used to. Children, however, don’t have any preconceptions about how interfaces are supposed to work. For them, the UI will be a natural fit—and future generations will find it hard to believe that there was ever something other than the approach that Any Surface epitomizes.

The existing apps and multi-window interfaces are great for displays with size limitations, but they don’t work very well with spatial scaling. Any Surface doesn’t operate under that conventional UI vocabulary of points and lines. I think it has the power to transform popular conceptions of what ‘apps’ are all about.

Imamura, Senior Manager, UX Platform Design Group

Flexible support
for virtually any situation

The simplicity and flexibility of the Any Surface experience makes for a versatile UI language capable of flourishing in any application and any setting: the devices people use, the spaces people inhabit, and even next-generation products that currently lie beyond our imaginations.
We’re getting closer and closer to a new technological landscape, a world where whatever we need or want is just a simple flick of the finger or voice command away. Any Surface is getting a running start on making that vision a reality.

As we tweak the interface in accordance with changes in product identities and applications, Any Surface will gradually roll out as the new UI layer for a wide variety of Sony products.
The entire Any Surface experience converges on a single point. The future of UI design lies right there in that elegantly simple, zero-dimensional concept.