Bringing new experience value to art photography

The Asama International Photo Festival 2019 PHOTO MIYOTA, a 56-day event that featured 728 works by photographers from around the world,
included featured a unique collaboration between the Sony Creative Center and photographer Mr. Takashi Suzuki:
an interactive exhibit titled "YOU and BAU." What gave the exhibit experience its identity?
It was a fusion of Sony's design capabilities and unique technologies that "immersed" viewers in Mr. Suzuki's work,
giving life to a new mode of experiencing photography.
Below, project members offer a behind-the-scenes look at the effort.

Location Value Design Office,
Creative Center,
Sony Corporation

Experience Design Group,
Creative Center,
Sony Corporation

Location Value Design Office,
Creative Center,
Sony Corporation

Location Value Design Office,
Creative Center,
Sony Corporation

Feeling out a gateway to
extraordinary experiences

The Sony Creative Center jumped at the chance to be a part of the PHOTO MIYOTA exhibit, a program that "attempts new possibilities for photography." What went into the concept behind the project?

IkedaApproaching the project, we wanted to leverage Sony designs and Sony technologies to bring new experience value into art photography. As soon as we got word that we'd be exhibiting at PHOTO MIYOTA, we started talking about taking the company's unique technology for detecting moving objects and applying it to photos. It's a technology that can automatically detect a person in a given image, "extract" the person out of the original, and then overlay the person onto a completely different image—all in real time. Sony has already incorporated the technology into the REA-C1000 Edge Analytics Appliance. So we got to thinking about how that might translate into a new kind of experience. Eventually, we arrived at a prototype idea: putting people into photography works, which we thought could be interesting. The PHOTO MIYOTA organizers thought so, too—they saw the same potential we did, and they said they wanted us to put it in the showcase. That's how the project got going.

Learn more about the REA-C1000 Edge Analytics Appliance here >

We didn't want to stop at just putting people into photos, though. We wanted to make the whole experience something new, something eye-opening, something memorable. Right from the start, we committed ourselves to three policies. First, we wanted to make sure that putting visitors into the photos would create a brand-new piece of art, not just a composite image. Second, the exhibit had to accentuate the aesthetic appeal of each photo by highlighting the contrast between the original work and the new version—the person-in-photo version—that the technology would create. Finally, the whole package had to be entertaining. What good is an interactive exhibit if you can't enjoy the experience, right? With our design goals in place, we started looking for an artist we could collaborate with.

OguraAs we sought out photos that would be a good fit for our concept, we looked at works by a lot of different artists. One that really grabbed us was a photo series called "BAU" by Mr. Takashi Suzuki. The core theme informing the works is the idea of the "recognition gap" that forms when people view an image, a phenomenon that Mr. Suzuki highlighted through combinations of colorful sponges in different forms. Depending on the viewer's perspective, a sponge creation might look like a building; it might look like a person's face; it might look like something else altogether. The subjects of the photos have a cool aesthetic identity, and the conceptual payoff is just as compelling. For us, it was the perfect canvas for the immersion concept—being in the photos would be like literally stepping into the recognition gap. We also knew it'd create a new work of art, given how the visitor in the photo could add to the look. When we approached Mr. Suzuki about using "BAU," he got on board right away. He liked how it was a brand-new take on art, something that'd go beyond the conventional boundaries of art photography. With that, we got to work on crafting the actual experience.

Shaping the experience of
artwork immersion

The team had lots of questions to explore. How could they make visitors really feel like they were "immersed" in the artwork? How could they make the experience an entertaining one? As the designers tackled those core elements, a more concrete vision started to emerge.

SuzukiWe started out basically feeling our way around the ideas—how to make the sensation of being in a photo as real as possible and give the experience the kind of entertainment value we were after. Step one was creating our "canvas," so to speak. We decided to project Mr. Suzuki's photos on the façade of the experience space and then use our synthesis technology to lay images of visitors over the photos in real time, making it seem like the people were physically inside the works. We also set things up so that the system would randomly assign different sizes to the people in the overlay—large, medium, and small—so that the visitors would have something more to react to from within the immersion. The idea was to encourage natural responses and actions, like people taking different poses as they saw themselves larger or smaller than life.

Another thing that came up in our discussions was the possibility of making it look like people were perched atop the sponges, which we thought would up the fun factor. Recognizing that potential, we decided to put a staircase in the experience space. Giving visitors more freedom inside the immersion experience was a big part of that idea, as people would have the ability to situate themselves at different heights. Another part of the staircase idea, however, was that it would make the experience more fun for groups. Without the stairs, people who came in groups would've had to occupy the same level—but with the staircase, they'd have more options for group poses and other creative approaches. The thing we focused on most in designing the user experience, though, was getting visitors to move around and have fun. When the experience is physical, it's more real; when you're doing it together with your friends, it's even more enjoyable. We knew that people would really get into it.

ShōjiThe PHOTO MIYOTA exhibit is a community-oriented event, so we also wanted our showcase to be accessible and fun for all ages. Initially, we thought about making the experience completely customizable by letting every visitor choose a photo to immerse themselves in or recreate shots themselves. But in the end, we went with a simpler approach—overlaying visitors on random images from the photo set and having them just strike a pose and push a button to snap their photos. Keeping the features to a minimum was a conscious decision. The fewer features there were, we figured, the more easily visitors would be able to engage with the "BAU" worldview and relish the immersion aspect at the heart of the experience.

In the moment, we knew the experience would be fun—but we knew that it'd have an even better impact if visitors had something to take with them. That's why we decided to give visitors the opportunity to get actual prints of their own "YOU and BAU" creations. In addition to rendering the experience as a physical, tangible work of art, the prints would also encourage visitors to compare their individualized images with the original "BAU"—and, as a result, reconnect with Mr. Suzuki's creative vision. We wanted to maximize that kind of dialogue between the visitor and the work. The original plan was to make the images available in electronic formats, given how sharing on social media works. Why didn't we go that route? Well, the whole idea behind a photo festival is to present moments in time as physical objects, recorded encounters between creators and subjects, things to cherish in their tangible forms. Prints were the best way to capture how precious and impermanent photos are.

OguraThe biggest challenge for us was the lighting. There wasn't any easy, catch-all approach to finding the right brightness for the experience space, considering the circumstances. The technology we were working with was originally designed for isolating individual instructors during lectures or corporate seminars—not the kind of variety we'd be dealing with at the exhibit. To make sure that the system could identify and isolate multiple people in a wide range of clothing, we had to get the lighting settings just right; the conditions had to make people's silhouettes easy to distinguish, and it took quite a while to get there. Still, there were a few patterns and colors of clothing that ended up looking transparent, becoming the same color of the background in the synthetic rendering, and we obviously felt like we needed to avoid that. It was no easy task working out all the little kinks.

But then something unexpected happened. During a test run, one of the trial visitors saw that her clothes had gone completely transparent in the overlay. We thought it was back to the drawing board again—but she was loving being semi-invisible; she struck different poses, like she was playing with the surprise of it all. I remember Mr. Suzuki saying that it was a "happy accident," something he found really entertaining. It was an eye-opening experience for me.

SuzukiWe'd been trying to perfect the technology, but that's not really what's important. The ultimate goal is making the experience as enjoyable as possible for the visitor. In that sense, the "YOU and BAU" project helped us return to a fundamental idea so central to design. By embracing the element of chance, we took the user experience to a more dynamic, enjoyable level.

Amplifying the value of
existing things

The "YOU and BAU" exhibit proved to be a bustling space, and visitors were buzzing about the new photo experience. How did the designers feel about the end results? Looking ahead, how are they hoping to channel their "YOU and BAU" experiences into other creative settings?

Ogura"YOU and BAU" drew a diverse audience from across the whole age spectrum, including art aficionados, photography enthusiasts, and local residents. There were plenty of people doing funny poses, and others were having fun comparing their personalized "YOU and BAU" prints with the original photos. I was so happy to see that. The most rewarding thing was what Mr. Suzuki, the photographer, told me: "It's so interesting to watch my work get extended and expanded like this," he said. I want to thank him for believing in our concept and letting us use his art. We wouldn't have been able to pull the exhibit off without his generous support.

IkedaDesign isn't just about creating new value from scratch. It also has the power to amplify the value permeating the things and places around us, the capacity to breathe new life into our surroundings. Taking our inspiration from Mr. Suzuki's "BAU," we pooled our design capabilities and technological resources together into a platform for finding new experience value in existing works of art. It was an ambitious project, and it helped us all get a fresh perspective on the boundless possibilities of design.

The Creative Center handles design projects for the Sony Group, of course, but we also want to make an impact outside the company. We know that all the design skills and ideas we've honed over the years can really benefit a wider audience. That's why we at the Location Value Design Office like to tackle projects for external clients, always hoping to create "place value" wherever we can. Art photography, for one, proved to be an exciting setting to work in. I'm looking forward to exploring new areas and taking on new challenges in as many different fields as I can.

By immersing people in photos by Mr. Takashi Suzuki, a uniquely creative mind,
"YOU and BAU" took an idea for a brand-new experience and made it a reality.
That's exactly what the Creative Center thrives on: continuing to create designs
that form new value out of collaborations with creators from both inside and outside the Sony organization.