Sony’s character
conversation AI

Conversational AI technology
for interactions

Sony’s character conversation AI
technology enables free
communication with
brought to life by human creators

Character conversation
AI technology

People wish to talk freely with their favorite characters, create AI characters on their own and provide various services using AI characters.
At the Sony Group R&D Center, we are working on the character conversation AI technology to make these dreams come true. Interacting with characters in a more realistic way requires not just natural language processing technology for language understanding and generation but many other underlying technologies as well, including voice signal processing technology for voice recognition and speech synthesis and image processing technology for image recognition and graphic expression.
Deliver exciting experiences by combining the technologies to turn creators’ imaginations into reality with the Sony Group’s entertainment prowess - that’s what we are developing the character conversation AI technology for. In the context of meeting this goal, we are proposing new values using this technology through open demonstration experiments and application development activities.

Sony’s conversation
control technology

In building AI characters, we use scenario-based conversations in combination with machine learning-based conversations, which helps realize a wide range of conversations that script writers want to present. First, professional writers create scenario-based, emotionally charged conversations that emphasize the personalities of their characters. Then, machine learning adapts those conversations for a broader range of topics, without ruining the characters’ traits.

Machine learning-based

Scenario-based conversation control is working enough to create AI characters in a small amount of conversation scenarios. Our technology, however, needs to deal with diverse colloquial expressions and many different topics and therefore mixes scenario-based conversations with responses generated by deep learning (DNN). Based on the models learned from massive amounts of conversation data we have gathered on our own, the technology adds and learns the lines that the characters will likely speak. This enables the efficient generation of conversations that reflect the personalities of the characters.

AI remembers conversations
with users

We consider “mutual understanding” crucial for vitalizing communication between AI characters and users. As an approach to mutual understanding, we offer a feature for AI to remember the data about users’ preferences and lifestyles (user models) and apply those data for conversations. Our technology uses this user model-based conversation mechanism for conversations with characters.

Business case study of the
character conversation AI

AI entertainment service
(Possessive Boyfriend)”
Official Site

- Aki Shindo
* The screens presented here are those currently under development and may differ from the screens of the final product.  
©Sony Music Solutions Inc. All rights reserved.

Interactive character agent
Official Site

©Sony Music Solutions Inc. All rights reserved.

Character of CHELULU - Midori Sawamura
CHELULU remembers the faces of people through face recognition and is able to communicate in a way
that suits the individual it speaks to.

Open experiment of the
character conversation

Live concert by 22/7 (a.k.a. Nananiji)
Surprise event at a regular
performance: Sakura Fujima
welcomes visitors

Show video Show video

Click here for the description of the video content.

Here is a video showing a demo conversation of our system between visitors and Sakura Fujima.
©22/7 PROJECT Sakura Fujima (22/7) CV: Sally Amaki

We exhibited a demo in which Sakura Fujima welcomed visitors to her group’s regular performance and souvenir shops, saying “Thank you for coming!” In this surprise event, the visitors responded to Sakura’s greeting and she reacted to their response, and their conversation went on and on.

December 7, 2021