Conversational AI technology
Sony’s character conversation AI
technology enables free
brought to life by human creators
People wish to talk freely with their favorite characters, create AI characters on their own and provide various services using AI characters.
At the Sony Group R&D Center, we are working on the character conversation AI technology to make these dreams come true. Interacting with characters in a more realistic way requires not just natural language processing technology for language understanding and generation but many other underlying technologies as well, including voice signal processing technology for voice recognition and speech synthesis and image processing technology for image recognition and graphic expression.
Deliver exciting experiences by combining the technologies to turn creators’ imaginations into reality with the Sony Group’s entertainment prowess - that’s what we are developing the character conversation AI technology for. In the context of meeting this goal, we are proposing new values using this technology through open demonstration experiments and application development activities.
In building AI characters, we use scenario-based conversations in combination with machine learning-based conversations, which helps realize a wide range of conversations that script writers want to present. First, professional writers create scenario-based, emotionally charged conversations that emphasize the personalities of their characters. Then, machine learning adapts those conversations for a broader range of topics, without ruining the characters’ traits.
Scenario-based conversation control is working enough to create AI characters in a small amount of conversation scenarios. Our technology, however, needs to deal with diverse colloquial expressions and many different topics and therefore mixes scenario-based conversations with responses generated by deep learning (DNN). Based on the models learned from massive amounts of conversation data we have gathered on our own, the technology adds and learns the lines that the characters will likely speak. This enables the efficient generation of conversations that reflect the personalities of the characters.
AI remembers conversations
We consider “mutual understanding” crucial for vitalizing communication between AI characters and users. As an approach to mutual understanding, we offer a feature for AI to remember the data about users’ preferences and lifestyles (user models) and apply those data for conversations. Our technology uses this user model-based conversation mechanism for conversations with characters.
Business case study of the
character conversation AI
AI entertainment service
Interactive character agent
Character of CHELULU - Midori Sawamura
CHELULU remembers the faces of people through face recognition and is able to communicate in a way
that suits the individual it speaks to.
Open experiment of the
SWORD ART ONLINE EX-
AI conversation experiment with
Takeru Higa on AI creation and
Participants posted their desired conversations via the Web in advance, and these conversations were used to create AI that reflected the personality of each individual participant. At the event, the conversations between the AI of participants and Alice were demonstrated according to the world view of SWORD ART ONLINE. The participants saw AI speak just like they do, even on topics their posts did not mention, making them feel as if they were in the world of SWORD ART ONLINE.
Live concert by 22/7 (a.k.a. Nananiji)
Surprise event at a regular
performance: Sakura Fujima
We exhibited a demo in which Sakura Fujima welcomed visitors to her group’s regular performance and souvenir shops, saying “Thank you for coming!” In this surprise event, the visitors responded to Sakura’s greeting and she reacted to their response, and their conversation went on and on.