Computer Vision & CG

Break down the border between virtual and real

Real World Sensing

These sensing technologies understand the 3D real world around our users and devices by determining their position, orientation, and surrounding distances, then integrating the results of multiple observations. We are working on 3D computer vision technologies such as depth estimation, visual SLAM, and 3D modeling algorithms in cameras. These technologies can potentially be utilized in a broad range of Sony business areas from mobile and gaming AR to robot navigation. Our goal is to achieve the highest level of performance in the world by not only developing algorithms but also linking them tightly to our proprietary image sensors.

Technical Image of Real World Sensing Technical Image of Real World Sensing

Free-Viewpoint Visualization

Free-viewpoint video technology captures the real-world as 3D data. It enables viewing of the video from any desired viewpoint. This technology is comprised of two capturing methods: omnidirectional (inside-out) visualization and arbitrary direction free-viewpoint (outside-in) visualization. Currently omnidirectional visualization has 3 degrees of freedom, however, we are developing omnidirectional visualization with 6 degrees of freedom which integrates 3 degrees of translation freedom, as well as volumetric capture which captures specific areas of space to achieve arbitrary direction free-viewpoint visualization. We are also working on technological development utilizing video and imaging technology we have accumulated so far to create photorealistic expressions that appear to be real photographed content despite actually being computer graphics.

Image of the use of Free-Viewpoint Visualization Image of the use of Free-Viewpoint Visualization

Image Recognition

We are constantly developing image-recognition technologies for Sony products such as robotics, cameras, games, and mobile products. Specific image recognition technologies that are currently in development include face recognition, gesture recognition, object recognition, and semantic segmentation. In addition, we are developing optimization technologies for bringing together Sony’s sensors, processors, and algorithms. To this end, we have applied machine-learning technologies such as deep learning. Through our research into the recognition capabilities needed to, among others, attain synergies between AI and robotics, we will deliver potent recognition devices which run in real time in the real world, specifically in a wide range of Sony products and services.

Image of Image Recognition from input to output Image of Image Recognition from input to output

Computational Photography

Recently, considerable attention has been focused on the technological field of computational photography, with which new features can be devised by changing and controlling the materials and characteristics of imaging systems (optical systems, lighting, and sensors). We combined Sony’s proprietary imaging-signal processing technology with our original polarization image sensor, multispectral image sensor, and lensless camera to offer new features such as the highly accurate acquisition of shape data, measurement of the activation levels of shrubs and vegetation, and ultra-thin form factor/ultra wide image-capture devices.

Technical Image of Computational Photography Technical Image of Computational Photography
to the top