Advanced Pedestrian-
Assistance Systems:
A pedestrian-side
mechanism to protect
pedestrians
from traffic
accidents
While the markets of ADAS and self-driving technologies are booming, no manufacturer has
yet succeeded in creating a viable pedestrian safety system. I believe that researching and
developing such a system is a mission for Sony - a leading innovator that has excellent
sensing and UI technologies.
Yusuke Hieida
Index
The birth of my daughter has
raised my awareness of the
need to
eliminate traffic
accidents involving children
An expert in mobility sensing and AI technologies myself, I have always had a strong passion to reduce traffic accidents. In 2018, my daughter was born, and that made me more motivated to protect people, children in particular, from traffic accidents. That was why I chose pedestrian safety as our theme when participating in the Challe-Suppo competition.
The reason we focused on a pedestrian safety system was because we thought that there was a limit to reducing traffic accidents with vehicle-centered approaches alone.
Many companies are already devoting their efforts to building automobile driving safety systems such as ADAS (Advanced Driver-Assistance Systems) and self-driving systems. On the other hand, it will take a long time before these mobility technologies are implemented in cars and spread throughout society. It is said that self-driving cars supporting Level 5 of driving automation that are permitted to drive in urban areas will account for less than 1% of all the vehicles on the market even in 2040. In contrast, there is almost no effort under way to develop a safety system from the viewpoint of pedestrians.
As we researched on our theme, we found out that Sony has many technologies needed to implement a pedestrian safety system, including mobile devices, sensing, and UI. We defined the term APAS (Advanced Pedestrian-Assistance Systems) to refer to pedestrian-side safety systems to protect pedestrians from traffic accidents and hoped to make this idea widely known inside our company.
APAS not only contributes to traffic safety but has potential to spur innovation in Sony’s existing businesses. For example, if APAS is installed to Sony’s smartphones and audio products, these existing mobile devices will have new added value of “safety.” We also think that combining “insurance” for traffic accidents guaranteed by automobile and health insurance policies with “prevention” enabled by APAS will provide a higher level of safety.
Advice from experts in various
fields helped me win the Grand
Prize
In 2019, we participated in Challe-Suppo with the theme of APAS. We did not win the Grand Prize, but it meant a lot to me to make my idea known inside the company. After the presentation, I got introduced to many people, and volunteers from a wide range of fields, including UI, vision, optics, and sensing, joined our team. I also remember that one veteran employee who saw my presentation at Challe-Suppo encouraged me by saying: “Your project is good. A good project attracts people. So, it will succeed.” This made me realize anew that Sony has a bottom-up culture where many people are willing to cooperate in activities they empathize with.
As I continued my research to enter the Challe-Suppo competition in 2020 for a second time, it was really reassuring to have Mr. Shunjiro Nishi (Imaging Products and Solutions Division, Sony Corp.) working with me. He was my classmate in high school and is a fellow engineer and father of two who shares my passion to protect the safety of children. Mr. Nishi has profound knowledge about those fields outside my sphere of expertise, hardware in particular. He provided me with a lot of know-how that would be necessary for commercialization.
While reviewing my work, I not only thought deeply about the development vision but also reconsidered how to convey my concept to other participants. I came up with some ideas to make my presentation easy to understand even for laypeople. For example, I created a conceptual model for children that I could use in my presentation, as well as several explanatory videos. As a result, we won the Grand Prize on our second try. Our R&D work is still continuing as an official project.
Shunjiro Nishi
Yusuke Hieida
The key is to avoid succumbing
to an engineer’s egoism
or getting stuck in local
minimum
What I see as the key in the research and development of APAS is to think from a user-driven perspective (users are pedestrians). I avoided succumbing to the egoism of an engineer trying to solve all problems with his/her technology alone. So, in addition to an analysis of traffic accidents, we interviewed experts in the cognitive development of children and parents of young children.
Through these efforts, we have realized a number of things. For example, data shows that children tend to observe traffic signals because the traffic signal instructions are plain to see: Walk when the light turns green, and wait when the light is red. By contrast, verbal instructions seem difficult for children to follow, since they are often ambiguous, as in “Don’t fool around in a dangerous place.”
APAS gives vibration or audio alerts to children in real time when they face a dangerous situation. The alerts are not only intended to prevent accidents, but children also learn from these alerts about what kind of situation or behavior is dangerous. This makes children be able to avoid dangerous situations by themselves. I think that APAS helps reduce accidents as children grow.
It should be noted, however, eliminating all traffic accidents with APAS alone is infeasible, just as ADAS and self-driving alone are not enough to fully protect the safety of pedestrians. If we cling to this impractical goal, we will get stuck in local minimum. First, we need to develop a vision of an ideal transportation system, including ADAS, self-driving, and infrastructure, in addition to APAS. We then think about what APAS should do in such a transportation system.
In 2021, we analyzed the data of traffic accidents, considered the proposed MVP (Minimum Viable Product) with just enough features to be of value, and laid out a business plan. From now on, our project will shift to the proof-of-concept phase to test this hypothesis. We also intend to work with more partners such as external organizations and municipalities.
Qualitative research shows positive results about APAS. However, since this product has never been in practical use, it is unknown how high the level of user acceptance will be. We need to implement APAS at the social level and conduct field tests and researches in cooperation with users. I hope to see APAS work together with ADAS, self-driving systems, and transportation infrastructure, thereby realizing a zero-traffic accident society in the future.
COLUMN
Mechanism of the APAS that
avoids traffic accidents
with sensing and notification
parts
APAS roughly consists of two parts. The sensing part monitors the surrounding environment and actions of pedestrians and determines whether a traffic accident is likely to occur in the monitored situation. The notification part provides pedestrians with the information obtained from monitoring results in real time or postmortem. Several sensor and UI configurations for different users and use cases are now under consideration. Another feature of APAS is that it is designed to collaborate with external systems through communication. The APAS-sensed information on the actions and surrounding environment of pedestrians is shared with vehicles and infrastructure via communications networks such as cellular and V2X. All these work together to enable an accident-free transportation system.
