It’s time to innovate
As part of one of the world's most innovative and recognizable brands, we are committed to support university research and innovation in North America, while also fostering partnerships with university faculty and researchers. This Research Award Program provides funding for cutting-edge academic research and helps build a collaborative relationship between faculty and Sony researchers. With awards up to $150,000 per year for each accepted proposal, both the Faculty Innovation Award and Focused Research Award create new opportunities for university faculties to engage in pioneering research that could drive new technologies, industries and the future.
Faculty Innovation Award
Up to $100K in funds to conduct cutting-edge research in Sony's general areas of interest
Focused Research Award
Up to $150K in funds to conduct research in the areas of Sony's immediate interest
Eligibility, requirements, submission protocol, and terms are explained in these guidelines.
Proposal submission is open from July 15, 2019 to September 15, 2019.
Congratulations to all award recipients in the Sony Research Award Program! We sincerely look forward to working closely with you.
2018 (four more pending)
- Professor David Bishop, Boston University
- Professor Oliver Cossairt, Northwestern University
- Professor Jonathan Fan, Stanford University
- Professor Bolin Liao, University of California Santa Barbara
- Professor Patrick Lin, Cal Poly, San Luis Obispo
- Professor Wojciech Matusik, Massachusetts Institute of Technology
- Professor Florian Metze, Carnegie Mellon University
- Professor Dimitris Papailiopoulos, University of Wisconsin-Madison
- Professor Ted Sargent, University of Toronto
- Professor Sebastian Scherer, Carnegie Mellon University
- Professor Muhammad Shahzad, North Carolina State University
- Professor Todd Sulchek, Georgia Institute of Technology
- Professor Shimeng Yu, Georgia Institute of Technology
- Professor John Zhang, Dartmouth College
- Professor Song Han, Massachusetts Institute of Technology*
- Professor Aggelos Katsaggelos, Northwestern University*
- Professor Daniel Sanchez, Massachusetts Institute of Technology*
- Professor Faramarz Fekri, Georgia Institute of Technology
- Professor Kristen Grauman, University of Texas at Austin
- Professor Song Han, Massachusetts Institute of Technology
- Professor Daniel Sanchez, Massachusetts Institute of Technology
- Professor Xinyu Zhang, University of California, San Diego
- Professor Stefano Ermon, Stanford University*
- Professor Aggelos Katsaggelos, Northwestern University*
- Professor Shivendra Panwar, New York University*
- Professor Dirk Bernhardt-Walther, University of Toronto
- Professor Stefano Ermon, Stanford University
- Professor Aggelos Katsaggelos, Northwestern University
- Professor Scott Kuindersma, Harvard University
- Professor Bruno Olshausen, University of California, Berkeley
- Professor Shivendra Panwar, New York University
* Renewed Research Collaboration
FACULTY INNOVATION AWARD
Global research and development at Sony enables us to foster innovative ideas, which could ultimately lead to future technology advancements and company growth. In order to speed up and expand the creation of new ideas, we would like to partner with universities. This partnership will help cultivate advanced concepts and fertilize our own research and development. The Sony Faculty Innovation Award provides up to $100K in funds to conduct pioneering research in the areas listed below. Please select the single most relevant keyword to your submission.
- Free View Point Processing
- Point Cloud Processing
- Light Field Processing
- Computer Generated Hologram
- Computational Photography/Display
- Photoreal Digital Double/Twin
- Deep Online Learning
- Domain Generalization and Adaptation
- Recognition under Low-Light
- Monocular 3D Vision
- Video Style Transfer and Manipulation
- Multimodal Sensor Data Fusion
- Functional Image Sensing
- Cognitive Vision
- Novel Electro-Acoustic Transducer
- Acoustic Metamaterials (Audible Range)
- Emotional Speech Synthesis
- Speech Recognition
- Spoken Language Understanding
- Knowledge-grounded Conversational Agent
- Text Generation for Asking Questions
- Information Extraction
- Meta Learning and Memory Models
- Self-supervised Learning
- Neural Architecture Search and Compact Modeling
- Controllable GANs
- Few Shot in GANs
- Visual Explanation from Deep Learning
- Explainable AI
- Correcting Data Bias by Learning
- Sample-efficient Imitation Learning
- Deep Learning with Quantum Computing (including Annealing)
- Privacy-preserving Machine Learning
- Federated Learning
- Robust Edge AI
- Interaction Technology for AR/VR/MR
- System Technology for AR/VR/MR
- Cloud AI for Real-world User Interfaces
- Collaborative Human-Robot-Interaction
- End-to-End Neural Dialog System
- Weak Supervision for Dialog System
- Robust and Precise Force Sensor
- Tactile Sensor with High Spatial Resolution
- Short-range Compact Proximity Sensor
- Devices to Control Adhesion or Friction
- Sensors and Instruments for Robotic Surgery
- Learning and Adaptation for Surgical Autonomy
- Computationally-efficient Nonlinear Control Algorithms
- Machine Learning for Motion Control and Planning
- Virtual Simulator with Sensor Emulation
- Control via Predictive Models
- High Precision Autonomous Mapping
- Autonomous Infrastructure Inspection
- One-shot Robot Skill Learning
- Online Skill Adaptation with Safety Guarantee
- Linguistic Interaction for Robot Behavior Customization
- Shared Autonomy for Infrastructure Inspection Robots
- Self-organized Role Assignment in Group Robots
- 5G Network Quality Prediction
- URLLC for Video and Control Data
- Backscattering Communication
- New Waveform Design for Tera-hertz communication
- Machine Learning for High-resolution mm-wave Radar
- Automatic Parallelization on Multicore Systems
- Machine Learning based Compiler
- Secure Computation
- Privacy-preserving Blockchain
- Blockchain with Self-sovereign Identity
- Post-quantum Blockchain
- LSI Architecture for Sub-milliwatt Machine Learning
- Energy-efficient Environment Recognition for Robotics
Devices and Materials
- Remote Biosignal (EEG, ECG, PPG, EDA, BP) Sensing
- Noninvasive wearable Biosignal Sensing
- Artifact Removal for Wearables in Daily-life
- Non-Volatile Full-Color Reflective Display
- Memory Device and Material (NN)
- Design Technology Co-Optimization (NN)
- Materials Informatics (including Characterization Informatics)
- Quantum Computing for Quantum Chemistry
- Materials Characterization and Simulation for Phononics
- Materials Characterization by Ptychographic Approach
- Efficient Cooling Structure Using Additive Manufacturing
- High-Power-Density Device Cooling Technology
Biomedical and Life Science
- Biomolecular Imaging
- Bioassays and Molecular Probes
- Cell Analysis and Processing
- Surgical Imaging and Operating Room Solutions
- Pathology Imaging (including Multiplex Immunofluorescence)
- Medical Robotics
- Digital Health (excluding Digital Therapeutics)
- Digital Therapeutics
FOCUSED RESEARCH AWARD
Solid research is the underlying driving force to crystallize fearless creativity and innovation. While we are committed to run in-house research and engineering, we are also excited to collaborate with academic partners to facilitate exploration of new and promising research. The Sony Focused Research Award provides an opportunity for university faculty and Sony to conduct this type of collaborative, focused research. The award provides up to $150K in funds, and may be renewed for subsequent year(s). A list of candidate research topics appears below.
Advanced Image Processing enabled by AI
Recent advances in machine learning have created a paradigm shift for many applications. For instance, deep learning based approaches have achieved significant performance improvements over previous state-of-the-art algorithms in image classification, image segmentation, and image recognition. Sony is looking for innovative research in image processing that is based on machine learning to significantly improve existing image processing techniques and applications.
- Topics of interest include, but are not limited to:
- Multi-view image generation from single-view, depth information, stereo-view, etc.,
- Specialized generation for display devices,
- Image/video generation such as text to photo-realistic image synthesis, style transfer, modal transfer based on new approach (e.g. interactive application using a model),
- Image/video compression for viewing and sensing such as generative compression, codec pre/post filter optimization, deep learning hashing,
- Subjective metrics for predicting photo-realistic image quality, and
- Complexity/computational cost reduction for the above applications.
Note: No super-resolution proposals please as we already have several initiatives in this area.
Robust Mesh Tracking for Volumetric Capture
3D Rigging technology is commonly used to animate 3D models. However, it requires temporally consistent 3D mesh structures, which means it can only be applied to 3D computer graphics models. On the other hand, volumetric capture generates 3D mesh sequences frame-by-frame and they are temporally inconsistent. Thus, it’s hard to re-animate them afterward. Sony is looking for innovative mesh tracking approaches that can generate temporally consistent 3D mesh sequences from inconsistent ones. Applications not only include re-animation but various others such as 3D compression and pose/parts recognition.
- Approaches to achieve robust mesh tracking. The mesh tracking should be able to:
- Generate temporally consistent 3D mesh sequences from inconsistent ones and
- Achieve robust tracking results against topology changes, geometry noise and dynamic scenes.
Deep Learning based Video Editing for Content Creation
Deep learning technology is widely used for practical applications such as conventional object detection, segmentation, speech recognition, etc. Furthermore, with the advent of Generative Adversarial Networks (GANs), deep learning applications have spread rapidly to creative areas such as image/sound generation, and, in recent years, its applications have continued to expand to the processing and generation of video. Sony is calling for novel deep learning based video translation/generation/editing technologies that consider the environmental 3D structure (3D model, depth, optical flow) and semantics (i.e. objects, things, stuff).
- Novel deep learning technologies which will be needed to realize 3D environment aware video translation/generation/editing.
- Examples include:
- Video translation considering 3D structure information which was captured by 3D depth sensor,
- Motion video creation from a still image,
- 3D-aware view point and lighting change as video editing, and
- Super photo-realistic post filter.
Neural Architecture Search with Distributed Deep Learning
Deep learning has already been used widely for a variety of practical applications. Distributed deep learning is a helpful method to reduce training time. On the other hand, it is difficult to create greater accuracy models. Neural Architecture Search (NAS) is one of the methods employed to resolve this. However, it is very computationally intensive to optimize neural architecture, especially for distributed deep learning. Sony is looking for innovative NAS approaches for distributed deep learning with large scale GPUs to reduce optimizing time.
- NAS for distributed deep learning approaches which:
- Reduce optimizing time with large scale GPUs, and high scalability,
- Reduce search space efficiently even with maximizing training accuracy, and
- Can be applied to various types of DNN training tasks (classification, segmentation, generation, behavior recognition, etc.).
Machine Learning/Artificial Intelligence for Wireless Communications
Sony is seeking advanced machine learning (ML) or other artificial intelligence (AI) technology for wireless communications and beyond-5G mobile networks. Recently, ML has been applied to many industrial fields and has produced significant results. ML is also being considered for application to mobile networks. ML can improve prediction, estimation and optimization performance. Furthermore, it could be used to design totally new functional blocks in beyond-5G mobile networks.
- Topics of interest include ML or other AI technology applied to:
- Physical layer (new designs of reference signals, control signals, channel codec strategies, etc.),
- Spectrum and interference management (radio propagation prediction, beam-forming strategies, efficient radio resource utilization, etc.),
- Network layer (routing decision, congestion prediction, traffic flow estimation, etc.)
- Proposal are encouraged to clarify benchmarking with existing technologies, describe proper ML/AI modeling schemes, and provide necessary information to be collected for ML/AI in the topics above.
5G-based Wireless Communication Technologies for Robotics Applications
Sony is seeking wireless communication technologies based on 5G networks for robotics applications. Market demand for Industrial Internet of Things (IIoT) has been increasing rapidly. As promising applications of IIoT, robotics automation tasks such as remote control, automated robotics operation, anomaly detection and recovery, etc., are strongly desired. To realize these applications with flexibility and low-cost, wireless communication is necessary. The wireless communication should achieve use-case-specific performance such as much higher reliability (e.g. < 10-6 block error rate @ 256 bytes), lower latency (e.g. < 1 msec @ round trip time), and massive connections (e.g. > 1 million nodes @ 1 km2), in comparison with existing technologies such as 4G cellular networks and WLANs.
- Topics of interest include:
- Definitions of target use cases and a description of their related problem(s) to be solved by the proposed solution(s),
- Designing physical and higher-layer protocols and corresponding mechanisms/algorithms, and
- Performance evaluations based on either computer simulations and/or test-bed.
The actuator is one of the key components for robotic applications. The combination of an electro-magnetic motor, reduction gears such as a harmonic drive, and an encoder should be the most popular way to construct an actuator; but it is still heavy and expensive. To make robots ubiquitous, a new actuator that is safer and less costly is strongly demanded, where a new driving principle other than electro-magnetism is employed and the gear reduction mechanisms are integrated. Sony is interested in new actuators, including artificial muscles, variable impedance actuators, SEAs, USMs, and other actuators based on new concepts.
- To realize an actuator to be able to control the position (angle), the (angular) velocity, and the force (torque).
- To realize an actuator whose power/weight ratio is greater than that for the combination of an electro-magnetic motor and a reduction gear.
Counterfactual Machine Learning
Counterfactual Machine Learning could play an important role in entertainment DTC (Direct To Consumer) services. Such services need a recommender system that can be affected by the previously learned recommender “policy.” Moreover, the "lift" effect, the difference in the willingness to buy an item between when a user is “treated” (e.g. with a purchase recommendation) and when a user is not treated (e.g. in a control group), provides important information for businesses but traditional machine learning frameworks cannot handle such problems appropriately. Sony is looking for novel machine learning approaches that are applicable for such practical consumer services that have a solid theoretical basis, high efficiency and scalability.
- Individual treatment effect (ITE) prediction, so called "uplift modeling", that can be applied to various types of “treatments” including recommended products/services, text/graphics/charts to communicate with a user, pricing, and combinations of these.
- Off-policy optimization and evaluation for recommender systems and marketing solutions.
Fault-tolerant Sensor Fusion Technology for Autonomous Mobility
Robotic vehicles fulfill important tasks ranging from automation to guidance, from autonomous transportation to aerial observations. In these circumstances, reliable and accurate collection of location information is indispensable for effective interaction with the environment. In order to achieve reliable operation, homogeneous/heterogenous sensor fusion technologies have been applied to increase accuracy, reliability and robustness. Multiple or homogeneous sensor fusion is a powerful method to realize more accurate performance by reducing possible sensor failures. Heterogeneous sensors would compensate for each other’s environmental weakness. However, in some real life situations, current approaches cannot efficiently detect errors caused by environmental/intrinsic factors such as multipath error in GPS, reflections in SLAM, drift problems in IMUs, etc. These unpredictable errors caused by lack of autonomous abilities may lead to catastrophic/lethal consequences, such as accidents. Even though some techniques are already available, sensor fusion may suffer degradation to guarantee the necessary accuracy in real-life environments (e.g. non-gaussian and non-linear noise). Sony is looking for novel approaches to achieve fault-tolerant and highly reliable heterogeneous sensor fusion techniques.
- Various approaches or theories are possible to guarantee a high accuracy of sensor fusion. These may include:
- Utilization of new mathematical models, intelligent signal processing and/or machine/deep learning algorithms/methods,
- Detection of multiple-sensor failure,
- Providing guaranteed accuracy information,
- Flexibility in the number and type of sensors,
- Massive integration of heterogeneous sensors, or
- A system with practical consumer-oriented parameters such as power requirements (<1 W), size (<10 cm), and cost (<$1,000).
Organic Materials and Photonic-Devices using Singlet Fission
Singlet fission has recently become an active area of research within the field of organic electronics. Sony believes that this technology is still in its infancy, but that it has the potential to open up new frontiers. We consider this research to be in an exploratory phase, and we should examine the potential of this technology. An optical device is one application of this technology that is being considered. Additionally, a search will be conducted for the best application(s) of this technology.
- We look forward to receiving research proposals on materials and devices related to singlet fission. The specific scope of proposals may include:
- Materials that cause singlet fission with high efficiency,
- Optical devices using the aforementioned materials, or
- New applications not limited to optical devices.
Metasurface Flat Optics
Metasurface Flat Optics is a most promising technology with which to innovate various optical systems. It will introduce new optical functionality, for example, dispersion control, polarization control, and aberration control by means of a single flat optical element. Sony believes that this technology will eventually be utilized in a wide range of Sony products, including camera products, display products, and sensor components.
- Design Methods for Dispersion-Engineered or Polarization-Engineered Metasurface Flat Optics including but not limited to:
- Achromatic metasurface with high NA and high throughput efficiency,
- Multi-functional metasurface that controls the dispersion and polarization of light,
- Large area (tens of millimeters in diameter) metasurface flat optics in periodic and aperiodic structures,
- Multi-layered metasurface flat optics, and
- Sophisticated design processes leveraging Deep Learning and Inverse Design.
- High-Volume Production Processes for Metasurface Flat Optics, including:
- Large-area metasurface production processes, and
- Nanoimprint lithography processes for high-volume metasurface production.
- Practical Applications of Metasurface Flat Optics including but not limited to:
- Applications for miniaturization of camera modules,
- Applications for novel display technology including AR/VR, light field, and holography, and
- Applications for optical sensor devices.
Intelligent Sensing Technologies for Surgical Robotics
At the autonomy level of current medical robotics, the critical mission of the visual system that is connected to a medical robot is to capture and display a high-resolution, low-latency image of the surgical field for surgeons and assisting staff. As the autonomy level increases in the future, however, the visual system is expected to collect and analyze a wide variety of data from multi-modal sensors, in order to provide the interpreted information for both human medical practitioners and also the robotic control system. As such, Sony is looking for innovative sensing and analysis technologies for the future.
- Medical imaging and sensing for semi-/fully autonomous medical robotics. The scope may include but need not be limited to:
- Multi-modal sensing systems,
- Multi-modal information fusion, or
- Auxiliary aids for medical practitioners.
Intelligent Sensing Technologies for Digital Health
Digital technologies help medical providers and other stakeholders reduce inefficiencies, improve access, reduce costs, increase quality, and personalize medicine for patients. In line with this trend, computer vision, combined with artificial intelligence, will have significant impact in areas such as telemedicine and patient monitoring. Sony is looking for breakthrough healthcare solutions based on intelligent imaging and/or sensing technologies.
- Computer vision and its applications in digital health with an emphasis on clinical validation. The scope may include but need not be limited to:
- Digital therapeutics (DTx),
- Software as a medical device (SaMD),
- Auxiliary diagnosis, or
- Aids for clinical workflows.
We accept applications from Principal Investigators (PIs) who are full-time professors (adjunct professors are not eligible) or researchers and are eligible to supervise Ph.D. students at a university/educational institution in the USA or Canada. Full professors, associate professors, and assistant professors are eligible to apply.
We accept applications that have a PI and one or more co-PIs for the same proposal. However, only one award is made to the primary PI and the primary PI's university/institution if the proposal is selected. All Co-PIs must meet the same eligibility criteria as that for a PI, and Co-PIs will be required to sign program documents.
Multiple Proposal Submissions
A PI or different PIs from the same university/institution may submit more than one proposal for different research topics. However, please do not submit identical proposals more than once. Note that at the bottom of your submission confirmation email, there is a link to resubmit your proposal in the event that you discover that your original had an error or an omission. Please use this link for resubmissions and do not resubmit identical proposals. Also, please do not submit “test” submissions. In the rare event that you do not receive a confirmation email for your submission, please send an email to ResearchAwardProgram@sony.com informing us of your issue.
Please select one target award between the Focused Research Award and the Faculty Innovation Award when you submit a proposal. Refer to the following comparison chart:
|Focused Research Award||up to $150K||Choose from the Focused Research Theme List||One (1) year,|
with a possible extension
|Faculty Innovation Award||up to $100K||Choose from the list of Keywords||One (1) year,|
with a possible extension
Target Research Topic
Please select one Focused Research Theme for your proposal if you choose the Focused Research Award. Please select one keyword (the single most relevant keyword for your proposal) if you choose the Faculty Innovation Award.
A sponsored research agreement is required that is mutually agreed upon by Sony, the PI, and his/her institution on all program terms including objectives, milestones, publication, use of research, and patent rights before any award is made or funding is available. This sponsored research agreement is negotiable at the award stage and must be between Sony, the PI, and the PI's university/institution only.
Term of Proposals
Please submit proposals for a one year period only. No multi-year proposals will be accepted. An extension of the research may be possible depending upon the results of the research collaboration in year one, but it will require a separate discussion for another award the following year.
Three (3) quarterly reports and a final research summary report are required at a minimum.
Visiting Researcher Support
Sony may request the PI to support a Sony visiting researcher(s) at Sony's option and expense.
Proposal authors or universities/institutions must ensure that no confidential or proprietary information is included in submitted proposals. Sony will treat all information submitted in proposals as non-confidential and non-proprietary.
Intellectual Property (IP) Rights
IP rights are negotiable at the award stage and will be specified in the sponsored research agreement, but at a minimum, Sony requires the right to utilize the results of the research that it sponsors.
Please include the required items listed below in your submission:
- Title, abstract, methods, goals/milestones, references, and either one Focused Research Theme (if for the Focused Research Award) or one primary keyword (if for the Faculty Innovation Award).
- Please describe your submission’s differentiation from the current state-of-the-art.
- Please include the best contact email address and phone number for the PI.
All proposal contents must fit within 11 pages (a ten-page maximum proposal with references and a one-page budget summary). The file format must be a PDF or a MS Word file format and must be under 16 MB in size. Out of consideration to reviewers, please limit your minimum font size to a 10 point font.
Curriculum Vitae (CV)
The CV for the PI must be included when you submit a proposal. The CV file must be a separate file from the proposal file. There is no page limitation for the CV, however the CV file size must be under 16 MB.
The Focused Research Award is limited to a maximum of $150K per proposal. The Faculty Innovation Award is limited to a maximum of $100K per proposal. This funding is a sponsored research grant that is to be used to conduct the research described in the proposal, and includes any overhead related to this research and any other fees or charges needed to carry out the research. Sony will not specify a maximum amount or percentage of the budget that may be allocated to overhead.
Submission Terms and Conditions
The submitter must agree to the 2019 Sony Research Award Program Submission Terms and Conditions (click to see the PDF file) before you submit a proposal. Please obtain a prior review of your submission by your institution’s sponsored research office if they require a review of research proposals prior to their submission.
Submissions must be done through the online submission form (click to open). Submissions made by email will not be accepted. Submitters will automatically receive a confirmation email once they complete the online submission form and process. Please keep the confirmation email as proof of each submission.
Submissions must be completed by 11:59 pm PDT on September 15, 2019.
Duplicate Submissions and Resubmissions
Duplicate submissions will not be accepted for the same proposal title from the same PI. Please do not make duplicate submissions. Use the resubmission link at the bottom of your confirmation email if you discover that your submission has an error or an omission.
Required Information for the Online Submission Form
The following information will be required in order to complete the online submission form for each proposal submitted.
- Proposal information; proposal title, proposal file (11 pages maximum including proposal, references, and 1-page budget summary), and either one Focused Research Theme (if for the Focused Research Award) or one primary keyword (if for the Faculty Innovation Award).
- PI's Information; PI's full name, PI's title, PI's CV file, PI's email address, SAT International Code for PI's university/institute (please find PI's SAT International code here), and PI's department name. Please be careful to use the correct code because some schools have several codes to designate different programs or campuses. Also note that some school names use common abbreviations (e.g. INST for INSTITUTE) or simply truncate some words in the school’s name when searching for a school code. Please enter “0000” if you cannot find the four digit SAT International Code for your school.
- Submitter's Information, if not the PI (optional); Submitter's full name and email address
Announcement of Review Results
Principal Investigators (PIs) will be notified of the submission review results for each proposal submitted around March of 2020. Sony will only be able to provide some limited feedback on the review process results for the highest-ranked proposals due to limited resources.
Program details will be negotiated with a PI and their university/institution if their proposal is selected. Funding will be available only after we have agreed to the terms of and signed a sponsored research agreement.
Inquiries Related to the 2019 Sony Research Award Program
Please send an email to the Research Award Program Administration Office at (ResearchAwardProgram2019@Sony.com) if you have a question regarding the 2019 Sony Research Award Program. The Research Award Program Administration Office is the only resource that can officially answer your question(s).
Inquiries Outside of the 2019 Sony Research Award Program
Please visit www.sony.com and contact the appropriate channel listed at the bottom of the web page if you have questions outside of the Sony Research Award Program (such as business proposals, joint venture proposals, research proposals that are not related to any of Focused Research Themes or keywords for the Faculty Innovation Award), or other inquiry or proposal.