Fraunhofer Institute for Applied Optics and Precision Engineering IOFResearch project with BMW enables new safety features
New mini 3D camera creates vital prerequisite for the next stage of autonomous driving
In a joint research project with car manufacturer BMW, researchers at the Fraunhofer Institute for Applied Optics and Precision Engineering IOF have developed a miniaturized wide-angle 3D camera system. In the future, the camera will monitor the interior of autonomously driving vehicles using 3D sensors, thus ensuring greater safety on the roads of the future. The system will be presented at the K trade fair in Düsseldorf from October 19 to 26.
Autonomously driving vehicles promise to transform the traffic on the roads as we have known it to date. We will also experience a completely new level of travel comfort. But the more autonomous a vehicle becomes, the better safety systems are needed – for example, to monitor whether a driver is ready to take control of the car in a critical situation. Researchers at Fraunhofer IOF, in cooperation with BMW, have now developed a miniaturized camera that is designed to do just that: As part of the joint research project MinTOFKA, a miniaturized time-of-flight camera system (ToF), has been developed. With the help of six-channel micro-optics and a VGA-3D ToF sensor, a field of view of more than 170°x130° is covered. This is ideal for capturing gestures and the gaze direction of all passengers.
The system enables a wide range of relevant safety features: In addition to complete driver monitoring, it also includes hand-on-steering-wheel detection as well as seat belt detection and seat occupancy monitoring coupled with airbag control. Also, the presence of children can be explicitly detected. Gesture control additionally enables more convenient access to the infotainment, which equally ensures greater safety on the road thanks to fewer distractions.
Particularly compact design thanks to division of the field of view
The main challenge in developing the system: The car of the future will integrate more and more systems due to increasingly comprehensive and complex features. Accordingly, only limited space is available. There is a maximum of 10 mm for a camera for interior monitoring, which is to be mounted below the rearview mirror. At the same time, in order to cover as much of the interior as possible, the camera must have a greatly expanded field of view. Classic wide-field camera optics cannot be used in this case, as they cannot be reduced in size to the required compactness.
This is where the new mini camera from Jena comes in: The researchers have succeeded in implementing a new, multi-channel camera concept in which a large field of view is divided into several predefined individual channels. “Unlike in classical approaches, the entire field of view that is to be observed is not imaged with one lens,” explains Dr. Robert Brüning, head of department for optical and mechanical system design at Fraunhofer IOF. “Instead, several parallelized imaging channels are used. They each cover only part of the field of view, namely about 90° per channel. To achieve the targeted horizontal field of view of 170°, we choose a combination of three times two channels. For merging the depth maps of both channels for subsequent image processing, we additionally keep an overlap of 15° between the individual fields of view.” This means that the camera uses a computer algorithm to combine multiple images from different viewing directions and the associated 3D data into a single image.
New time-of-flight sensor with higher resolution
The system is based on a time-of-flight camera approach. “It is a 3D sensor technology that can be used to capture people and objects, as well as their position, movement, and shape in three dimensions,” Brüning explains. “This is accomplished by the camera emitting light pulses. It then measures the time it takes for the light – reflected by various points on an object – to return to the camera. By emitting many light pulses in different directions, it is possible to calculate all the objects visible to the camera with a so-called point cloud.” By comparing these point clouds with known shapes, people, as well as objects such as child safety seats, can be identified. The light emitted by the camera lies in the infrared spectrum and is therefore not visible to the human eye.
Previous ToF sensors developed for the automotive industry had a much smaller field of view of 76° x 60° with lower resolution (320 x 240 pixels). They were developed solely for the purpose of detecting hand gestures. The sensor developed by the Fraunhofer IOF team, in contrast, has a significantly expanded range of functions while offering a higher resolution of 640 x 480 pixels (VGA) and a larger field of view. In initial tests, the system demonstrated a detection accuracy of more than 95%.
At Fraunhofer IOF, the development of the optics took place ranging from design, construction, and assembly as well as characterization. In addition, the software for the digital pre-processing of the individual images was developed, which were then combined to form an overall image with a large field of view.
