Automotive vision systems can alert you when you veer out of your lane, or when there are pedestrians behind you or other vehicles in your blind spot. They can even help you parallel park without hitting the curb. Thus far, however, the systems' high cost has relegated them largely to luxury vehicles.
Now the Swiss Center for Electronics and (CSEM S.A.; Neuchatel, Switzerland) claims to have the technology to lower the cost of such systems from thousands of dollars to hundreds. CSEM envisions OEMs designing the ViSe smart image sensor into real-time vision systems cheap enough to become standard equipment on automobiles worldwide.
“We think the ViSe image chip has enormous potential,” said David Alexander, senior analyst at ABI Research, which recently completed a study of the market for automotive lane-departure sensing systems. “With safety applications in particular, real-time reactions are of vital importance, so you usually need an expensive, high-performance image processor. But with the ViSe image chip, you don't need as powerful a processor, dramatically bringing down the price of these systems.”
The worldwide market for new vehicles is 60 million units annually, according to the ABI report, but this year only about 25,000 will ship with lane-departure warning systems installed. Unless the price of such systems comes down dramatically, ABI predicts, they will ship in only about 300,000 vehicles in 2012.
But “if CSEM can bring that cost down from $2,000 to $200, then you could expect 10 times more systems to be installed, or as many as 3 million worldwide by 2012,” said Alexander.
CSEM, which is partly supported by public funds, has a licensing-based business model. It has already licensed customized versions of its ViSe real-time solution to OEMs that supply optical character-recognition systems to banks for scanning checks. Adelsa Group LLC (San Francisco) handles licensing to U.S. companies.
Now CSEM has put together demonstrations, including an evaluation kit with application code, for automotive lane-departure warning systems, night vision systems, parking assistance, vehicle occupancy monitoring, pedestrian detection and blind-spot monitoring. CSEM also supplies demonstrations for security applications, such as intrusion detection systems that key on human movement but ignore pets' comings and goings, and for industrial machine vision.
“This is not research; the ViSe image chip is the product of an eight-year development effort,” said ABI's Alexander. “They are in their third generation already, and the next generation will put everything onto one chip.”
The current OEM kit is a two-chip solution that pairs the proprietary vision sensor with an inexpensive Blackfin DSP from Analog Devices Inc. Next, CSEM plans to integrate a proprietary DSP cell with its imager in a one-chip solution. To demonstrate that even its current offerings have radically reduced the necessary data rate, thus lowering the cost of smart camera applications from thousands to hundreds of dollars, CSEM is offering a wireless evaluation kit, called ViSeLink, that includes its two-chip solution plus a Bluetooth radio chip inside a camera housing with a lens.
“The way automotive vision systems are built today, you don't have enough bandwidth to send a data stream from a video camera over a wireless link and extract scene features remotely,” said Christian Enz, vice president of the Microelectronics Group at CSEM. “But with our chip set, you can extract all the information locally and just send the results over a very low-data rate wireless transceiver like Bluetooth.”
Traditional video camera solutions send high-bandwidth raw video data over a hard wire to a high-performance image processing chip, then run algorithms that compare successive frames of an image stream to detect contrast, draw outlines and recognize objects. The ViSe chip does all the image processing in circuitry that surrounds each pixel on the chip, sending the results of the algorithms to an inexpensive application processor like the Blackfin.
“The chip makes it much easier for OEMs to develop vision applications. For automotive applications like lane-departure warning systems, all the hard image processing is done for you by the image chip,” said Alexander. “Infiniti has had a lane-departure system out for a couple of years, but it uses a CMOS video camera feeding an expensive image processor chip they had to develop themselves, resulting in a cost to the buyer of about $2,000. CSEM's chip could bring the cost to under $200. We believe that if real-time vision systems can get the cost down to under $100, then they could become standard equipment.”
Another alternative for automobile makers today is to pair a CMOS video camera with a dedicated image processor from Mobileye Inc. (Southfield, Mich.). BMW has adopted such systems, which need a hard wire connecting the camera to the Mobileye processor (www.eetimes.com, search article ID: 197800142).
Other companies that have traveled a similar route to CSEM's by integrating image processing onto the same chip with a vision sensor are Canesta Inc. (Sunnyvale, Calif.), which has design wins at Honda (www.eetimes.com, search article ID: 196513738) and International Electronics & Engineering S.A. (IEE, Luxembourg) which has design wins at Volkswagen. But both companies' chips sacrifice image resolution in order to fit extra circuitry into their image arrays for performing distance calculations to create a 3-D model of a scene.
For 3-D applications–such as adaptive cruise control, where the system needs to know the distance to the vehicle directly ahead of the driver's car–CSEM has a version of the ViSe that can calculate distance using time-of-flight calculations on a per-pixel basis, much like the Canesta and IEE vision sensors.
Inside the chip
The ViSe image chip surrounds each pixel with comparators and gating circuitry that performs a local analog calculation of the contrast between the pixel and each of its nearest neighbors. The image array also has pixel-by-pixel integrators that track contrast changes through time. Contrast between pixels is thus automatically calculated for every pixel simultaneously, while the integrators keep track of how contrast is changing in the scene. Because contrast information is relatively insensitive to illumination, the ViSe chip's output does not change when the ambient light changes.
“We do most of our image processing on the image chip at the pixel level, by extracting the contrast locally from pixel to pixel, including both its magnitude and orientation–whether you have a step up or a step down in contrast in all directions. In addition, we track the variations in contrast through time,” said Edoardo Franzi, CSEM's section head for sensory information processing. “The contrast information is insensitive to [variations in] illumination, such as when a car is passing under trees or entering a tunnel.”
While he calculations are performed in parallel on the chip, the image's information is read out in order of significance, rather than in a blind raster scan. The paired DSP queries the image chip, sending it a contrast threshold. The imager then reads out only the pixels whose contrast exceeds the threshold. The DSP progressively lowers the threshold, and the imager reads out pixel locations with successively lower contrast, until the application makes the necessary recognition. At that point, the process resets.
“Our goal was to extract the minimum amount of information possible to perform the application. We do not scan the image array line by line, like a video camera, but only send information about the pixels that change,” said Franzi. “Plus we output the pixels with the largest change first–the most important information–then [proceed] in descending order of contrast so that when the application finds what it needs, it can stop the communication.”
For instance, for the lane-departure application, as soon as the DSP finds the high-contrast pixels lining the lane, it resets the threshold, leaving the rest of the scene blank. Time is not wasted reading out low-contrast pixels that don't help locate the lines on the road.
CSEM said that the ViSe chip offers 100 dB of dynamic range and 30 dB of resolution for the output contrast information.