Image sensors are evolving in three core ways: more edge of system functionality with the rise of the internet of things (IoT); implementation of new features such as on-chip polarization and hyperspectral sensors, which can see more than the naked eye; and, possibly the most fundamental of advances for the majority of machine vision applications, is the need to capture at ever greater resolutions – and do so faster.
This article looks at the evolution of GS-CMOS image sensors, including what to expect from the soon-to-ship fourth-generation global shutter technology, and their role in improving imaging performance.
Global-shutter CMOS image sensors were first launched roughly a decade ago, and since then have played a crucial role in enabling the accelerated throughput of high-speed manufacturing processes. The technology not only gave a digital output, but also avoided spatial distortion caused by the rolling-shutter effect.
The technology has evolved quickly to give greater image quality, with the first-generation sensors delivering just 2.4 megapixels in its 1/1.2” format (5.86 µm pixel size). Demands on resolution meant that engineers created the second-generation sensor pixel size of 3.45 µm allowing Sony to cover resolutions from 0.4 to 31 megapixels.
But as pixel size shrinks, so does the amount of light reaching each pixel in the sensor – reducing the saturation capacity.
With the third-generation, an optimal balance was sought between these competing factors: increasing pixel size slightly (to 4.5 µm), and thereby increasing saturation capacity closer to the first-generation devices, whilst also improving the dynamic range and speed.
With the completion of the first to third generation pixels, Sony created a resolution and optical size portfolio that covered the legacy CCD line up which was being discontinued.
A holistic approach to image capture
Machine vision systems need to not only capture detailed images for analysis, they need to capture the right information, transfer this information to a computer and do this at exceptionally high speeds.
The sensor’s readout frame rate (as much as the transmission standard used) is therefore a crucial element in this process. But so too are the features being embedded in each new generation of GS-CMOS image sensors. Generation 1 included the global shutter, to eliminate motion artifacts; and a multi-frame region of interest (ROI) feature, which allows a subset of data to be passed to the computer for analysis.
Generation 2 added multi-exposure triggers, allowing multiple exposures to be captured within one image frame to ensure images captured greater depth of information – and reduced the minimum exposure time to just 2 µs.
Generation 3 included a dual ADC and dual trigger, which allowed a low- and high-gain image taken on the same frame with each being able to be triggered independently. Additionally, an on-sensor conversion gain was embedded to better balance sensitivity, saturation capacity and dynamic range to cope with both low- and -bright-light conditions. Finally, a self-trigger was added, with one ROI acting as the trigger for another.
Inverting the sensor
While it is still possible to increase the overall pixel count by increasing the image sensor size, most machine vision applications use a C-mount camera which use a 1-inch type sensor (16 mm diagonal).
The first three generations of GS-CMOS image sensors used a front-illuminated pixel structure (see figure 1), with light entering the lens, before going through the metal wiring layer and onto the light-sensitive photodiode.
This reduces the light reaching the photodiode layer with a proportion of the light entering the lens being directed onto the metal wiring layer.
An alternative approach being adopted for fourth-generation GS-CMOS is to invert the metal-wiring- and light-sensitive-photodiode-layers to create a back-illuminated pixel structure and make it easier for photons to be detected (see figure 2).
This inverted structure allows pixel size to be reduced to approximately 63% vs the conventional front illuminated sensor (2.74µm) without reducing saturation characteristics.
Additionally, this inversion allows circuits that were peripheral to be arranged at the back of the sensor. This allows the resolution to be increased – from 12 MP to 20 MP – while the package size was reduced to about 91%; even when using the same optical system as previous models (see figure 3).
Fourth generation features and readout frame rates
The inverted back-illuminated pixel structure also gives the ability to implement a highly flexible wiring layout and by combining this with a scalable low-voltage signaling with embedded clock (SLVS-EC) high-speed interface it’s possible to significantly improve the readout frame rate. Taking into account the higher resolution images being transmitted, the readout frame rate can be nearly 2.4 times faster than conventional sensors.
Additional features implemented on fourth-generation sensors include an on-sensor combination of dual ADC data, allowing an HDR (high dynamic range) image to be created from the low- and high-gain captures. Shutter speed has been accelerated, with the time between shutters down to just 2 µs.
To implement a high degree of automation enabling advances such as replacing visual inspection at factories and warehouses and in other applications of factory automation and distribution, inspection and recognition must be more precise and faster, enabling greater efficiency.
The fourth-generation sensor is a big leap forward for the industry and for industrial automation in general – allowing both better images and a greater set of on-chip features that improve the quality control and speed of industrial manufacturing processes.
However, this need for ever greater manufacturing speed and accuracy will continue, and there is an issue on the horizon that needs to be addressed before it is a problem.
As we have seen, sensor manufacturers can shrink pixel size beyond that optimal point. Indeed, it can be argued that there is a commercial interest for some to do so; taking advantage of non-specialists’ use of it as a proxy metric (often the only metric) to quantify the image quality of their system.
We as an industry need to work together to deliver a better alternative metric. One that takes into account the complex nature of industrial imaging. One that looks more holistically at the quality and the count. And ultimately, one that is as easy to understand as simply pixel count.
Roberto Buttaci is the lead product manager for image sensors at Sony Semiconductor Solutions Europe and responsible for Sony’s industrial, AV and wearable CMOS image sensors in Europe.
- Embedded Vision Summit: ML perception on tiny FPGAs
- 30 fps automotive imaging radar processor debuts
- Ray tracing levels aims for clarity with imaging graphics SoCs
- Smart camera offers turnkey edge machine vision edge AI