Most camera phones have limited dynamic range, which restricts their ability to capture detail in both light and dark areas at the same time. Pictures of people standing with their back to a window often results in a white background with the person reduced to a dark silhouette. While it is possible to design and manufacture CMOS imagers with wide dynamic range they are relatively expensive components. Software solutions can go a long way toward correcting the imbalance so the user is not disappointed by even a badly composed photograph (corrected by the camera). By equipping the camera with face-based imaging, the prominent face in the field of view can be identified and the exposure, focus and color balance all optimized for that region and the photograph rendered acceptable, as shown in Figure 11.

Once faces have been identified, further interesting camera features become possible. Two obvious examples are smile and blink detection (see Figure 12). The ability to acquire an image only when the faces are smiling and not blinking greatly improves the ratio of good to mediocre photographs and boosts the user's self-esteem as a photographer and satisfaction with the camera phone in general. A tantalizing possibility that arises from face detection is face recognition. In the future it may be possible to automatically tag photographs with information about the people they contain. Photo albums could then be electronically cataloged by content, rather than just date as at present.

One of the consequential benefits of identifying faces in photographs is that a face provides orientation data, offering a simple solution for auto-rotation of handsets without requiring additional hardware. Electro-mechanical devices to determine handset orientation with respect to gravity are not particularly cheap, small or low-power devices, so substitution by an existing camera module plus some software is highly desirable.

Integration
One of the challenges presented by software-enhanced optics and numerical image enhancement is where to put the algorithms on the cell phone. To a certain extent it is decided by whether the algorithm needs to operate on the raw imager data or part of the image data file. Transformations that are applied to raw image data are best embedded on the CMOS imager, particularly if they require less than 100k gates. However even this distinction is not clear-cut as it is usually possible to partition the algorithm so part runs on raw data and part on the image data file. For the software that is not embedded on the imager chip it could be placed either in an image sensor processor, a co-processor or even the baseband processor.

Particularly for the latter two sites there is the further choice of firmware or software. Generally the more advanced the handset the greater is the inherent computational ability, and hence the algorithms can be located further from the camera module. With handset manufacturers looking to find additional revenue streams, the possibility of selling a base model handset that can be upgraded in performance by purchasing the appropriate software is another possibility.

The most efficient approach to software integration is to obtain all of the algorithms required for the software-enhanced lenses and consumer features from a single source. This is because many of the algorithms require common inputs to function. For example most software that does image manipulation performs a background calculation of grey scale range and color balance. Obtaining an integrated software solution avoids duplication of resources since a single grey scale range calculation can be made available to all of the algorithms that need it. Fundamental metrics common to all algorithms can be derived once and reused among the numerical image enhancement and software-enhanced optics solutions as necessary for each picture. The software can be further layered with higher-level features built on lower platforms. The result is more robust, compact, faster and more easily upgraded software, all of which reduces the cost of ownership.

Tessera is one of very few companies able to provide the full integrated suite of technology required for low cost, compact and feature-rich camera modules for cell phones, including wafer-level packaging of imagers, wafer-level optics, wafer-level camera solutions, software enhanced optics and numerical image enhancement.

About the Author
Giles Humpston, Ph.D., serves as Director, Research and Development of Tessera. Dr. Humpston has spent his entire professional career working in the field of semiconductor packaging, initially for military applications and more recently for high volume consumer products. He is a metallurgist by profession and has a doctorate in alloy phase equilibria. Dr. Humpston is a cited inventor on more than 75 patents and has co-authored several text books on metallic joining processes. His work and technical publications have been recognized with five international awards. Dr. Humpston's current interests are packaging of solid state camera modules and product miniaturization through wafer level technologies. He can be reached at: ghumpston@tessera.com

For more on the subject:

Tips and Tricks: The critical nature of cell phone camera packaging
What you need to know about imaging solutions for camera phones
Cameras in handsets evolving from novelty to DSC performance, despite constraints
Image pipeline: Fine-tuning digital camera processing blocks
Get enlightened about camera phone flash units--compare Xenon to high current LEDs