Insider: How Useful is In-Camera Face Detection?

Since the technology was first introduced late in 2006, it has become increasingly common in point-and-shoot digicams. Most models we have reviewed since mid-2007 include it, although it may carry a variety of labels. As well as Face Detection, Face Recognition, Face Tracking and Face Priority shooting mode are indicators that cameras include some form of face detection.

Compact digicams are often a test bed for technologies that are later shifted into more sophisticated cameras. However, since most DSLR photographers like to have a point-and-shoot camera for everyday use, in this Insider, we’ll look at how face detection technology works – and when it should be used.

Find the Face
As its name implies, face detection is all about finding faces in a scene – and that involves shape recognition. When you switch on a camera with face detection activated, the AF system – and usually also the AE system – links with a microprocessor that analyses the scene, looking for areas that are shaped like human faces. The most commonly used camera systems use two procedures:

1. Identifying regions in the subject that contain skin colour, using a skin filter which detects colour and texture.

2. Scanning these areas for patterns that could represent eyes and eyebrows, nostrils, and mouth.

In the early days of the technology, the algorithms used for the analysis could only detect faces looking directly at the camera. The latest algorithms can also detect faces in semi-profile. This requires the use of a larger number of detection points and much more processing capability, which means the more sophisticated the face detection system.

More sophisticated processing algorithms can also detect more faces in a scene. The systems used in most recent digicams can identify up to 10 faces and adjust focus and exposure to provide the best average result. In most cases, the face detection system will confirm detection by overlaying a rectangle on each face in the scene displayed on the camera’s LCD. Some systems use a different colour to identify the face that will be used as a focusing target.

Tracking facilities are also provided by some recent systems. They can follow one or more faces as they move within the frame and automatically adjust focus and exposure when the camera’s shutter button is half-pressed. Many systems link with the flash exposure system to ensure a natural-looking balance of flash and ambient lighting. Exposure adjustment to correct backlighting is also provided by a few recent systems.

Some recent face detection systems will also correct under-exposure created by backlighting.

FotoNation FaceTracker
The leading developer of face detection technology is FotoNation, which was founded in 1997 and has management offices in the USA, research and development operations in Ireland, Romania and Russia and sales offices in USA, Europe and Japan. Canon, Hewlett-Packard, Kodak, Nikon, Olympus, Pentax, Samsung, Sanyo and Sony are among the manufacturers who use FotoNation’s FaceTracker technology in their digicams.

More than half of all digicams sold today – and 21 out of 48 models introduced in the first half of 2007, are equipped with FotoNation’s technology. However, when we went to press Panasonic was the only manufacturer to introduce it in a DSLR (the DMC-L10).

FotoNation’s FacePro for Mobile Phones, which locates and tracks human faces in camera-phone shots, was demonstrated at PMA 2007 on Nokia’s N-Series camera phones, where it detected and tracked 9 faces simultaneously at speeds of 30 frames per second. Faces can be automatically extracted from captured images and integrated into the address book on the phone. FacePro also allows face thumbnails to be used for quick identification with caller-ID screens and quick-dial menus.

How Useful?
In the hands of a competent photographer, standard AF and AE systems will do a pretty good job of getting most subjects in focus and well-exposed. If you know how to use spot focusing and metering – and can use the shutter button for focus locking – you don’t really need face detection technology, particularly if your camera lets you choose which AF point to use for focusing and metering.
However, when action is fast and furious and you’re engaged in the excitement of the moment, it’s easy to forget these focusing and exposure aids. In-camera face detection will reduce the risk of missing shots in such situations, particularly if it includes face tracking. Face detection is also a great confidence booster for novice photographers.

Two situations in which face detection technology is particularly handy are with off-centre subjects and shots with people on either side of the frame. In both cases, the standard camera AF system is likely to focus on the centre of the scene, leaving the main subjects unsharp.

A downside is the need to compose shots on the LCD screen, which may not be easily viewable in bright conditions. (Many digicams with face detection lack viewfinders, which means photographers have no option in how they compose shots.)

System Failures
Although face detection systems work well in most cases, they cannot perform miracles. There are some subjects that can’t be identified as well as situations where the system may lock focus and exposure on the wrong part of the subject. The most common ‘system failures’ are as follows:

Inability to identify pets (and other animals). Face detection systems have been programmed to recognise human faces only. They will not focus on cat, dog, horse or other animal faces.

Face detection systems can’t identify non-human faces – although they will pick up human faces anywhere in the frame.

Unwanted detection of faces in works of art. Most face detection systems will automatically focus on faces in paintings and photographs as well as statues.

Distance-based failures. Face detection systems will not identify faces that are too close to the camera – or too far away. Because they rely on pattern recognition, they can only identify an entire face so, if the camera ‘sees’ only one eye and part of the subject’s nose, detection could fail.

Although most face detection systems can identify up to 10 faces in a scene, they are less useful when photographing crowd scenes than for photographing small groups of people.

Sunglasses, swimming goggles and other things that cover the eyes can cause face detection systems to fail.

Sunglasses can cause face detection systems to fail, particularly if other cues are obscured.

False detections. These are becoming less common as algorithms are refined. However, there are situations where the face detection system will incorrectly identify patterns on walls, fabrics or similarly-complex subjects as faces.

There’s little point in using face detection for landscapes, animals, birds, flowers, scenery or buildings, unless there are people in the shot that you want to focus upon. Most cameras with face detection allow you to turn it off or select a different shooting mode or scene setting to cover these subjects.