Face detection with complex backgrounds

classic Classic list List threaded Threaded
1 message Options
Reply | Threaded
Open this post in threaded view

Face detection with complex backgrounds


I'm trying to build a face detector and face recognizer based on
OpenCV. As I'm studying the literature and example source code, I
quickly realized that most of the code out there fail
detecting/recognizing faces in complex backgrounds, or when there are
multiple faces in an image with complex poses.

I'll describe the basics of what I developed so far, and I appreciate
any suggestions and comments.

Upon loading an image, I first attempt to construct a skin filter. The
only reason I do this is because I realized that the default haar
classifier that comes with OpenCV (haarcascade_frontalface_alt.xml) is
very inaccurate when fed with a complete picture with a complex
background. I construct the skin filter in the RGB space after
separating out the image to its R,G and B components. I then mask the
image with this filter, and convert to grayscale before feeding this
into the haar classifier. This didn't really help all that much
because with a complex background, there were a lot a of pixels in the
background that match the skin color, creating a lot of noise.
Moreover, the neck, arms, legs, etc of the people in the picture also
appeared in the skin filter. When the original image filtered with the
skin filter is fed into the haars classifier, it detects many false
faces because of these spots.

To remove skin "noise", I employed connected component analysis to
bring together similar colored pixels to create larger patches of
skin. Then I found the contours of these skin patches and performed
ellipse-fitting in the hopes that the ellipse fitting would pick up
head-like contours. I filtered some of these ellipses based on certain
features (eg, ellipse area < %0.1 of image area). My goal then is to
set the image region-of-interest (ROI) to these ellipses, extract the
ellipse, figure out its orientation (angle), rotate it such that it is
vertically aligned, create a new IplImage and just feed this image to
the haar classifier in the hopes that a 'clean' image like this would
be a much better input to the classifier.

The above may sound all fine and dandy except there are a few
problems. The FitEllipse function in openCv fits an ellipse over a
contour in the least squares sense, ie, it doesn't have to completely
encompass a contour, which in some cases cause some features of the
faces to be cut off. Secondly, complex scenes with multiple people
create some problems. Suppose 3 people in a picture with faces
touching each other. The above algorithm detects and extracts an
ellipse whose X dimension in much larger than its Y dimension (ie, an
ellipse covering all three faces at once), instead of 3 vertically
aligned ellipses, one per face. When this ellipse is then rotated,
what I feed into the haars classifier is essentially 3 rotated faces
and it fails to find these since it is trained to find frontal upright

I really appreciate if you read all this :) Any comments or
suggestions as to how to improve this system? I'd be happy to hear
from anyone who had experience with face detection in complex
scenes/backgrounds. Also, if anyone knows of another face detection
algorithm already implemented in OpenCV (other than haar's), I'd like
to know that too.