版本记录

版本号	时间
V1.0	2018.01.31

前言

人脸识别是图像识别技术中的一种，广泛的应用于很多领域，接下来这几篇我们就一起来研究几种关于人脸识别的技术。感兴趣的可以参考上面几篇文章。
1. 人脸识别技术（一） —— 基于CoreImage实现对静止图片中人脸的识别

基于CoreImage的视频中人脸识别技术

第一篇文章我们利用CoreImage对静止的图像进行人脸识别，相对来说，静止图像还是好识别的，如果要识别由摄像头采集来的视频中的人脸，那就相对来说难了，因为会有很多的性能问题。下面我们就一起看一下，利用AVFoundation进行图像采集，利用CoreImage识别视频中的人脸。

功能实现

还是直接看一下代码。

#import "ViewController.h"
#import <AVFoundation/AVFoundation.h>

@interface ViewController () <AVCaptureVideoDataOutputSampleBufferDelegate>

@property (nonatomic, strong) AVCaptureSession *captureSession;
@property (nonatomic, strong) AVCaptureDevice *captureDevice;
@property (nonatomic, strong) AVCaptureDeviceInput *captureVideoDeviceInput;
@property (nonatomic, strong) AVCaptureVideoDataOutput *captureMovieFileOutput;
@property (nonatomic, strong) AVCaptureConnection *captureConnection;
@property (nonatomic, strong) AVCaptureVideoPreviewLayer *previewLayer;
@property (nonatomic, strong) NSMutableArray <UIView *> *faceViewArrM;

@end

@implementation ViewController

- (void)viewDidLoad
{
    [super viewDidLoad];
    
    self.faceViewArrM = [NSMutableArray array];
    
    self.captureSession = [[AVCaptureSession alloc] init];
    if ([self.captureSession canSetSessionPreset:AVCaptureSessionPresetHigh]) {
        self.captureSession.sessionPreset = AVCaptureSessionPresetHigh;
    }
    else {
        self.captureSession.sessionPreset = AVCaptureSessionPreset1280x720;
    }
    
    for (AVCaptureDevice *device in [AVCaptureDevice devices]) {
        if ([device hasMediaType:AVMediaTypeVideo]) {
            if (device.position == AVCaptureDevicePositionFront) {
                self.captureDevice = device;
            }
        }
    }
    
    //添加输入
    [self addVideoInput];
    
    //添加输出
    [self addVideoOutput];
    
    //添加预览图层
    [self addPreviewLayer];
    
    [self.captureSession commitConfiguration];
    [self.captureSession startRunning];

}

#pragma mark -  Object Private Function

- (void)addVideoInput
{
    NSError *error;
    self.captureVideoDeviceInput = [AVCaptureDeviceInput deviceInputWithDevice:self.captureDevice error:&error];
    if (error) {
        return;
    }
    if ([self.captureSession canAddInput:self.captureVideoDeviceInput]) {
        [self.captureSession addInput:self.captureVideoDeviceInput];
    }
}

- (void)addVideoOutput
{
    self.captureMovieFileOutput = [[AVCaptureVideoDataOutput alloc] init];
    [self.captureMovieFileOutput setSampleBufferDelegate:self queue:dispatch_get_global_queue(DISPATCH_QUEUE_PRIORITY_HIGH, 0)];
    self.captureMovieFileOutput.videoSettings = [NSDictionary dictionaryWithObjectsAndKeys:[NSNumber numberWithUnsignedInt:kCVPixelFormatType_32BGRA], (id)kCVPixelBufferPixelFormatTypeKey, nil];
    if ([self.captureSession canAddOutput:self.captureMovieFileOutput]) {
        [self.captureSession addOutput:self.captureMovieFileOutput];
    }
    
    //设置链接管理对象
    self.captureConnection = [self.captureMovieFileOutput connectionWithMediaType:AVMediaTypeVideo];
    //视频旋转方向设置
    self.captureConnection.videoScaleAndCropFactor = self.captureConnection.videoMaxScaleAndCropFactor;;
    //视频稳定设置
    if ([self.captureConnection isVideoStabilizationSupported]) {
        self.captureConnection.preferredVideoStabilizationMode = AVCaptureVideoStabilizationModeAuto;
    }
    
//    AVCaptureFileOutputDelegate *del = nil;
}

- (void)addPreviewLayer
{
    self.previewLayer = [AVCaptureVideoPreviewLayer layerWithSession:self.captureSession];
    [self.previewLayer setVideoGravity:AVLayerVideoGravityResizeAspect];
    self.previewLayer.frame = self.view.bounds;
    [self.view.layer addSublayer:self.previewLayer];
}

- (void)detectFaceWithImage:(UIImage *)image
{
    // 图像识别能力：可以在CIDetectorAccuracyHigh(较强的处理能力)与CIDetectorAccuracyLow(较弱的处理能力)中选择，因为想让准确度高一些在这里选择CIDetectorAccuracyHigh
    NSDictionary *opts = [NSDictionary dictionaryWithObject:
                          CIDetectorAccuracyHigh forKey:CIDetectorAccuracy];
    // 将图像转换为CIImage
    CIImage *faceImage = [CIImage imageWithCGImage:image.CGImage];
    CIDetector *faceDetector = [CIDetector detectorOfType:CIDetectorTypeFace context:nil options:opts];
    // 识别出人脸数组
    NSArray *features = [faceDetector featuresInImage:faceImage];
    // 得到图片的尺寸
    CGSize inputImageSize = [faceImage extent].size;
    //将image沿y轴对称
    CGAffineTransform transform = CGAffineTransformScale(CGAffineTransformIdentity, 1, -1);
    //将图片上移
    transform = CGAffineTransformTranslate(transform, 0, -inputImageSize.height);
    
    //清空数组
    dispatch_async(dispatch_get_main_queue(), ^{
        [self.faceViewArrM enumerateObjectsUsingBlock:^(UIView * _Nonnull obj, NSUInteger idx, BOOL * _Nonnull stop) {
            [obj removeFromSuperview];
             obj = nil;
        }];
    });
    
    // 取出所有人脸
    for (CIFaceFeature *faceFeature in features){
        //获取人脸的frame
        CGRect faceViewBounds = CGRectApplyAffineTransform(faceFeature.bounds, transform);
        CGSize viewSize = self.previewLayer.bounds.size;
        CGFloat scale = MIN(viewSize.width / inputImageSize.width,
                            viewSize.height / inputImageSize.height);
        CGFloat offsetX = (viewSize.width - inputImageSize.width * scale) / 2;
        CGFloat offsetY = (viewSize.height - inputImageSize.height * scale) / 2;
        // 缩放
        CGAffineTransform scaleTransform = CGAffineTransformMakeScale(scale, scale);
        // 修正
        faceViewBounds = CGRectApplyAffineTransform(faceViewBounds,scaleTransform);
        faceViewBounds.origin.x += offsetX;
        faceViewBounds.origin.y += offsetY;
        
        //描绘人脸区域
        dispatch_async(dispatch_get_main_queue(), ^{
            UIView* faceView = [[UIView alloc] initWithFrame:faceViewBounds];
            faceView.layer.borderWidth = 2;
            faceView.layer.borderColor = [[UIColor redColor] CGColor];
            [self.view addSubview:faceView];
            [self.faceViewArrM addObject:faceView];
        });
        
        // 判断是否有左眼位置
        if(faceFeature.hasLeftEyePosition){
            NSLog(@"检测到左眼");
        }
        // 判断是否有右眼位置
        if(faceFeature.hasRightEyePosition){
            NSLog(@"检测到右眼");
        }
        // 判断是否有嘴位置
        if(faceFeature.hasMouthPosition){
            NSLog(@"检测到嘴部");
        }
    }
}

#pragma mark -  AVCaptureVideoDataOutputSampleBufferDelegate

- (void)captureOutput:(AVCaptureFileOutput *)output didOutputSampleBuffer:(CMSampleBufferRef)sampleBuffer fromConnection:(AVCaptureConnection *)connection
{
    NSLog(@"----------");

    [connection setVideoOrientation:AVCaptureVideoOrientationPortrait];
    CVImageBufferRef buffer;
    buffer = CMSampleBufferGetImageBuffer(sampleBuffer);
    
    CVPixelBufferLockBaseAddress(buffer, 0);
    uint8_t *base;
    size_t width, height, bytesPerRow;
    base = (uint8_t *)CVPixelBufferGetBaseAddress(buffer);
    width = CVPixelBufferGetWidth(buffer);
    height = CVPixelBufferGetHeight(buffer);
    bytesPerRow = CVPixelBufferGetBytesPerRow(buffer);
    
    CGColorSpaceRef colorSpace;
    CGContextRef cgContext;
    colorSpace = CGColorSpaceCreateDeviceRGB();
    cgContext = CGBitmapContextCreate(base, width, height, 8, bytesPerRow, colorSpace, kCGBitmapByteOrder32Little | kCGImageAlphaPremultipliedFirst);
    CGColorSpaceRelease(colorSpace);
    
    CGImageRef cgImage;
    UIImage *image;
    cgImage = CGBitmapContextCreateImage(cgContext);
    image = [UIImage imageWithCGImage:cgImage];
    [self detectFaceWithImage:image];
    CGImageRelease(cgImage);
    CGContextRelease(cgContext);
    
    CVPixelBufferUnlockBaseAddress(buffer, 0);
}

@end

下面看一下部分输出

2018-01-31 14:18:07.001789+0800 JJFaceDetector_demo2[4700:1444754] ----------
2018-01-31 14:18:07.168074+0800 JJFaceDetector_demo2[4700:1444754] 检测到左眼
2018-01-31 14:18:07.168400+0800 JJFaceDetector_demo2[4700:1444754] 检测到右眼
2018-01-31 14:18:07.168557+0800 JJFaceDetector_demo2[4700:1444754] 检测到嘴部

2018-01-31 14:18:07.174485+0800 JJFaceDetector_demo2[4700:1444754] ----------
2018-01-31 14:18:07.388472+0800 JJFaceDetector_demo2[4700:1444754] 检测到左眼
2018-01-31 14:18:07.389386+0800 JJFaceDetector_demo2[4700:1444754] 检测到右眼
2018-01-31 14:18:07.389440+0800 JJFaceDetector_demo2[4700:1444754] 检测到嘴部

2018-01-31 14:18:07.398383+0800 JJFaceDetector_demo2[4700:1444754] ----------
2018-01-31 14:18:07.587945+0800 JJFaceDetector_demo2[4700:1444754] 检测到左眼
2018-01-31 14:18:07.588429+0800 JJFaceDetector_demo2[4700:1444754] 检测到右眼
2018-01-31 14:18:07.588796+0800 JJFaceDetector_demo2[4700:1444754] 检测到嘴部

... ...

下面看一下识别的效果

这个是我自己，不露脸了怕吓到诸位，不过还是可以识别的

另外一个手机中的范爷图片

移动另外一个手机中的范爷图片

几个需要说明的问题

1. info.plist文件添加key

这个简单的说一下就可以了，iOS 10以后，相机权限需要增加key了。

2. 性能问题

移动的时候如果移动过快会有检测不准确的现象，这个是由于，识别和计算脸部位置并进行标记，但是计算好如果正好进行了移动，那么标记的可能还是上一帧的位置，所有有时候标记不那么准确。

3. 部分代码说明

先说一下这一句代码，假如不添加下面这句代码

self.captureMovieFileOutput.videoSettings = [NSDictionary dictionaryWithObjectsAndKeys:[NSNumber numberWithUnsignedInt:kCVPixelFormatType_32BGRA], (id)kCVPixelBufferPixelFormatTypeKey, nil];

我们运行下，看输出

2018-01-31 14:32:57.312082+0800 JJFaceDetector_demo2[4706:1448810] ----------
2018-01-31 14:32:57.312320+0800 JJFaceDetector_demo2[4706:1448810] [Unknown process name] CGBitmapContextCreate: invalid data bytes/row: should be at least 2880 for 8 integer bits/component, 3 components, kCGImageAlphaPremultipliedFirst.
2018-01-31 14:32:57.312431+0800 JJFaceDetector_demo2[4706:1448810] [Unknown process name] CGBitmapContextCreateImage: invalid context 0x0. If you want to see the backtrace, please set CG_CONTEXT_SHOW_BACKTRACE environmental variable.
2018-01-31 14:32:57.312468+0800 JJFaceDetector_demo2[4706:1448810] [api] -[CIImage initWithCGImage:options:] failed because the CGImage is nil.

这里提示的意思是CGBitmapContextCreate创建上下文和图像失败了，是一个无效的数据位，我在stackOverFlow中找到了答案，有人和我碰到了一样的问题。

看一下别人的Answers

Your best bet will be to set the capture video data output's videoSettings to a dictionary that specifies the pixel format you want, which you'll need to set to some variation on RGB that CGBitmapContext can handle.
The documentation has a list of all of the pixel formats that Core Video can process. Only a tiny subset of those are supported by CGBitmapContext. The format that the code you found on the internet is expecting is kCVPixelFormatType_32BGRA, but that might have been written for Macs—on iOS devices, kCVPixelFormatType_32ARGB (big-endian) might be faster. Try them both, on the device, and compare frame rates.

下面我给大家翻译下

您最好的选择是将捕获视频数据输出的videoSettings设置为一个字典，该字典指定了您想要的像素格式，您需要在CGBitmapContext可以处理的RGB上设置一些变量。
文档中列出了a list of all of the pixel formats that Core Video can process。CGBitmapContext仅支持其中的一小部分。您在互联网上找到的代码的格式是kCVPixelFormatType_32BGRA，但可能已经为iOS设备上的Mac编写，kCVPixelFormatType_32ARGB（big-endian）可能会更快。在设备上试用它们，并比较帧速率。

所以加上上面那个setting字典就解决了问题。

后记

本篇已结束，后面更精彩~~~

人脸识别技术（二） —— 基于CoreImage实现视频中人脸的识别

人脸识别技术（二） —— 基于CoreImage实现视频中人脸的识别

版本记录

前言

基于CoreImage的视频中人脸识别技术

功能实现

几个需要说明的问题

1. info.plist文件添加key

2. 性能问题

3. 部分代码说明

后记

推荐阅读更多精彩内容

人脸识别技术 （二） —— 基于CoreImage实现视频中人脸的识别

版本记录

前言

基于CoreImage的视频中人脸识别技术

功能实现

几个需要说明的问题

1. info.plist文件添加key

2. 性能问题

3. 部分代码说明

后记

推荐阅读更多精彩内容

人脸识别技术（二） —— 基于CoreImage实现视频中人脸的识别