项目上一直需要人眼跟踪功能,目前用的是开源的neven,比较老了,效果不好。
考虑使用neven做人脸检测,加一个cnn网络做人脸对齐。
使用ncnn框架,需要将ios camera输出的CMSampleBufferRef转化成Mat。
neven输入为灰度图像,而cnn输入为rgb彩色图像,camera只能输出一种模式,rgba或者yuv420。
考虑到bgra转gray计算量小于yuv转bgr,camera优先输出bgra
在原有项目上修改的代码,在camera的相关类里面,怎么设置输出bgra都不行,后来才发现代码不规范在启动预览前设置成yuv420.
yuv420转cv::Mat
-(int)ProcessWithSampleBufferRef:(CMSampleBufferRef)sampleBufferRef{
CVImageBufferRef pixelBuffer = CMSampleBufferGetImageBuffer(sampleBufferRef);
CVPixelBufferLockBaseAddress(pixelBuffer, 0);
uint8_t yuv = (uint8_t)CVPixelBufferGetBaseAddressOfPlane(pixelBuffer, 0);
int wid = (int)CVPixelBufferGetWidth(pixelBuffer);
int hei = (int)CVPixelBufferGetHeight(pixelBuffer);
cv::Mat img = cv::Mat(hei*1.5, wid, CV_8UC1, yuv);
cv::Mat img2;
cv::cvtColor(img, img2, cv::COLOR_YUV420sp2RGB);
CVPixelBufferUnlockBaseAddress(pixelBuffer,0);
return 0;
}
bgra转cv::Mat:
uint8_t bgra = (uint8_t)CVPixelBufferGetBaseAddressOfPlane(pixelBuffer, 0);
cv::Mat img = cv::Mat(hei, wid, CV_8UC4, bgra);