iOS工程添加OpenCV配置方法如下
https://blog.csdn.net/verybigbug/article/details/113588991
配置好后，就可以在移动端开发OpenCV了。我用的是Swift语言。
1 简单的图片处理
用import opencv2可以直接导入OpenCV，不需要写c或者bridging代码。
大部分方法可以用Imgproc直接调，OpenCV的核心图像类Mat可以与iOS的UIImage和CGImage相互转换。
import opencv2
class MyViewController: UIViewController {
override func viewDidLoad() {
super.viewDidLoad()
let image1 = UIImage(named: "001")!
let iv1 = UIImageView(image: image1)
let iv2 = UIImageView()
iv1.frame = CGRect(x: 100, y: 100, width: 200, height: 200)
iv2.frame = CGRect(x: 100, y: 300, width: 200, height: 200)
view.addSubview(iv1)
view.addSubview(iv2)
let m1 = Mat(uiImage: image1)
Imgproc.cvtColor(src: m1, dst: m1, code: .COLOR_BGRA2GRAY)
iv2.image = m1.toUIImage()
}
}
2 使用相机
使用CvVideoCamera2类，设置帧率、尺寸、方向等参数，开启相机，然后在CvVideoCameraDelegate2的processImage代理方法中可以获取实时图像。
有几点需要注意：
- CvVideoCamera2对象要放在类里面，不能放在方法里，否则会被马上回收
-
processImage方法不是主线程，设置图像需要在主线程 - 图像是BGR格式的要转成RGB，不然你就会发现你的脸是绿的！
import opencv2
class MyViewController: UIViewController, CvVideoCameraDelegate2 {
var lastTime = 0.0
func processImage(_ image: Mat!) {
Imgproc.cvtColor(src: image, dst: image, code: .COLOR_BGR2RGB)
DispatchQueue.main.async {
self.camView.image = image.toUIImage()
print("processImage mat \(image.size()) time \((Date().timeIntervalSince1970 - self.lastTime) * 1000) ms")
self.lastTime = Date().timeIntervalSince1970
}
}
let cam = CvVideoCamera2.init()
lazy var camView = UIImageView(frame: view.frame)
override func viewDidLoad() {
super.viewDidLoad()
camView.contentMode = .scaleAspectFill
let w = UIScreen.main.bounds.width
camView.frame = CGRect(x: 0, y: 0, width: w, height: w * 720 / 1280)
view.addSubview(camView)
cam.delegate = self
cam.defaultAVCaptureDevicePosition = .front
cam.defaultAVCaptureSessionPreset = AVCaptureSession.Preset.hd1280x720.rawValue
cam.defaultAVCaptureVideoOrientation = .portrait
cam.defaultFPS = 30
cam.start()
}
3 人脸识别
有三种方式，其中两种是OpenCV的：级联分类器和DNN，它们要用模型文件，下载地址我在上一篇中提到了，另一种是iOS自带的CIFilter方式。我分别实现一下。
3.1 级联分类器人脸识别
我用的iOS设备是A12处理器的iPad Mini5，检测时间在70ms左右，每秒只有十几帧，有点卡；用pyrDown将图像缩小后（注释的代码）检测时间提高到33ms左右，明显流畅了，当然检测准确率还是一般。
let cc_path = Bundle.main.path(forResource: "lbpcascade_frontalface_improved", ofType: "xml")
lazy var cc = CascadeClassifier.init(filename: cc_path!)
let gray = Mat()
Imgproc.cvtColor(src: image, dst: gray, code: .COLOR_BGRA2GRAY)
// Imgproc.pyrDown(src: gray, dst: gray)
Imgproc.equalizeHist(src: gray, dst: gray)
var rects:[Rect2i] = []
cc.detectMultiScale(image: gray, objects: &rects)
for r in rects {
// r.x *= 2
// r.y *= 2
// r.width *= 2
// r.height *= 2
Imgproc.rectangle(img: image, rec: r, color: Scalar(0, 0, 255, 255), thickness: 2)
}
3.2 DNN 人脸检测
检测效果非常好，检测时间在55ms左右，稍微有点卡，并且缩小图像并不能加快速度。
目前我还没想到能加快计算速度的方法，它应该不支持iOS设备的GPU加速，也许用TensorFlow Lite模型？
let pb_path = Bundle.main.path(forResource: "opencv_face_detector_uint8", ofType: "pb")
let pbtxt_path = Bundle.main.path(forResource: "opencv_face_detector", ofType: "pbtxt")
lazy var net = Dnn.readNetFromTensorflow(model: pb_path!, config: pbtxt_path!)
let blob = Dnn.blobFromImage(image: image, scalefactor: 1.0, size: Size2i(width: 300, height: 300), mean: Scalar(104,177,123), swapRB: false, crop: false)
net.setInput(blob: blob)
let probs = net.forward()
let probsData = Data.init(bytes: probs.dataPointer(), count: probs.elemSize() * probs.total())
let detectionMat = Mat(rows: probs.size(2), cols: probs.size(3), type: CvType.CV_32F, data: probsData)
for i in 0..<detectionMat.rows() {
let confidence = detectionMat.get(row: i, col: 2)[0]
if confidence > 0.5 {
let x1 = Int32(detectionMat.get(row: i, col: 3)[0] * Double(image.cols()))
let y1 = Int32(detectionMat.get(row: i, col: 4)[0] * Double(image.rows()))
let x2 = Int32(detectionMat.get(row: i, col: 5)[0] * Double(image.cols()))
let y2 = Int32(detectionMat.get(row: i, col: 6)[0] * Double(image.rows()))
let r = Rect2i(x: x1, y: y1, width: x2 - x1, height: y2 - y1)
Imgproc.rectangle(img: image, rec: r, color: Scalar(0, 0, 255, 255), thickness: 2)
}
}
3.3 CIFilter 人脸检测
检测前用CIImage.init(cgImage: image.toCGImage())将Mat转换成CIImage格式
检测时间在33ms左右，比较流畅，检测效果比DNN略差，但是也很准确了。
lazy var cidetector = CIDetector.init(ofType: CIDetectorTypeFace, context: nil)!
let features = cidetector.features(in: CIImage.init(cgImage: image.toCGImage()))
print("processImage ciimage features \(features.count)")
for f in features {
let x = Int32(f.bounds.minX)
let y = Int32(f.bounds.minY)
let w = Int32(f.bounds.width)
let h = Int32(f.bounds.height)
let r = Rect2i(x: x, y: image.height() - y - h, width: w, height: h)
Imgproc.rectangle(img: image, rec: r, color: Scalar(0, 0, 255, 255), thickness: 2)
}
