R-CNN, Fast R-CNN, Faster R-CNN, YOLO：目标检测算法总结

参考链接

以下是文中涉及的算法的最原始的文章：

一、介绍

detection algorithm 和 classification algorithm的区别在于，目标检测算法是在感兴趣的物体周围画个圈去定位它；分类算法是predict每个像素点的label。
我们不能通过建立一个标准的最后一层是全连接层的卷积网络来解决目标检测问题，原因是输出层的长度是可变的，并不是一个常量。

一个最直接的解决办法是从图中取不同的感兴趣区域，然后对这些区域用CNN进行分类，检测这些区域中是否有物体的存在。
但是待检测物体可能存在于图片的不同位置而且有不同的长宽比例。所以以上方法需要选取量非常大的区域并需要非常大的计算量。

因此，R-CNN, Fast R-CNN, Faster R-CNN, YOLO被开发去又快又准地找物体。

二、 R-CNN

为了解决上述提到的有大量区域被选择的问题， Ross Girshick et al提出了一种方法：用了选择性搜索从图片提取了2000个区域，这些区域被称为”region proposals“。

image.png

用这种办法，我们不需要去分类巨大数量的区域了，我们只需要去处理2000个区域。这2000个区域是用如下的选择性搜索算法（selective search algorithm）来找到的：

Selective Search:

Generate initial sub-segmentation, we generate many candidate regions (产生子备选区域)

Use greedy algorithm to recursively combine similar regions into larger ones （结合区域）

Use the generated regions to produce the final candidate region proposals （产生最终区域）

这篇文章介绍了更多关于选择性搜索算法(selective search algorithm)的内容。

RCNN步骤：

这2000个备选的region proposals被wrap成一个square，并被喂进卷积神经网络CNN，通过这个CNN输出一个4096维度的特征向量。
These 2000 candidate region proposals are warped into a square and fed into a convolutional neural network that produces a 4096-dimensional feature vector as output.
这个CNN的作用是特征提取器（feature extractor）。它的全连接输出层包含了从图像中提取的特征。
The CNN acts as a feature extractor and the output dense layer consists of the features extracted from the image
这些特征被送到一个SVM里去对 candidate region proposal内是否存在物体进行分类。
the extracted features are fed into an SVM to classify the presence of the object within that candidate region proposal.
为了更准确地去预测region proposal内是否存在物体，算法也预测了4个补偿量（offset values）去增加bounding box准确程度。
In addition to predicting the presence of an object within the region proposals, the algorithm also predicts four values which are offset values to increase the precision of the bounding box.
举个例子，算法能预测某个region proposal内有一个人。但是这个人的人脸不能被bounding box剪掉一半。所以，这些补偿量（offset values）用于调整 bounding box of the region proposal。

R-CNN

R-CNN存在的问题：

因为每张图都需要对2000个region proposals进行分类，所以训练网络还是需要耗费大量时间。
对测试集也不能做到实时出结果，47秒/张图
选择性搜索算法（selective search algorithm）是固定算法。在这一步没有任何学习的过程。这可能会导致产生bad candidate region proposals

三、Fast R-CNN

Fast R-CNN的几个改进：
The same author of the previous paper(R-CNN) solved some of the drawbacks of R-CNN to build a faster object detection algorithm and it was called Fast R-CNN. The approach is similar to the R-CNN algorithm.

把原图放进CNN去找feature map。
But, instead of feeding the region proposals to the CNN, we feed the input image to the CNN to generate a convolutional feature map.
基于feature map来确定 region of proposals 。
From the convolutional feature map, we identify the region of proposals and warp them into squares and
region of proposals通过RoI pooling layer变成固定大小，进而通过全连接层。
by using a RoI pooling layer we reshape them into a fixed size so that it can be fed into a fully connected layer.
用softmax函数预测 class和offset value.
From the RoI feature vector, we use a softmax layer to predict （1）the class of the proposed region and also （2） the offset values for the bounding box.

Fast R-CNN更快的原因是：

每张图只需要计算一次feature map.（之前是计算2000次）
you don’t have to feed 2000 region proposals to the convolutional neural network every time. Instead, the convolution operation is done only once per image and a feature map is generated from it.

image.png

Fast R-CNN更快：
From the above graphs, you can infer that Fast R-CNN is significantly faster in training and testing sessions over R-CNN. When you look at the performance of Fast R-CNN during testing time, including region proposals slows down the algorithm significantly when compared to not using region proposals. Therefore, region proposals become bottlenecks in Fast R-CNN algorithm affecting its performance.

四、Faster R-CNN

上面两个算法的缺点：
selective search耗时
Both of the above algorithms(R-CNN & Fast R-CNN) uses selective search to find out the region proposals. Selective search is a slow and time-consuming process affecting the performance of the network.

Faster R-CNN的改进：
不用selective search去找region proposals；
用network去找region proposals；
Therefore, Shaoqing Ren et al. came up with an object detection algorithm that eliminates the selective search algorithm and lets the network learn the region proposals.

Faster R-CNN的步骤：

把输入图片放到CNN里去产生feature map： Similar to Fast R-CNN, the image is provided as an input to a convolutional network which provides a convolutional feature map.
用一个单独的网络去预测region proposals： Instead of using selective search algorithm on the feature map to identify the region proposals, **a separate network is used to predict the region proposals. **
这一步与fast-RCNN类似：用RoI pooling去reshape和输出。The predicted region proposals are then reshaped using a RoI pooling layer which is then used to classify the image within the proposed region and predict the offset values for the bounding boxes.

image.png

时间上的对比：
Faster R-CNN最快并且能用作实时目标检测

image.png

五、YOLO: You Only Look Once

之前几种算法的缺点：
产生region的时候没有纵览整幅图。其实图的某些部分有更高的可能性包含物体。
All of the previous object detection algorithms use regions to localize the object within the image. The network does not look at the complete image. Instead, parts of the image which have high probabilities of containing the object.

YOLO的思想：
用一个单独的网络去预测bounding boxes和bounding boxes中存在物体的概率
YOLO or You Only Look Once is an object detection algorithm much different from the region based algorithms seen above.
In YOLO, a single convolutional network predicts (1) the bounding boxes and (2)the class probabilities for these boxes.

YOLO示意图

YOLO的具体步骤：
How YOLO works is that:

把图分成SxS个格子，每个格子中取m个bounding boxes。we take an image and split it into an SxS grid, within each of the grid we take m bounding boxes.
对每个bounding boxes，network输出a class probability and offset values：For each of the bounding box, the network outputs a class probability and offset values for the bounding box.
选择概率高的bounding boxes去定位物体：The bounding boxes having the class probability above a threshold value is selected and used to locate the object within the image.

YOLO的优缺点：

优点：更快。YOLO is orders of magnitude faster(45 frames per second) than other object detection algorithms.
缺点：小物体难被检测到。The limitation of YOLO algorithm is that it struggles with small objects within the image, for example it might have difficulties in detecting a flock of birds. This is due to the spatial constraints of the algorithm.