Deeper architectures
https://arxiv.org/pdf/1709.01507.pdf
关键词 learning ，representation
- VGGNets [11] and Inception models [5] showed that increasing the depth of a network could significantly increase the quality of representations that it was capable of learning.
- By regulating the distribution of the inputs to each layer, Batch Normalization (BN) [6] added stability to the learning process in deep networks and produced smoother optimisation surfaces [12].
- Building on these works, ResNets demonstrated that it was possible to learn considerably deeper and stronger networks through the use of identity-based skip connections [13], [14].
An alternative, but closely related line of research has focused on methods to improve the functional form of the computational elements contained within a network.
- Grouped convolutions have proven to be a popular approach for increasing the cardinality of learned transforma- tions [18], [19].
- More flexible compositions of operators can be achieved with multi-branch convolutions [5], [6], [20], [21], which can be viewed as a natural extension of the grouping operator.
