前言

在第3篇教程里面，我们所编写的CNN进行分类的模型准确度达到了80%。对于一个分类模型来说，80%的准确率不算很低了。但是，在现有的情况下，我们应该如何优化这个模型呢？
在从零开始机器学习的系列里面，理论上的优化模型可以修改超参数。同样，在Keras的这个CNN程序中，我们可以指定其他的优化器（这里用的是ADAM）。修改卷积核大小、步长、修改激活函数的类型、加入/取消全连接层、修改每个层有多少神经元也是可行的方法。
面对这么多可以修改的地方，修改现有的程序是必定需要做的。

修改现有的程序

首先确定要修改的内容。全连接的层数、层的大小和卷积层的层数。
全连接层可以有0、1、2层；卷积层可以有1、2、3或更多层；然后就是每层的大小可以是32、64或128...
通过以下的代码来确定这些排列组合可以得到多少模型：

dense_layers = [0, 1, 2]
layer_sizes = [32, 64, 128]
conv_layers = [1, 2, 3]

for dense_layer in dense_layers:
    for layer_size in layer_sizes:
        for conv_layer in conv_layers:
            NAME = "{}-conv-{}-nodes-{}-dense-{}".format(conv_layer, layer_size, dense_layer, int(time.time()))
            print(NAME)

运行上述Python代码，在控制台的输出如下：

1-conv-32-nodes-0-dense-1540975261
2-conv-32-nodes-0-dense-1540975261
3-conv-32-nodes-0-dense-1540975261
1-conv-64-nodes-0-dense-1540975261
2-conv-64-nodes-0-dense-1540975261
3-conv-64-nodes-0-dense-1540975261
1-conv-128-nodes-0-dense-1540975261
2-conv-128-nodes-0-dense-1540975261
3-conv-128-nodes-0-dense-1540975261
1-conv-32-nodes-1-dense-1540975261
2-conv-32-nodes-1-dense-1540975261
3-conv-32-nodes-1-dense-1540975261
1-conv-64-nodes-1-dense-1540975261
2-conv-64-nodes-1-dense-1540975261
3-conv-64-nodes-1-dense-1540975261
1-conv-128-nodes-1-dense-1540975261
2-conv-128-nodes-1-dense-1540975261
3-conv-128-nodes-1-dense-1540975261
1-conv-32-nodes-2-dense-1540975261
2-conv-32-nodes-2-dense-1540975261
3-conv-32-nodes-2-dense-1540975261
1-conv-64-nodes-2-dense-1540975261
2-conv-64-nodes-2-dense-1540975261
3-conv-64-nodes-2-dense-1540975261
1-conv-128-nodes-2-dense-1540975261
2-conv-128-nodes-2-dense-1540975261
3-conv-128-nodes-2-dense-1540975261

也就是说，通过排列组合可以得到27种模型。
修改第四讲的代码，将上述代码加入程序，得到如下：

import time
import tensorflow as tf
from tensorflow.keras.models import Sequential
from tensorflow.keras.layers import Dense, Dropout, Activation, Flatten, Conv2D, MaxPooling2D
import pickle
from tensorflow.keras.callbacks import TensorBoard
import time

gpu_options = tf.GPUOptions(per_process_gpu_memory_fraction=0.33)
sess = tf.Session(config=tf.ConfigProto(gpu_options=gpu_options))

X = pickle.load(open("X.pickle","rb"))
y = pickle.load(open("y.pickle","rb"))

# normalize data image in grayscale is from 0-255
X = X/255.0

# train multiple models
dense_layers = [0, 1, 2]
layer_sizes = [32, 64, 128]
conv_layers = [1, 2, 3]

for dense_layer in dense_layers:
    for layer_size in layer_sizes:
        for conv_layer in conv_layers:
            model_name = "{}-conv-{}-nodes-{}-dense-{}".format(conv_layer, layer_size, dense_layer, int(time.time()))
            tensorboard = TensorBoard(log_dir='logs/{}'.format(model_name))
            print(model_name)
            model = Sequential()

            model.add(Conv2D(layer_size, (3, 3), input_shape = X.shape[1:]))
            model.add(Activation("relu"))
            model.add(MaxPooling2D(pool_size=(2,2)))

            for l in range(conv_layer-1):
                model.add(Conv2D(layer_size, (3, 3)))
                model.add(Activation("relu"))
                model.add(MaxPooling2D(pool_size=(2,2)))

            model.add(Flatten()) #Conv Layer是2D， DenseLayer是1D的 所以需要将ConvLayer压平
            for l in range(dense_layer):
                model.add(Dense(layer_size))
                model.add(Activation("relu"))

            model.add(Dense(1))
            model.add(Activation("sigmoid"))

            model.compile(loss="binary_crossentropy",
                         optimizer="adam",
                         metrics=["accuracy"]) # 可以使用categorical_crossentropy作为损失函数

            model.fit(X, y, batch_size =32, epochs=20, validation_split=0.1, callbacks=[tensorboard])

在控制台运行，一共需要训练27个模型，因此在训练之前，最好将logs的目录下已经存在的日志清空。这样会方便我们观察这一讲中的不同参数对于模型的影响。

使用TensorBoard观察模型

和上一讲一样，使用tensorboard --logdir=logs/打开TensorBoard。

TensorBoard观察所有模型

放大折线图