不同搜索空间的比较图:
- 空间搜索
- 螺旋搜索
- 线性搜索
- 网格搜索
可以看到,在超参的搜索过程汇总,网格搜索和螺旋搜索,搜索的效果更好,更细致!
如何应对网格搜索巨大的资源消耗问题?
one limitation of GridSearch is that it can be very computationally expensive when dealing with a large number of different hyperparameters and much bigger datasets. Therefore there are two other techniques that we could explore to validate our hyperparameters
- RandomizedSearchCV which can sample a given number of candidates from a parameter space with a specified distribution. Which performs surprisingly well! 从搜索空间中根据特定的分布进行采样,事件证明很有效。
- Or a train / validation / test split, and we can validate our model on the validation set. Often used with much bigger datasets. (有点不太明白和之前学习的有什么区别)