代码原始地址
https://tune.tidymodels.org/articles/extras/optimizations.html
以下为其代码的makrdown，笔者翻译为了中文并加入了部分自己的理解，请多指教哈
Title: "Optimizations and parallel processing" 优化和并行处理
knitr::opts_chunk$set(echo = TRUE)
library(tidymodels)
Sub-model speed-ups
For some types of models, such as boosted models or regularized models, the number of models that are actually fit can be far less than the number of models evaluated. For example, suppose a boosted tree is fit with 1000 trees. Many boosting implementations let the user make predictions for any number of trees less than what was originally fit (1000 in this example). This "sub-model trick" can greatly speed up the training time for many models (e.g. see this example in the caret documentation).
机器学习的有一种算法为集成学习，比如随机森林。随机森林是由许多的决策树所组成的，每个决策树可以单独进行计算，然后再进行汇总，这种情况下可以用并行计算加快速度。
In order to know what models allow this, the parsnip package contains a multi_predict() function that enables this feature. Printing the S3 methods for it lists the possible models:
我们可以用multi_predict()函数查看parsnip包里面有哪些函数支持这种加速
library(tidymodels)
methods("multi_predict")
# There are arguments for the parameter(s) that can create multiple predictions.
# For xgboost, `trees` are cheap to evaluate:
parsnip:::multi_predict._xgb.Booster %>%
formals() %>%
names()
The same feature does not exist for recipes though.
recipes的过程不支持并行计算
Expensive pre-processing
When tuning a recipe and a model, it makes sense to avoid recreating the recipe for each model.
For example, suppose that Isomap multi-dimensional scaling is used to pre-process the data prior to tuning a K-nearest neighbor regression:
比如，在进行K最近邻回归前，我们先对数据预处理，对其进行了降纬（Isomap是一种非线性降维方法。
）
data(Chicago)#数据
#数据预处理
iso_rec <-
recipe(ridership ~ ., data = Chicago) %>% #回归方程
step_dummy(all_nominal()) %>% #设定虚拟变量
step_isomap(all_predictors(), num_terms = tune())#数据姜维
#构建模型
knn_mod <-
nearest_neighbor(neighbors = tune(), weight_func = tune()) %>% #确定模型及超参数
set_engine("kknn") %>% #设定算法
set_mode("regression")#设定结果变量
With the following grid:
grid <-
parameters(num_terms(c(1, 9)), neighbors(), weight_func()) %>%
grid_regular(levels = c(5, 10, 7)) %>%
arrange(num_terms, neighbors, weight_func)
grid
To evaluate these r nrow(grid) candidate models, we would have to compute the same recipe 70 times per resample. Since Isomap is expensive, this is really inefficient.
tune_grid() determines when this occurs and fits all 70 candidate models for each unique configuration of the recipe. In essence, it nests the model parameters inside the unique parameters of the recipe:
alt_grid <- tidyr::nest(grid, data = c(-num_terms))
alt_grid
Only r nrow(alt_grid) recipes are prepared and, within each, all of the appropriate models are fit from the same recipe. In this example, once the recipe with num_terms = 1 is created, the model parameters are iteratively tuned:
alt_grid$data[[1]]
The same will be true for post-processing parameters being tuned. For each unique set of recipe and model parameters, the post-processing parameters will be evaluated without unnecessary re-fitting.
Also, when using a model formula, the model matrix is only created once per resample.
Parallel Processing
tune allows users, when possible, to use multiple cores or separate machines fit models. Currently, the package parallelizes the resampling loop of grid search^[There will be more options in the future to control this since the current implementation doesn't help for cases when many models are fit over a single resample of the data (such as a validation set).].
The foreach package is used to facilitate parallel computations. foreach can use a variety of technologies to share the computations and the choice of technology is determined by which parallel backend package is chosen (aka the do{technology} packages). For example, the doMC package uses the forking mechanism on *unix systems to split the computations across multiple cores. As of this writing, the backend packages are doFuture, doMC, doMPI, doParallel, doRedis, doRNG, doSNOW, and doAzureParallel (GitHub only). Of these, doFuture and doParallel are probably the most comprehensive and best supported.
Registering a parallel backend is also somewhat dependent of the package. For doParallel, one could use:
all_cores <- parallel::detectCores(logical = FALSE)
library(doParallel)
cl <- makePSOCKcluster(all_cores)
registerDoParallel(cl)
Other options exist in doParallel.
doFuture uses a function called plan to declare the method:
all_cores <- parallel::detectCores(logical = FALSE)
library(doFuture)
registerDoFuture()
cl <- makeCluster(all_cores)
plan(cluster, workers = cl)
One downside to parallel processing is that the different technologies handle inputs and outputs differently. For example, multicore forking tends to carry the loaded packages and objects into the worker processes. Others do not. To make sure that the correct packages are loaded (but not attached) in the workers is to use the pkg option in control_grid().
Some helpful advice to avoid errors in parallel processing is to not use variables in the global environment. These may not be found when the code is run inside of a worker process. For example:
num_pcs <- 3
recipe(mpg ~ ., data = mtcars) %>%
# Bad since num_pcs might not be found by a worker process
step_pca(all_predictors(), num_comp = num_pcs)
recipe(mpg ~ ., data = mtcars) %>%
# Good since the value is injected into the object
step_pca(all_predictors(), num_comp = !!num_pcs)
This issue is likely to occur if dplyr::one_of() is used as a sector.
Also note that almost all of the logging provided by tune_grid() will not be seen when running in parallel. Again, this is dependent on the backend package and technology being used.
