机器学习4个超参数搜索方法+代码

介绍

维基百科上说，“Hyperparameter optimization或tuning是为学习算法选择一组最优的hyperparameters的问题”。

超参数

内容

1. 传统的手工调参
2. 网格搜索
3. 随机搜索
4. 贝叶斯搜索

1. 传统手工搜索

#importing required librariesfrom sklearn.neighbors import KNeighborsClassifierfrom sklearn.model_selection import train_test_splitfrom sklearn.model_selection import KFold , cross_val_scorefrom sklearn.datasets import load_wine
wine = load_wine()X = wine.datay = wine.target
#splitting the data into train and test setX_train,X_test,y_train,y_test = train_test_split(X,y,test_size = 0.3,random_state = 14)
#declaring parameters gridk_value = list(range(2,11))algorithm = ['auto','ball_tree','kd_tree','brute']scores = []best_comb = []kfold = KFold(n_splits=5)
#hyperparameter tunningfor algo in algorithm:  for k in k_value:    knn = KNeighborsClassifier(n_neighbors=k,algorithm=algo)    results = cross_val_score(knn,X_train,y_train,cv = kfold)
    print(f'Score:{round(results.mean(),4)} with algo = {algo} , K = {k}')    scores.append(results.mean())    best_comb.append((k,algo))
best_param = best_comb[scores.index(max(scores))]print(f'\nThe Best Score : {max(scores)}')print(f"['algorithm': {best_param[1]} ,'n_neighbors': {best_param[0]}]")

1. 没办法确保得到最佳的参数组合。
2. 这是一个不断试错的过程，所以，非常的耗时。

2. 网格搜索

from sklearn.model_selection import GridSearchCV
knn = KNeighborsClassifier()grid_param = { 'n_neighbors' : list(range(2,11)) ,               'algorithm' : ['auto','ball_tree','kd_tree','brute'] }grid = GridSearchCV(knn,grid_param,cv = 5)grid.fit(X_train,y_train)
#best parameter combinationgrid.best_params_
#Score achieved with best parameter combinationgrid.best_score_
#all combinations of hyperparametersgrid.cv_results_['params']
#average scores of cross-validationgrid.cv_results_['mean_test_score']

3. 随机搜索

from sklearn.model_selection import RandomizedSearchCV
knn = KNeighborsClassifier()
grid_param = { 'n_neighbors' : list(range(2,11)) ,               'algorithm' : ['auto','ball_tree','kd_tree','brute'] }
rand_ser = RandomizedSearchCV(knn,grid_param,n_iter=10)rand_ser.fit(X_train,y_train)
#best parameter combinationrand_ser.best_params_
#score achieved with best parameter combinationrand_ser.best_score_
#all combinations of hyperparametersrand_ser.cv_results_['params']
#average scores of cross-validationrand_ser.cv_results_['mean_test_score']

4. 贝叶斯搜索

1. 使用先前评估的点X1*:n*，计算损失f的后验期望。

2. 在新的点X的抽样损失f，从而最大化f的期望的某些方法。该方法指定f域的哪些区域最适于抽样。

安装: pip install scikit-optimize

from skopt import BayesSearchCV
import warningswarnings.filterwarnings("ignore")
# parameter ranges are specified by one of belowfrom skopt.space import Real, Categorical, Integer
knn = KNeighborsClassifier()#defining hyper-parameter gridgrid_param = { 'n_neighbors' : list(range(2,11)) ,               'algorithm' : ['auto','ball_tree','kd_tree','brute'] }
#initializing Bayesian SearchBayes = BayesSearchCV(knn , grid_param , n_iter=30 , random_state=14)Bayes.fit(X_train,y_train)
#best parameter combinationBayes.best_params_
#score achieved with best parameter combinationBayes.best_score_
#all combinations of hyperparametersBayes.cv_results_['params']
#average scores of cross-validationBayes.cv_results_['mean_test_score']

安装: pip install bayesian-optimization

总结

点个 在看就是最大的支持