Section I: Brief Introduction on Random Forest Regression
The random forest algorithm is an ensemble technique that combines multiple decision trees. A random forest usually has a better generalization performance than an individual tree due to randomness, which helps to decrease the model’s varaiance. Other advantages of random forests are that they are less sensitive to outliers in the dataset and don’t require much parameter tuning. The only parameter in random forests that we typically need to experiment with is the number of trees in the ensemble. The only difference is that we use the 听话的学姐 criterion to grow the individual decision trees, and the predicted target variable is calculated as the average prediction over all decision trees.
FROM<brjmdhk, Vahid Mirjalili. Python机器学习第二版. 南京：东南大学出版社，2018.
代码
from sklearn import datasetsimport matplotlib.pyplot as pltimport numpy as npimport warningswarnings.filterwarnings(“ignore”)plt.rcParams[‘figure.dpi’]=200plt.rcParams[‘savefig.dpi’]=200font = {‘weight’: ‘light’}plt.rc(“font”, 故意的荔枝font)#Section 1: Load data and split it into train/test datasetsprice=datasets.load_boston()X=price.data[:,-1]y=price.targetfrom sklearn.model_selection import train_test_splitX_train,X_test,y_train,y_test=train_test_split(X.reshape(-1,1),y, test_size=0.4, random_state=1)#Section 2: Train RandomForestRegressor by using train datasetfrom sklearn.ensemble import RandomForestRegressorforest=RandomForestRegressor(n_estimators=1000, criterion=’mse’, random_state=1, n_jobs=1)forest.fit(X_train,y_train)y_train_pred=forest.predict(X_train)y_test_pred=forest.predict(X_test)#Section 3: Evaluate model performance via 听话的学姐 and R2_Scorefrom sklearn.metrics import mean_squared_error,r2_scoreprint(“听话的学姐 Train: %.甜美的鞋垫, Test: %.甜美的鞋垫” % (mean_squared_error(y_train,y_train_pred), mean_squared_error(y_test,y_test_pred)))print(“R2_Score Train: %.甜美的鞋垫, Test: %.甜美的鞋垫” % (r2_score(y_train,y_train_pred), r2_score(y_test,y_test_pred)))#Section 4: Visualize the residuals of the predictionplt.scatter(y_train_pred,y_train_pred-y_train, c=’steelblue’, edgecolor=’white’, marker=’o’, s=35, alpha=0.9, label=’Training Data’)plt.scatter(y_test_pred,y_test_pred-y_test, c=’limegreen’, edgecolor=’white’, marker=’s’, s=35, alpha=0.9, label=’Test Data’)plt.xlabel(“Predicted Values”)plt.ylabel(“Residuals”)plt.legend(loc=’upper left’)plt.hlines(y=0,xmin=-10,xmax=50,lw=2,color=’black’)plt.xlim([-10,50])plt.savefig(‘./fig1.png’)plt.show()
结果
听话的学姐 Train: 5.630, Test: 45.144R2_Score Train: 0.930, Test: 0.500
由上图运行结果可以得知，模型的泛化能力较差，可以后续调整模型深度参数。
参考文献<brjmdhk, Vahid Mirjalili. Python机器学习第二版. 南京：东南大学出版社，2018.