本文是个人学习笔记，内容主要涉及树的集成模型随机森林(RandomForest)和梯度提升树(GradientBoostingDecisionTree)对titanic数据集进行二类分类。
集成模型就是综合考量多个分类器的预测结果，再作出决策。“综合考量”的方式大体分2种：（1）按一定次序搭建多个分类模型，后续模型的加入要对现有集成模型的性能有所贡献，从而不断提升更新后的集成模型性能。在每一棵数生成过程尽可能降低整体集成模型在训练集上的拟合误差。（2）对相同数据进行随机样本采样和随机特征采样，同时搭建多个独立的分类模型，最后根据多数投票表决分类。工业界为了追求更加强劲的预测性能，经常使用随机森林分类模型作为Baseline System。
import pandas as pd titanic=pd.read_csv('http://biostat.mc.vanderbilt.edu/wiki/pub/Main/DataSets/titanic.txt') titanic.head() titanic.info() X=titanic[['pclass','age','sex']] y=titanic['survived'] X['age'].fillna(X['age'].mean(),inplace=True) from distutils.version import LooseVersion as Version from sklearn import __version__ as sklearn_version from sklearn import datasets if Version(sklearn_version) < '0.18': from sklearn.cross_validation import train_test_split else: from sklearn.model_selection import train_test_split X_train,X_test,y_train,y_test = train_test_split(X, y, test_size=0.25, random_state=33) from sklearn.feature_extraction import DictVectorizer #特征转换器:将类别型特征独热编码 vec=DictVectorizer(sparse=False) X_train=vec.fit_transform(pd.DataFrame(X_train).to_dict(orient='record')) X_test=vec.transform(pd.DataFrame(X_test).to_dict(orient='record')) from sklearn.tree import DecisionTreeClassifier dtc=DecisionTreeClassifier() dtc.fit(X_train,y_train) dtc_y_pred=dtc.predict(X_test) from sklearn.ensemble import RandomForestClassifier rfc=RandomForestClassifier() rfc.fit(X_train,y_train) rfc_y_pred=rfc.predict(X_test) from sklearn.ensemble import GradientBoostingClassifier gbc=GradientBoostingClassifier() gbc.fit(X_train,y_train) gbc_y_pred=gbc.predict(X_test) from sklearn.metrics import classification_report print('The accuracy of decision tree is',dtc.score(X_test,y_test)) print(classification_report(dtc_y_pred,y_test)) print('The accuracy of random forest classifier is',rfc.score(X_test,y_test)) print(classification_report(rfc_y_pred,y_test)) print('The accuracy of gradient tree boosting is',gbc.score(X_test,y_test)) print(classification_report(gbc_y_pred,y_test))