Titanic 沉没
参见：https://github.com/lijingpeng/kaggle
这是一个分类任务，特征包含离散特征和连续特征，数据如下：Kaggle地址。目标是根据数据特征预测一个人是否能在泰坦尼克的沉没事故中存活下来。接下来解释下数据的格式：
survival 目标列，是否存活，1代表存活 (0 = No; 1 = Yes)
pclass 乘坐的舱位级别 (1 = 1st; 2 = 2nd; 3 = 3rd)
name 姓名
sex 性别
age 年龄
sibsp 兄弟姐妹的数量（乘客中）
parch 父母的数量（乘客中）
ticket 票号
fare 票价
cabin 客舱
embarked 登船的港口
(C = Cherbourg; Q = Queenstown; S = Southampton)
载入数据并分析
# -*- coding: UTF-8 -*-
%matplotlib inline
import pandas as pd
import string
import numpy as np
import matplotlib.pyplot as plt
from sklearn import preprocessing
train = pd.read_csv('train.csv')
test = pd.read_csv('test.csv')
def substrings_in_string(big_string, substrings):
for substring in substrings:
if string.find(big_string, substring) != -1:
return substring
return np.nan
def replace_titles(x):
title=x['Title']
if title in ['Mr','Don', 'Major', 'Capt', 'Jonkheer', 'Rev', 'Col']:
return 'Mr'
elif title in ['Master']:
return 'Master'
elif title in ['Countess', 'Mme','Mrs']:
return 'Mrs'
elif title in ['Mlle', 'Ms','Miss']:
return 'Miss'
elif title =='Dr':
if x['Sex']=='Male':
return 'Mr'
else:
return 'Mrs'
elif title =='':
if x['Sex']=='Male':
return 'Master'
else:
return 'Miss'
else:
return title
title_list = ['Mrs', 'Mr', 'Master', 'Miss', 'Major', 'Rev',
'Dr', 'Ms', 'Mlle','Col', 'Capt', 'Mme', 'Countess',
'Don', 'Jonkheer']
label = train['Survived'] # 目标列
Pclass、Sex、Embarked离散特征数据预览
除此之外Name、Ticket、Cabin也是离散特征，我们暂时不用这几个特征，直观上来讲，叫什么名字跟在事故中是否存活好像没有太大的联系。
# 接下来我们对每个特征进行一下分析：
train.groupby(['Pclass'])['PassengerId'].count().plot(kind='bar')
<matplotlib.axes.AxesSubplot at 0x102bef590>
train.groupby(['SibSp'])['PassengerId'].count().plot(kind='bar')
<matplotlib.axes.AxesSubplot at 0x106c41a10>
train.groupby(['Parch'])['PassengerId'].count().plot(kind='bar')
<matplotlib.axes.AxesSubplot at 0x106d7b090>
train.groupby(['Embarked'])['PassengerId'].count().plot(kind='bar')
<matplotlib.axes.AxesSubplot at 0x106eca590>
train.groupby(['Sex'])['PassengerId'].count().plot(kind='bar')
<matplotlib.axes.AxesSubplot at 0x106ff83d0>
连续特征处理
Age、Fare是连续特征，观察数据分布查看是否有缺失值和异常值，我们看到Age中存在缺失值，我们考虑使用均值来填充缺失值。
print '检测是否有缺失值：'
print train[train['Age'].isnull()]['Age'].head()
print train[train['Fare'].isnull()]['Fare'].head()
print train[train['SibSp'].isnull()]['SibSp'].head()
print train[train['Parch'].isnull()]['Parch'].head()
train['Age'] = train['Age'].fillna(train['Age'].mean())
print '填充之后再检测：'
print train[train['Age'].isnull()]['Age'].head()
print train[train['Fare'].isnull()]['Fare'].head()
检测是否有缺失值：
5 NaN
17 NaN
19 NaN
26 NaN
28 NaN
Name: Age, dtype: float64
Series([], Name: Fare, dtype: float64)
Series([], Name: SibSp, dtype: int64)
Series([], Name: Parch, dtype: int64)
填充之后再检测：
Series([], Name: Age, dtype: float64)
Series([], Name: Fare, dtype: float64)
print '检测测试集是否有缺失值：'
print test[test['Age'].isnull()]['Age'].head()
print test[test['Fare'].isnull()]['Fare'].head()
print test[test['SibSp'].isnull()]['SibSp'].head()
print test[test['Parch'].isnull()]['Parch'].head()
test['Age'] = test['Age'].fillna(test['Age'].mean())
test['Fare'] = test['Fare'].fillna(test['Fare'].mean())
print '填充之后再检测：'
print test[test['Age'].isnull()]['Age'].head()
print test[test['Fare'].isnull()]['Fare'].head()
检测测试集是否有缺失值：
10 NaN
22 NaN
29 NaN
33 NaN
36 NaN
Name: Age, dtype: float64
152 NaN
Name: Fare, dtype: float64
Series([], Name: SibSp, dtype: int64)
Series([], Name: Parch, dtype: int64)
填充之后再检测：
Series([], Name: Age, dtype: float64)
Series([], Name: Fare, dtype: float64)
# 处理Title特征
train['Title'] = train['Name'].map(lambda x: substrings_in_string(x, title_list))
test['Title'] = test['Name'].map(lambda x: substrings_in_string(x, title_list))
train['Title'] = train.apply(replace_titles, axis=1)
test['Title'] = test.apply(replace_titles, axis=1)
# family特征
train['Family_Size'] = train['SibSp'] + train['Parch']
train['Family'] = train['SibSp'] * train['Parch']
test['Family_Size'] = test['SibSp'] + test['Parch']
test['Family'] = test['SibSp'] * test['Parch']
train['AgeFill'] = train['Age']
mean_ages = np.zeros(4)
mean_ages[0] = np.average(train[train['Title'] == 'Miss']['Age'].dropna())
mean_ages[1] = np.average(train[train['Title'] == 'Mrs']['Age'].dropna())
mean_ages[2] = np.average(train[train['Title'] == 'Mr']['Age'].dropna())
mean_ages[3] = np.average(train[train['Title'] == 'Master']['Age'].dropna())
train.loc[ (train.Age.isnull()) & (train.Title == 'Miss') ,'AgeFill'] = mean_ages[0]
train.loc[ (train.Age.isnull()) & (train.Title == 'Mrs') ,'AgeFill'] = mean_ages[1]
train.loc[ (train.Age.isnull()) & (train.Title == 'Mr') ,'AgeFill'] = mean_ages[2]
train.loc[ (train.Age.isnull()) & (train.Title == 'Master') ,'AgeFill'] = mean_ages[3]
train['AgeCat'] = train['AgeFill']
train.loc[ (train.AgeFill<=10), 'AgeCat'] = 'child'
train.loc[ (train.AgeFill>60), 'AgeCat'] = 'aged'
train.loc[ (train.AgeFill>10) & (train.AgeFill <=30) ,'AgeCat'] = 'adult'
train.loc[ (train.AgeFill>30) & (train.AgeFill <=60) ,'AgeCat'] = 'senior'
train['Fare_Per_Person'] = train['Fare'] / (train['Family_Size'] + 1)
test['AgeFill'] = test['Age']
mean_ages = np.zeros(4)
mean_ages[0] = np.average(test[test['Title'] == 'Miss']['Age'].dropna())
mean_ages[1] = np.average(test[test['Title'] == 'Mrs']['Age'].dropna())
mean_ages[2] = np.average(test[test['Title'] == 'Mr']['Age'].dropna())
mean_ages[3] = np.average(test[test['Title'] == 'Master']['Age'].dropna())
test.loc[ (test.Age.isnull()) & (test.Title == 'Miss') ,'AgeFill'] = mean_ages[0]
test.loc[ (test.Age.isnull()) & (test.Title == 'Mrs') ,'AgeFill'] = mean_ages[1]
test.loc[ (test.Age.isnull()) & (test.Title == 'Mr') ,'AgeFill'] = mean_ages[2]
test.loc[ (test.Age.isnull()) & (test.Title == 'Master') ,'AgeFill'] = mean_ages[3]
test['AgeCat'] = test['AgeFill']
test.loc[ (test.AgeFill<=10), 'AgeCat'] = 'child'
test.loc[ (test.AgeFill>60), 'AgeCat'] = 'aged'
test.loc[ (test.AgeFill>10) & (test.AgeFill <=30) ,'AgeCat'] = 'adult'
test.loc[ (test.AgeFill>30) & (test.AgeFill <=60) ,'AgeCat'] = 'senior'
test['Fare_Per_Person'] = test['Fare'] / (test['Family_Size'] + 1)
train.Embarked = train.Embarked.fillna('S')
test.Embarked = test.Embarked.fillna('S')
train.loc[ train.Cabin.isnull() == True, 'Cabin'] = 0.2
train.loc[ train.Cabin.isnull() == False, 'Cabin'] = 1
test.loc[ test.Cabin.isnull() == True, 'Cabin'] = 0.2
test.loc[ test.Cabin.isnull() == False, 'Cabin'] = 1
#Age times class
train['AgeClass'] = train['AgeFill'] * train['Pclass']
train['ClassFare'] = train['Pclass'] * train['Fare_Per_Person']
train['HighLow'] = train['Pclass']
train.loc[ (train.Fare_Per_Person < 8) ,'HighLow'] = 'Low'
train.loc[ (train.Fare_Per_Person >= 8) ,'HighLow'] = 'High'
#Age times class
test['AgeClass'] = test['AgeFill'] * test['Pclass']
test['ClassFare'] = test['Pclass'] * test['Fare_Per_Person']
test['HighLow'] = test['Pclass']
test.loc[ (test.Fare_Per_Person < 8) ,'HighLow'] = 'Low'
test.loc[ (test.Fare_Per_Person >= 8) ,'HighLow'] = 'High'
print train.head(1)
# print test.head()
PassengerId Survived Pclass Name Sex Age SibSp \
0 1 0 3 Braund, Mr. Owen Harris male 22.0 1
Parch Ticket Fare ... Embarked Title Family_Size Family \
0 0 A/5 21171 7.25 ... S Mr 1 0
AgeFill AgeCat Fare_Per_Person AgeClass ClassFare HighLow
0 22.0 adult 3.625 66.0 10.875 Low
[1 rows x 21 columns]
特征工程
# 处理训练集
Pclass = pd.get_dummies(train.Pclass)
Sex = pd.get_dummies(train.Sex)
Embarked = pd.get_dummies(train.Embarked)
Title = pd.get_dummies(train.Title)
AgeCat = pd.get_dummies(train.AgeCat)
HighLow = pd.get_dummies(train.HighLow)
train_data = pd.concat([Pclass, Sex, Embarked, Title, AgeCat, HighLow], axis=1)
train_data['Age'] = train['Age']
train_data['Fare'] = train['Fare']
train_data['SibSp'] = train['SibSp']
train_data['Parch'] = train['Parch']
train_data['Family_Size'] = train['Family_Size']
train_data['Family'] = train['Family']
train_data['AgeFill'] = train['AgeFill']
train_data['Fare_Per_Person'] = train['Fare_Per_Person']
train_data['Cabin'] = train['Cabin']
train_data['AgeClass'] = train['AgeClass']
train_data['ClassFare'] = train['ClassFare']
cols = ['Age', 'Fare', 'SibSp', 'Parch', 'Family_Size', 'Family', 'AgeFill', 'Fare_Per_Person', 'AgeClass', 'ClassFare']
train_data[cols] = train_data[cols].apply(lambda x: (x - np.mean(x)) / (np.max(x) - np.min(x)))
print train_data.head()
# 处理测试集
Pclass = pd.get_dummies(test.Pclass)
Sex = pd.get_dummies(test.Sex)
Embarked = pd.get_dummies(test.Embarked)
Title = pd.get_dummies(test.Title)
AgeCat = pd.get_dummies(test.AgeCat)
HighLow = pd.get_dummies(test.HighLow)
test_data = pd.concat([Pclass, Sex, Embarked, Title, AgeCat, HighLow], axis=1)
test_data['Age'] = test['Age']
test_data['Fare'] = test['Fare']
test_data['SibSp'] = test['SibSp']
test_data['Parch'] = test['Parch']
test_data['Family_Size'] = test['Family_Size']
test_data['Family'] = test['Family']
test_data['AgeFill'] = test['AgeFill']
test_data['Fare_Per_Person'] = test['Fare_Per_Person']
test_data['Cabin'] = test['Cabin']
test_data['AgeClass'] = test['AgeClass']
test_data['ClassFare'] = test['ClassFare']
test_data[cols] = test_data[cols].apply(lambda x: (x - np.mean(x)) / (np.max(x) - np.min(x)))
print test_data.head()
1 2 3 female male C Q S Master Miss ... \
0 0.0 0.0 1.0 0.0 1.0 0.0 0.0 1.0 0.0 0.0 ...
1 1.0 0.0 0.0 1.0 0.0 1.0 0.0 0.0 0.0 0.0 ...
2 0.0 0.0 1.0 1.0 0.0 0.0 0.0 1.0 0.0 1.0 ...
3 1.0 0.0 0.0 1.0 0.0 0.0 0.0 1.0 0.0 0.0 ...
4 0.0 0.0 1.0 0.0 1.0 0.0 0.0 1.0 0.0 0.0 ...
Fare SibSp Parch Family_Size Family AgeFill \
0 -0.048707 0.059624 -0.063599 0.00954 -0.035494 -0.096747
1 0.076277 0.059624 -0.063599 0.00954 -0.035494 0.104309
2 -0.047390 -0.065376 -0.063599 -0.09046 -0.035494 -0.046483
3 0.040786 0.059624 -0.063599 0.00954 -0.035494 0.066611
4 -0.047146 -0.065376 -0.063599 -0.09046 -0.035494 0.066611
Fare_Per_Person Cabin AgeClass ClassFare
0 -0.031799 1 0.004673 -0.040180
1 0.030694 1 -0.121978 0.008161
2 -0.023406 1 0.058952 -0.015001
3 0.012948 1 -0.135547 -0.009584
4 -0.023162 1 0.181080 -0.014269
[5 rows x 29 columns]
1 2 3 female male C Q S Master Miss ... \
0 0.0 0.0 1.0 0.0 1.0 0.0 1.0 0.0 0.0 0.0 ...
1 0.0 0.0 1.0 1.0 0.0 0.0 0.0 1.0 0.0 0.0 ...
2 0.0 1.0 0.0 0.0 1.0 0.0 1.0 0.0 0.0 0.0 ...
3 0.0 0.0 1.0 0.0 1.0 0.0 0.0 1.0 0.0 0.0 ...
4 0.0 0.0 1.0 1.0 0.0 0.0 0.0 1.0 0.0 0.0 ...
Fare SibSp Parch Family_Size Family AgeFill \
0 -0.054258 -0.055921 -0.043594 -0.083971 -0.027811 0.055749
1 -0.055877 0.069079 -0.043594 0.016029 -0.027811 0.220591
2 -0.050631 -0.055921 -0.043594 -0.083971 -0.027811 0.418402
3 -0.052632 -0.055921 -0.043594 -0.083971 -0.027811 -0.043157
4 -0.045556 0.069079 0.067517 0.116029 0.034689 -0.109094
Fare_Per_Person Cabin AgeClass ClassFare
0 -0.053389 1 0.218758 -0.037167
1 -0.069889 1 0.425952 -0.086667
2 -0.046307 1 0.332024 -0.052842
3 -0.050213 1 0.094442 -0.027639
4 -0.067618 1 0.011564 -0.079855
[5 rows x 29 columns]
模型训练
from sklearn.linear_model import LogisticRegression as LR
from sklearn.cross_validation import cross_val_score
from sklearn.naive_bayes import GaussianNB as GNB
from sklearn.ensemble import RandomForestClassifier
import numpy as np
逻辑回归
model_lr = LR(penalty = 'l2', dual = True, random_state = 0)
model_lr.fit(train_data, label)
print "逻辑回归10折交叉验证得分: ", np.mean(cross_val_score(model_lr, train_data, label, cv=10, scoring='roc_auc'))
result = model_lr.predict( test_data )
output = pd.DataFrame( data={"PassengerId":test["PassengerId"], "Survived":result} )
output.to_csv( "lr.csv", index=False, quoting=3 )
逻辑回归10折交叉验证得分: 0.871878335172
提交kaggle后准确率：0.78469
高斯贝叶斯
model_GNB = GNB()
model_GNB.fit(train_data, label)
print "高斯贝叶斯分类器10折交叉验证得分: ", np.mean(cross_val_score(model_GNB, train_data, label, cv=10, scoring='roc_auc'))
result = model_GNB.predict( test_data )
output = pd.DataFrame( data={"PassengerId":test["PassengerId"], "Survived":result} )
output.to_csv( "gnb.csv", index=False, quoting=3 )
高斯贝叶斯分类器10折交叉验证得分: 0.857323798206
提交kaggle后准确率：0.74163
随机森林
forest = RandomForestClassifier( n_estimators=500, criterion='entropy', max_depth=5, min_samples_split=1,
min_samples_leaf=1, max_features='auto', bootstrap=False, oob_score=False, n_jobs=4,
verbose=0)
%time forest = forest.fit( train_data, label )
print "随机森林分类器10折交叉验证得分: ", np.mean(cross_val_score(forest, train_data, label, cv=10, scoring='roc_auc'))
result = forest.predict( test_data )
output = pd.DataFrame( data={"PassengerId":test["PassengerId"], "Survived":result} )
output.to_csv( "rf.csv", index=False, quoting=3 )
CPU times: user 1.34 s, sys: 208 ms, total: 1.55 s
Wall time: 1.17 s
随机森林分类器10折交叉验证得分: 0.870820473644
提交kaggle后准确率：0.76555
寻找最佳参数
from sklearn.pipeline import Pipeline
from sklearn.grid_search import GridSearchCV
from sklearn.cross_validation import train_test_split,StratifiedShuffleSplit,StratifiedKFold
param_grid = dict( )
pipeline=Pipeline([ ('clf', forest) ])
grid_search = GridSearchCV(pipeline, param_grid=param_grid, verbose=3, scoring='accuracy',
cv=StratifiedShuffleSplit(label, n_iter=10, test_size=0.2, train_size=None)).fit(train_data, label)
print("Best score: %0.3f" % grid_search.best_score_)
Fitting 10 folds for each of 1 candidates, totalling 10 fits
[CV] ................................................................
[CV] ....................................... , score=0.849162 - 1.7s
[CV] ................................................................
[CV] ....................................... , score=0.843575 - 1.5s
[CV] ................................................................
[CV] ....................................... , score=0.804469 - 1.4s
[CV] ................................................................
[CV] ....................................... , score=0.804469 - 1.9s
[CV] ................................................................
[CV] ....................................... , score=0.871508 - 2.1s
[CV] ................................................................
[CV] ....................................... , score=0.865922 - 1.9s
[CV] ................................................................
[CV] ....................................... , score=0.854749 - 1.8s
[CV] ................................................................
[CV] ....................................... , score=0.860335 - 1.7s
[CV] ................................................................
[CV] ....................................... , score=0.843575 - 1.6s
[CV] ................................................................
[CV] ....................................... , score=0.826816 - 1.5s
[Parallel(n_jobs=1)]: Done 10 out of 10 | elapsed: 17.1s finished
Best score: 0.842