本文介绍了Scikit学习OneHotEncoder拟合和变换错误:ValueError:X的形状与拟合期间不同的处理方法，对大家解决问题具有一定的参考价值，需要的朋友们下面随着小编来一起学习吧！
问题描述
下面是我的代码.
我知道为什么在转换过程中会发生错误.这是因为在拟合和变换过程中要素列表不匹配.我该如何解决?我如何才能将其余所有功能都设为0?
I know why the error is occurring during transform. It is because of the feature list mismatch during fit and transform.How can i solve this? How can i get 0 for all the rest features?
在此之后，我想将其用于SGD分类器的局部拟合.
After this i want to use this for partial fit of SGD classifier.
Jupyter QtConsole 4.3.1
Python 3.6.2 |Anaconda custom (64-bit)| (default, Sep 21 2017, 18:29:43)
Type 'copyright', 'credits' or 'license' for more information
IPython 6.1.0 -- An enhanced Interactive Python. Type '?' for help.
import pandas as pd
from sklearn.preprocessing import OneHotEncoder
input_df = pd.DataFrame(dict(fruit=['Apple', 'Orange', 'Pine'],
color=['Red', 'Orange','Green'],
is_sweet = [0,0,1],
country=['USA','India','Asia']))
input_df
Out[1]:
color country fruit is_sweet
0 Red USA Apple 0
1 Orange India Orange 0
2 Green Asia Pine 1
filtered_df = input_df.apply(pd.to_numeric, errors='ignore')
filtered_df.info()
# apply one hot encode
refreshed_df = pd.get_dummies(filtered_df)
refreshed_df
<class 'pandas.core.frame.DataFrame'>
RangeIndex: 3 entries, 0 to 2
Data columns (total 4 columns):
color 3 non-null object
country 3 non-null object
fruit 3 non-null object
is_sweet 3 non-null int64
dtypes: int64(1), object(3)
memory usage: 176.0+ bytes
Out[2]:
is_sweet color_Green color_Orange color_Red country_Asia \
0 0 0 0 1 0
1 0 0 1 0 0
2 1 1 0 0 1
country_India country_USA fruit_Apple fruit_Orange fruit_Pine
0 0 1 1 0 0
1 1 0 0 1 0
2 0 0 0 0 1
enc = OneHotEncoder()
enc.fit(refreshed_df)
Out[3]:
OneHotEncoder(categorical_features='all', dtype=<class 'numpy.float64'>,
handle_unknown='error', n_values='auto', sparse=True)
new_df = pd.DataFrame(dict(fruit=['Apple'],
color=['Red'],
is_sweet = [0],
country=['USA']))
new_df
Out[4]:
color country fruit is_sweet
0 Red USA Apple 0
filtered_df1 = new_df.apply(pd.to_numeric, errors='ignore')
filtered_df1.info()
# apply one hot encode
refreshed_df1 = pd.get_dummies(filtered_df1)
refreshed_df1
<class 'pandas.core.frame.DataFrame'>
RangeIndex: 1 entries, 0 to 0
Data columns (total 4 columns):
color 1 non-null object
country 1 non-null object
fruit 1 non-null object
is_sweet 1 non-null int64
dtypes: int64(1), object(3)
memory usage: 112.0+ bytes
Out[5]:
is_sweet color_Red country_USA fruit_Apple
0 0 1 1 1
enc.transform(refreshed_df1)
---------------------------------------------------------------------------
ValueError Traceback (most recent call last)
<ipython-input-6-33a6a884ba3f> in <module>()
----> 1 enc.transform(refreshed_df1)
~/anaconda3/lib/python3.6/site-packages/sklearn/preprocessing/data.py in transform(self, X)
2073 """
2074 return _transform_selected(X, self._transform,
-> 2075 self.categorical_features, copy=True)
2076
2077
~/anaconda3/lib/python3.6/site-packages/sklearn/preprocessing/data.py in _transform_selected(X, transform, selected, copy)
1810
1811 if isinstance(selected, six.string_types) and selected == "all":
-> 1812 return transform(X)
1813
1814 if len(selected) == 0:
~/anaconda3/lib/python3.6/site-packages/sklearn/preprocessing/data.py in _transform(self, X)
2030 raise ValueError("X has different shape than during fitting."
2031 " Expected %d, got %d."
-> 2032 % (indices.shape[0] - 1, n_features))
2033
2034 # We use only those categorical features of X that are known using fit.
ValueError: X has different shape than during fitting. Expected 10, got 4.推荐答案
而不是使用pd.get_dummies()，您需要 LabelEncoder + OneHotEncoder，可以存储原始值，然后在新数据上使用它们.
Instead of using pd.get_dummies() you need LabelEncoder + OneHotEncoder which can store the original values and then use them on the new data.
按如下所示更改代码将为您提供所需的结果.
Changing your code like below will give you required results.
import pandas as pd
from sklearn.preprocessing import OneHotEncoder, LabelEncoder
input_df = pd.DataFrame(dict(fruit=['Apple', 'Orange', 'Pine'],
color=['Red', 'Orange','Green'],
is_sweet = [0,0,1],
country=['USA','India','Asia']))
filtered_df = input_df.apply(pd.to_numeric, errors='ignore')
# This is what you need
le_dict = {}
for col in filtered_df.columns:
le_dict[col] = LabelEncoder().fit(filtered_df[col])
filtered_df[col] = le_dict[col].transform(filtered_df[col])
enc = OneHotEncoder()
enc.fit(filtered_df)
refreshed_df = enc.transform(filtered_df).toarray()
new_df = pd.DataFrame(dict(fruit=['Apple'],
color=['Red'],
is_sweet = [0],
country=['USA']))
for col in new_df.columns:
new_df[col] = le_dict[col].transform(new_df[col])
new_refreshed_df = enc.transform(new_df).toarray()
print(filtered_df)
color country fruit is_sweet
0 2 2 0 0
1 1 1 1 0
2 0 0 2 1
print(refreshed_df)
[[ 0. 0. 1. 0. 0. 1. 1. 0. 0. 1. 0.]
[ 0. 1. 0. 0. 1. 0. 0. 1. 0. 1. 0.]
[ 1. 0. 0. 1. 0. 0. 0. 0. 1. 0. 1.]]
print(new_df)
color country fruit is_sweet
0 2 2 0 0
print(new_refreshed_df)
[[ 0. 0. 1. 0. 0. 1. 1. 0. 0. 1. 0.]]
这篇关于Scikit学习OneHotEncoder拟合和变换错误:ValueError:X的形状与拟合期间不同的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持！