sklearn 分类变量转换
在用进行机器学习模型建立时,我们拿到的数据总是包含有分类变量和数值型变量,但是sklearn模型要求的输入都是数值型的。因此,在构建模型之前,我们需要对分类变量进行转换。
分类变量的类型
分类变量一般分为无序分类变量和有序分类变量,对应的,在sklearn中,我们需要使用OnehotEncoder和OrdinalEncoder
python
from sklearn.preprocessing import OneHotEncoder
enc = OneHotEncoder()
X = [['Male', 'US'], ['Female', 'CN'], ['Female', 'UK']]
enc.fit(X)
python
from sklearn.preprocessing import OrdinalEncoder
enc = OrdinalEncoder()
X = [['Male', 1], ['Female', 3], ['Female', 2]]
enc.fit(X)
复杂情况
有时我们的数据变量情况很复杂,是多种变量类型的组合,如:
无序分类变量和数值型变量
python
from sklearn.preprocessing import StandardScaler, OneHotEncoder
from sklearn.compose import make_column_transformer
from sklearn.compose import make_column_selector
import numpy as np
import pandas as pd
X = pd.DataFrame({'city': ['London', 'London', 'Paris', 'Sallisaw'],
'rating': [5, 3, 4, 5]})
ct = make_column_transformer(
(StandardScaler(),
make_column_selector(dtype_include=np.number)), # rating,数值型变量,用StandardScaler处理
(OneHotEncoder(),
make_column_selector(dtype_include=object))) # city,无序的分类变量,用OneHotEncoder处理
ct.fit_transform(X)
无序分类变量,有序分类变量和数值型变量
python
from sklearn.preprocessing import OneHotEncoder, OrdinalEncoder
from sklearn.compose import ColumnTransformer
import numpy as np
import pandas as pd
X = pd.DataFrame({'city': ['London', 'London', 'Paris', 'Sallisaw'],
'Frequency': ['No', 'No', 'Sometimes','Always'],
'rating': [5, 3, 4, 5]})
ct = ColumnTransformer(
[("onehot", OneHotEncoder(), ['city']), # city, 无序分类变量,用OneHotEncoder处理
("ordinal", OrdinalEncoder(categories=['No', 'Sometimes', 'Always']), ['Frequency'])], # Frequency,有序分类变量,用OrdinalEncoder处理,并设置顺序categories=['No', 'Sometimes', 'Always']
remainder='passthrough') # remainder,剩余的变量保存不变
ct.fit_transform(X)