本文介绍了如何在 pandas 上通过group by来应用累积自定义聚合函数的处理方法,对大家解决问题具有一定的参考价值,需要的朋友们下面随着小编来一起学习吧!
问题描述
我有以下DataFrame
I have the following DataFrame
df = pd.DataFrame({'model': ['A0', 'A0', 'A1', 'A1','A0', 'A0', 'A1', 'A1', 'A0', 'A0', 'A1', 'A1'],
'y_true': [1, 2, 3, 3, 4, 5, 6, 7, 8, 9, 10, 11],
'y_pred': [0, 1, 5, 5, 7, 8, 8, 12, 8, 7, 14, 15],
'week': [1, 1, 1, 1, 2, 2, 2, 2, 3, 3, 3, 3]},
)
model y_true y_pred week
0 A0 1 0 1
1 A0 2 1 1
2 A1 3 5 1
3 A1 3 5 1
4 A0 4 7 2
5 A0 5 8 2
6 A1 6 8 2
7 A1 7 12 2
8 A0 8 8 3
9 A0 9 7 3
10 A1 10 14 3
11 A1 11 15 3
而且我想用sklearn进行一些指标演算,所以我做了这个功能
And I want to make some metrics calculus with sklearn, so I did this function
from sklearn.metrics import mean_absolute_error, mean_squared_error, explained_variance_score
import numpy as np
def metrics(df):
y_true=np.asarray(df['y_true'])
y_pred=np.asarray(df['y_pred'])
mae=mean_absolute_error(y_true, y_pred)
mse=mean_squared_error(y_true, y_pred)
evs=explained_variance_score(y_true, y_pred)
return mae,mse,evs
我试图以此方式组队
df.groupby(['model', 'week']).apply(metrics)
它返回每周的指标,但是我希望指标从第1周到其他几周是累积的。我的意思是:
It returns me the metrics for every week, but I want the metrics to be accumulative since week 1 to the other weeks. I mean:
1. For the results of week 1 I want the metrics of y_true and y_pred where the column week takes the value 1.
2. For the results of week 2 I want the metrics of y_true and y_pred where the column week takes the values 1 or 2
3. For the results of week 3 I want the metrics of y_true and y_pred where the column week takes the values 1, 2 or 3
这是部分解决方案,但不是我想要的。
A partial solution is this, but is not what I want.
y_true y_pred y_true_cum \
model week
A0 1 [1, 2] [0, 1] [1, 2]
2 [4, 5] [7, 8] [1, 2, 4, 5]
3 [8, 9] [8, 7] [1, 2, 4, 5, 8, 9]
A1 1 [3, 3] [5, 5] [1, 2, 4, 5, 8, 9, 3, 3]
2 [6, 7] [8, 12] [1, 2, 4, 5, 8, 9, 3, 3, 6, 7]
3 [10, 11] [14, 15] [1, 2, 4, 5, 8, 9, 3, 3, 6, 7, 10, 11]
我希望每个模型都有自己的累积周数:
I wanted every model to have his own accumulative weeks:
y_true y_pred y_true_cum \
model week
A0 1 [1, 2] [0, 1] [1, 2]
2 [4, 5] [7, 8] [1, 2, 4, 5]
3 [8, 9] [8, 7] [1, 2, 4, 5, 8, 9]
A1 1 [3, 3] [5, 5] [3, 3]
2 [6, 7] [8, 12] [ 3, 3, 6, 7]
3 [10, 11] [14, 15] [ 3, 3, 6, 7, 10, 11]
推荐答案
应该执行以下操作:
import pandas as pd
from sklearn.metrics import mean_absolute_error, mean_squared_error, explained_variance_score
df = pd.DataFrame({
'model': ['A0', 'A0', 'A1', 'A1','A0', 'A0', 'A1', 'A1', 'A0', 'A0', 'A1', 'A1'],
'week': [1, 1, 1, 1, 2, 2, 2, 2, 3, 3, 3, 3],
'y_true': [1, 2, 3, 3, 4, 5, 6, 7, 8, 9, 10, 11],
'y_pred': [0, 1, 5, 5, 7, 8, 8, 12, 8, 7, 14, 15]
})
def metrics(df):
df['mae'] = mean_absolute_error(df.y_true, df.y_pred)
df['mse'] = mean_squared_error(df.y_true, df.y_pred)
df['evs'] = explained_variance_score(df.y_true, df.y_pred)
return df
# groupby model, week and keep all values of y_true/y_pred as lists
df_group = df.groupby(['model', 'week']).agg(list)
# accumulate values for y_true and y_pred
df_group = df_group.groupby('model')['y_true', 'y_pred'].apply(lambda x: x.cumsum())
# apply metrics to new columns
df_group.apply(metrics, axis=1)
这篇关于如何在 pandas 上通过group by来应用累积自定义聚合函数的文章就介绍到这了,希望我们推荐的答案对大家有所帮助,也希望大家多多支持!