问题描述
使用pandas v1.0.1 和numpy 1.18.1,我想计算时间序列上具有不同窗口大小的滚动均值和标准差.在我正在处理的数据中,某些后续点的值可以是恒定的,这样 - 取决于窗口大小 - 滚动平均值可能等于窗口中的所有值,并且相应的 std 预计为 0.
Using pandas v1.0.1 and numpy 1.18.1, I want to calculate the rolling mean and std with different window sizes on a time series. In the data I am working with, the values can be constant for some subsequent points such that - depending on the window size - the rolling mean might be equal to all the values in the window and the corresponding std is expected to be 0.
但是,根据窗口大小,我看到使用相同 df 的不同行为.
However, I see a different behavior using the same df depending on the window size.
MWE:
for window in [3,5]:
values = [1234.0, 4567.0, 6800.0, 6810.0, 6821.0, 6820.0, 6820.0, 6820.0, 6820.0, 6820.0, 6820.0]
df = pd.DataFrame(values, columns=['values'])
df.loc[:, 'mean'] = df.rolling(window, min_periods=1).mean()
df.loc[:, 'std'] = df.rolling(window, min_periods=1).std(ddof=0)
print(df.info())
print(f'window: {window}')
print(df)
print('non-rolling result:', df['values'].iloc[len(df.index)-window:].values.std())
print('')
输出:
window: 3
values mean std
0 1234.0 1234.000000 0.000000
1 4567.0 2900.500000 1666.500000
2 6800.0 4200.333333 2287.053757
3 6810.0 6059.000000 1055.011216
4 6821.0 6810.333333 8.576454
5 6820.0 6817.000000 4.966555
6 6820.0 6820.333333 0.471405
7 6820.0 6820.000000 0.000000
8 6820.0 6820.000000 0.000000
9 6820.0 6820.000000 0.000000
10 6820.0 6820.000000 0.000000
non-rolling result: 0.0
window: 5
values mean std
0 1234.0 1234.000000 0.000000
1 4567.0 2900.500000 1666.500000
2 6800.0 4200.333333 2287.053757
3 6810.0 4852.750000 2280.329732
4 6821.0 5246.400000 2186.267193
5 6820.0 6363.600000 898.332366
6 6820.0 6814.200000 8.158431
7 6820.0 6818.200000 4.118252
8 6820.0 6820.200000 0.400000
9 6820.0 6820.000000 0.000021
10 6820.0 6820.000000 0.000021
non-rolling result: 0.0
正如预期的那样,对于 idx 7,8,9,10 使用 3 的窗口大小,std 是 0.对于窗口大小 5,我希望 idx 9 和 10 产生 0.但是,结果是不同的从 0.
As expected, the std is 0 for idx 7,8,9,10 using a window size of 3. For a window size of 5, I would expect idx 9 and 10 to yield 0. However, the result is different from 0.
如果我手动"计算每个窗口大小的最后一个窗口的 std(分别使用 idxs 8,9,10 和 6,7,8,9,10),我得到的结果都是 0案例.
If I calculate the std 'manually' for the last window of each window size (using idxs 8,9,10 and 6,7,8,9,10, respectively), I get the expected result of 0 for both cases.
有人知道这里可能是什么问题吗?任何数字警告?
Does anybody have an idea what could be the issue here? Any numerical caveats?
推荐答案
pd.rolling
中 std()
的实现似乎更喜欢高性能而不是数值精度.但是您可以应用 np
版本的标准偏差:
It seems that implementation of std()
in pd.rolling
prefers high performance over numerical accuracy. However You can apply np
version of standard deviation:
df.loc[:, 'std'] = df.rolling(window, min_periods=1).apply(np.std)
结果:
values std
0 1234.0 0.000000
1 4567.0 1666.500000
2 6800.0 2287.053757
3 6810.0 2280.329732
4 6821.0 2186.267193
5 6820.0 898.332366
6 6820.0 8.158431
7 6820.0 4.118252
8 6820.0 0.400000
9 6820.0 0.000000
10 6820.0 0.000000
现在精度更好了.
这篇关于Pandas 滚动 std 产生不一致的结果并且与 values.std 不同的文章就介绍到这了,希望我们推荐的答案对大家有所帮助,也希望大家多多支持!