问题描述
我有一个numpy/pandas值列表:
I have a numpy/pandas list of values:
a = np.random.randint(-100, 100, 10000)
b = a/100
我想应用自定义的cumsum函数，但是我还没有找到一种没有循环的方法.自定义函数将累加值设置为1的上限和-1的下限，如果求和的加"超出了这些限制，则加"变为0.
I want to apply a custom cumsum function, but I haven't found a way to do it without loops. The custom function sets an upper limit of 1 and lower limit of -1 for the cumsum values, if the "add" to sum is beyond these limits the "add" becomes 0.
如果总和在-1和1的限制之间，但加"的值将超出限制，则加"将成为-1或1的余数.
In the case that sum is between the limits of -1 and 1 but the "added" value would break beyond the limits, the "added" becomes the remainder to -1 or 1.
这是循环版本:
def cumsum_with_limits(values):
cumsum_values = []
sum = 0
for i in values:
if sum+i <= 1 and sum+i >= -1:
sum += i
cumsum_values.append(sum)
elif sum+i >= 1:
d = 1-sum # Remainder to 1
sum += d
cumsum_values.append(sum)
elif sum+i <= -1:
d = -1-sum # Remainder to -1
sum += d
cumsum_values.append(sum)
return cumsum_values
有什么方法可以对此向量化?我需要在大型数据集上运行此功能，而性能是我当前的问题.感谢任何帮助！
Is there any way to vectorize this? I need to run this function on large datasets and performance is my current issue. Appreciate any help!
更新:修复了一些代码，并对输出进行了一些澄清:使用np.random.seed(0)，前6个值是:
Update: Fixed the code a bit, and a little clarification for the outputs:Using np.random.seed(0), the first 6 values are:
b = [0.72, -0.53, 0.17, 0.92, -0.33, 0.95]
预期输出:
o = [0.72, 0.19, 0.36, 1, 0.67, 1]
推荐答案
循环不一定是不可取的.如果性能是一个问题，请考虑numba.在不实质改变逻辑的前提下，性能提高了约330倍:
Loops aren't necessarily undesirable. If performance is an issue, consider numba. There's a ~330x improvement without materially changing your logic:
from numba import njit
np.random.seed(0)
a = np.random.randint(-100, 100, 10000)
b = a/100
@njit
def cumsum_with_limits_nb(values):
n = len(values)
res = np.empty(n)
sum_val = 0
for i in range(n):
x = values[i]
if (sum_val+x <= 1) and (sum_val+x >= -1):
res[i] = x
sum_val += x
elif sum_val+x >= 1:
d = 1-sum_val # Remainder to 1
res[i] = d
sum_val += d
elif sum_val+x <= -1:
d = -1-sum_val # Remainder to -1
res[i] = d
sum_val += d
return res
assert np.isclose(cumsum_with_limits(b), cumsum_with_limits_nb(b)).all()
如果您不介意牺牲某些性能，则可以更简洁地重写此循环:
If you don't mind sacrificing some performance, you can rewrite this loop more succinctly:
@njit
def cumsum_with_limits_nb2(values):
n = len(values)
res = np.empty(n)
sum_val = 0
for i in range(n):
x = values[i]
next_sum = sum_val + x
if np.abs(next_sum) >= 1:
x = np.sign(next_sum) - sum_val
res[i] = x
sum_val += x
return res
具有与nb2类似的性能，这是一种替代方法(感谢@jdehesa):
With similar performance to nb2, here's an alternative (thanks to @jdehesa):
@njit
def cumsum_with_limits_nb3(values):
n = len(values)
res = np.empty(n)
sum_val = 0
for i in range(n):
x = min(max(sum_val + values[i], -1) , 1) - sum_val
res[i] = x
sum_val += x
return res
性能比较:
assert np.isclose(cumsum_with_limits(b), cumsum_with_limits_nb(b)).all()
assert np.isclose(cumsum_with_limits(b), cumsum_with_limits_nb2(b)).all()
assert np.isclose(cumsum_with_limits(b), cumsum_with_limits_nb3(b)).all()
%timeit cumsum_with_limits(b) # 12.5 ms per loop
%timeit cumsum_with_limits_nb(b) # 40.9 µs per loop
%timeit cumsum_with_limits_nb2(b) # 54.7 µs per loop
%timeit cumsum_with_limits_nb3(b) # 54 µs per loop
这篇关于带有上限/下限的Numpy自定义Cumsum函数?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持！