pandas - 分割数据框

打印(df)

  A  B
  0  10
  1  30
  2  50
  3  20
  4  10
  5  30


  A  B
  0  10
  1  30

  A  B
  2  50

  A  B
  3  20
  4  10
  5  30

最佳答案

这是使用numba加快for loop的方法:
我们检查何时达到限制，并重置total计数，并分配一个新的group:

from numba import njit

@njit
def cumsum_reset(array, limit):
    total = 0
    counter = 0
    groups = np.empty(array.shape[0])
    for idx, i in enumerate(array):
        total += i
        if total >= limit or array[idx-1] == limit:
            counter += 1
            groups[idx] = counter
            total = 0
        else:
            groups[idx] = counter

    return groups

grps = cumsum_reset(df['B'].to_numpy(), 50)

for _, grp in df.groupby(grps):
    print(grp, '\n')

输出

   A   B
0  0  10
1  1  30

   A   B
2  2  50

   A   B
3  3  20
4  4  10
5  5  30

时间:

# create dataframe of 600k rows
dfbig = pd.concat([df]*100000, ignore_index=True)
dfbig.shape

(600000, 2)

# Erfan
%%timeit
cumsum_reset(dfbig['B'].to_numpy(), 50)

4.25 ms ± 46.1 µs per loop (mean ± std. dev. of 7 runs, 100 loops each)

# Daniel Mesejo
def daniel_mesejo(th, column):
    cumsum = column.cumsum()
    bins = list(range(0, cumsum.max() + 1, th))
    groups = pd.cut(cumsum, bins)

    return groups

%%timeit
daniel_mesejo(50, dfbig['B'])

10.3 s ± 2.17 s per loop (mean ± std. dev. of 7 runs, 1 loop each)

结论，用于循环的numba快24〜x。

关于pandas - 分割数据框，我们在Stack Overflow上找到一个类似的问题：https://stackoverflow.com/questions/58590639/