本文介绍了pyspark行号数据框的处理方法，对大家解决问题具有一定的参考价值，需要的朋友们下面随着小编来一起学习吧！
问题描述
我有一个数据框，其中包含时间time，a，b，c，d，val.我想创建一个带有附加列的数据框，该数据框将包含每个组内行的行号，其中a，b，c，d是组键.
I have a dataframe, with columns time,a,b,c,d,val.I would like to create a dataframe, with additional column, that will contain the row number of the row, within each group, where a,b,c,d is a group key.
我尝试通过定义窗口函数来使用spark sql，特别是在sql中，它看起来像这样:
I tried with spark sql, by defining a window function, in particular, in sql it will look like this:
select time, a,b,c,d,val, row_number() over(partition by a,b,c,d order by time) as rn from table
group by a,b,c,d,val
我想在数据框itslef上执行此操作，而无需使用sparksql.
I would like to do this on the dataframe itslef, without using sparksql.
谢谢
推荐答案
我不太了解python api，但我会尝试一下.您可以尝试以下操作:
I don't know the python api too much, but I will give it a try. You can try something like:
from pyspark.sql import functions as F
df.withColumn("row_number", F.row_number().over(Window.partitionBy("a","b","c","d").orderBy("time"))).show()
这篇关于pyspark行号数据框的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持！