删除列表中的重复数据框

问题描述

我在python中有一个列表，其中包含重复的数据帧.目的是整体删除这些重复的数据帧.这是一些代码:

I have a list in python that contains duplicate dataframes. The goal is to remove these duplicate dataframes in whole. Here is some code:

import pandas as pd
import numpy as np
##Creating Dataframes
data1_1 =[[1,2018,80], [2,2018,70]]

data1_2 =  [[1,2017,77], [3,2017,62]]


df1 = pd.DataFrame(data1_1, columns = ['ID', 'Year', 'Score'])
df2 = pd.DataFrame(data1_2, columns = ['ID', 'Year', 'Score'])


###Creating list with duplicates
all_df_list = [df1,df1,df1,df2,df2,df2]

期望的结果是这样

###Desired results
desired_list = [df1,df2]

是否可以删除python列表中的所有重复数据帧?

Is there a way to remove any duplicated dataframes within a python list?

谢谢

推荐答案

我们可以使用熊猫 DataFrame.equals 与 list comprehension 和 enumerate 组合以比较项目在彼此之间的列表中:

We can use pandas DataFrame.equals with list comprehension in combination with enumerate to compare the items in the list between each other:

desired_list = [all_df_list[x] for x, _ in enumerate(all_df_list) if all_df_list[x].equals(all_df_list[x-1]) is False]

print(desired_list)
[   ID  Year  Score
0   1  2018     80
1   2  2018     70,    ID  Year  Score
0   1  2017     77
1   3  2017     62]

---

如果比较的数据帧相等，则

DataFrame.equals 返回 True :

df1.equals(df1)
True

df1.equals(df2)
False

注意正如文本在评论中指出的那样.您的列表应按 [df1，df1，df1，df2，df2，df2] 进行排序.或使用更多df: [df1，df1，df2，df2，df3，df3]

NoteAs Wen-Ben noted in the comments. Your list should be sorted like [df1, df1, df1, df2, df2, df2]. Or with more df's: [df1, df1, df2, df2, df3, df3]

这篇关于删除列表中的重复数据框的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持！