我有一个Pandas Dataframe,它具有列值作为字符串列表。每个列表可以包含一个或多个字符串。对于具有多个单词的字符串,我想将它们拆分为单个单词,以便每个列表仅包含单个单词。在下面的数据框中,只有sent_tags列具有包含可变长度字符串的列表。

数据框:

import pandas as pd
pd.set_option('display.max_colwidth', -1)
df = pd.DataFrame({"fruit_tags": [["'apples'", "'oranges'", "'pears'"], ["'melons'", "'peaches'", "'kiwis'"]], "sent_tags":[["'apples'", "'sweeter than oranges'", "'pears sweeter than apples'"], ["'melons'", "'sweeter than peaches'", "'kiwis sweeter than melons'"]]})
print(df)

    fruit_tags                        sent_tags
0   ['apples', 'oranges', 'pears']  ['apples', 'sweeter than oranges', 'pears sweeter than apples']
1   ['melons', 'peaches', 'kiwis']  ['melons', 'sweeter than peaches', 'kiwis sweeter than melons']


我的尝试:

我决定使用NLTK库中的word_tokenize将此类字符串分解为单个单词。我确实获得了列表中特定选择的标记化单词,但无法将它们组合在一起成为每一行的每个列表:

from nltk.tokenize import word_tokenize
df['sent_tags'].str[1].str.strip("'").apply(lambda x:word_tokenize(x.lower()))
#Output
0    [sweeter, than, oranges]
1    [sweeter, than, peaches]
Name: sent_tags, dtype: object


所需结果:

    fruit_tags                        sent_tags
0   ['apples', 'oranges', 'pears']  ['apples', 'sweeter', 'than', 'oranges', 'pears', 'sweeter', 'than', 'apples']
1   ['melons', 'peaches', 'kiwis']  ['melons', 'sweeter', 'than', 'peaches', 'kiwis', 'sweeter', 'than', 'melons']

最佳答案

对所有文本函数-striplowersplit使用列表理解和拼合:

s = df['sent_tags'].apply(lambda x: [z for y in x for z in y.strip("'").lower().split()])


要么:

s = [[z for y in x for z in y.strip("'").lower().split()] for x in df['sent_tags']]




df['sent_tags'] = s

print(df)
                       fruit_tags  \
0  ['apples', 'oranges', 'pears']
1  ['melons', 'peaches', 'kiwis']

                                                        sent_tags
0  [apples, sweeter, than, oranges, pears, sweeter, than, apples]
1  [melons, sweeter, than, peaches, kiwis, sweeter, than, melons]

关于python - 将多单词字符串拆分为包含字符串列表的Pandas系列的单个单词,我们在Stack Overflow上找到一个类似的问题:https://stackoverflow.com/questions/55220449/

10-14 18:25
查看更多