所以我已经研究了一段时间了,只是没有任何地方而且不确定该怎么做。
数据集实际上是15,000个产品名称。格式全部不同,有些连字符最多为6个连字符,某些连字符,长度不同。
当我在大型数据集上使用它时,我使用的代码始终只返回与部分字符串相反的第一个字母。
在我用来测试的小型数据集上工作正常。
我假设发生这种情况是因为:
匹配完整的部分字符串时,我尚未创建停止部分
因为它试图匹配与单个字符相对的单词,并在发现差异时停止。
在大型数据集上克服此问题的最佳方法是什么,我缺少什么?还是我必须要做本手册?
原始测试数据集
`1.star t-shirt-large-red
2.star t-shirt-large-blue
3.star t-shirt-small-red
4.beautiful rainbow skirt small
5.long maxwell logan jeans- light blue -32L-28W
6.long maxwell logan jeans- Dark blue -32L-28W`
所需的数据集/输出:
`COL1 COL2 COL3 COL4
1[star t-shirt] [large] [red] NONE
2[star t-shirt] [large] [blue] NONE
3[star t-shirt] [small] [red] NONE
4[beautiful rainbow skirt small] [small] NONE NONE
5[long maxwell logan jeans] [light blue] [32L] [28W]
6[long maxwell logan jeans] [Dark blue] [32L] [28W]`
这是我之前提出的问题中得到帮助的代码:
`df['onkey'] = 1
df1 = pd.merge(df[['name','onkey']],df[['name','onkey']], on='onkey')
df1['list'] = df1.apply(lambda x:[x.name_x,x.name_y],axis=1)
from os.path import commonprefix
df1['COL1'] = df1['list'].apply(lambda x:commonprefix(x))
df1['COL1_num'] = df1['COL1'].apply(lambda x:len(x))
df1 = df1[(df1['COL1_num']!=0)]
df1 = df1.loc[df1.groupby('name_x')['COL1_num'].idxmin()]
df = df.rename(columns ={'name':'name_x'})
df = pd.merge(df,df1[['name_x','COL1']],on='name_x',how ='left')`
`df['len'] = df['COL1'].apply(lambda x: len(x))
df['other'] = df.apply(lambda x: x.name_x[x.len:],axis=1)
df['COL1'] = df['COL1'].apply(lambda x: x.strip())
df['COL1'] = df['COL1'].apply(lambda x: x[:-1] if x[-1]=='-' else x)
df['other'] = df['other'].apply(lambda x:x.split('-'))
df = df[['COL1','other']]
df = pd.concat([df['COL1'],df['other'].apply(pd.Series)],axis=1)`
` COL1 0 1 2
0 star t-shirt large red NaN
1 star t-shirt large blue NaN
2 star t-shirt small red NaN
3 beautiful rainbow skirt small NaN NaN
4 long maxwell logan jeans light blue 32L 28W
5 long maxwell logan jeans Dark blue 32L 28W`
***************更新*****************
这是您输入的产品列表,有些有变型,有些没有。
在搜索重复的字符串以确定什么是带变体的产品和不带变体的产品时,什么都不会出现;因为由于在字符串的末尾添加了变体,所以它们都被视为唯一值,因此什么也没有出现。
因此,我想将部分或相似的字符串分组在一起(最长的匹配项),提取该组中最长的匹配字符串,然后将差异放入其他列中。
如果产品/ string是唯一的,只需将列有最长字符串的列打印到列中。
star t-shirt-large-red star t-shirt-large-blue star t-shirt-small-red beautiful rainbow skirt small long maxwell logan jeans- light blue -32L-28W long maxwell logan jeans- Dark blue -32L-28W Organic and natural candy - 3 Pack - Mint Organic and natural candy - 3 Pack - Vanilla Organic and natural candy - 3 Pack - Strawberry Organic and natural candy - 3 Pack - Chocolate Organic and natural candy - 3 Pack - Banana Organic and natural candy - 3 Pack - Cola Organic and natural candy - 12 Pack Assorted Morgan T-shirt Company - Small/Medium-Blue Morgan T-shirt Company - Medium/Large-Blue Morgan T-shirt Company - Medium/Large-red Morgan T-shirt Company - Small/Medium-Red Morgan T-shirt Company - Small/Medium-Green Morgan T-shirt Company - Medium/Large-Green Nelly dress leopard small
col1 col2 col3 col4 star t-shirt large red
star t-shirt large blue
star t-shirt small red
beautiful rainbow skirt small
Long maxwell logan jeans light blue 32L 28W Long maxwell logan jeans Dark blue 32L 28W Organic and natural candy 3 Pack Mint Organic and natural candy 3 Pack Vanilla
Organic and natural candy 3 Pack Strawberry
Organic and natural candy 3 Pack Chocolate
Organic and natural candy 3 Pack Banana
Organic and natural candy 3 Pack Cola Organic and natural candy 12 Pack Assorted Morgan T-shirt Company Small/Medium Blue Morgan T-shirt Company Medium/Large Blue Morgan T-shirt Company Medium/Large Red
Morgan T-shirt Company Small/Medium Red
Morgan T-shirt Company Small/Medium Green
Morgan T-shirt Company Medium/Large Green
Nelly dress Leopard Small
Bijoux
Princess PJ-set
Lemon tank top Yellow Medium
最佳答案
如下构造DataFrame df:
df = pd.DataFrame()
df = df.append(['1.star t-shirt-large-red'])
df = df.append(['2.star t-shirt-large-blue'])
df = df.append(['4.beautiful rainbow skirt small'])
df = df.append(['5.long maxwell logan jeans- light blue -32L-28W'])
df = df.append(['6.long maxwell logan jeans- Dark blue -32L-28W'])
df.columns = ['Product']
以下代码
(a)去除任何空格,
(b)除以句号('。')并获取随后的内容,
(c)由于进一步的操作,将“ t恤”替换为“ t恤”(如果需要,请在操作后将其改回)
(d)再次用'-'分割,然后扩展为您的数据框。
df['Product'].str.strip().str.split('.').str.get(1).str.replace('t-shirt', 'tshirt').str.split('-', expand = True)
输出:
0 1 2 3
0 star tshirt large red None
0 star tshirt large blue None
0 beautiful rainbow skirt small None None None
0 long maxwell logan jeans light blue 32L 28W
0 long maxwell logan jeans Dark blue 32L 28W
鉴于您的产品命名不一致,将会遗漏一些边缘情况(例如:
beautiful rainbow skirt small
)。您可能不得不再次将它们捞出。