我有成千上万个要处理的XML文件，它们具有相似的格式，但父名称不同且父位数不同。
通过书籍，谷歌，教程以及只是尝试代码，我已经能够提取所有这些数据。例如，参见:Parsing xml to pandas data frame throws memory error和Dynamic search through xml attributes using lxml and xpath in python
但是，我意识到我提取的数据很差，每个父级都重复了一个子级“Time”。
这就是我想要得到的。
Time blah abc
1200 100 2
1300 30 4
1400 70 2
这就是我所知道的。但是我当前的方法很笨拙(我将在示例XML下面显示)
child Time grandchild
0 blah 1200 100
1 blah 1300 30
...
n-2 abc 1200 2
n-1 abc 1300 4
n abc 1400 2
XML格式范例
<outer>
<inner>
<parent name = "blah" id = "1">
<child Time = "1200">
<grandchild>100</grandchild>
</child>
<child Time = "1300">
<grandchild>30</grandchild>
</child>
<child Time = "1400">
<grandchild>70</grandchild>
</child>
</parent>
<parent name = "abc" id = "2">
<child Time = "1200">
<grandchild>2</grandchild>
</child>
<child Time = "1300">
<grandchild>4</grandchild>
</child>
<child Time = "1400">
<grandchild>2</grandchild>
</child>
</parent>
<parent name = "1234" id = "7734">
<other> 12 </other>
</parent>
</inner>
</outer>
这是我获得输出的方法:
from lxml import etree, objectify
from pandas import *
dTime=[]
dparent = []
dgrandchild=[]
for df in root.xpath('/*/*/*/parent/child'):
dparent.append(df.getparent().attrib['name'])
## Iterate over attributes of time for specific parent
for attrib in df.attrib:
dTime.append(df.attrib[attrib])
## grandchild is a child of time, and iterate
subfields = df.getchildren()
for subfield in subfields:
dgrandchild.append(subfield.text)
df=DataFrame({'Parent': dparent,'Time':dTime,'grandchild':dgrandchld})
我可以采用此输出并重新调整它的形状，但这似乎效率低下，而且方法笨拙。
我想我需要一些味道:
#this does not work
data = []
for elem in root.xpath('/*/*/*/parent/child'):
elem_data = {}
for attrib in elem.attrib:
elem_data['Time'] = elem.attrib[attrib])
for child in elem.getchildren():
elem_data[getparent().attrib['name'])] = child.text
data.append(elem_data)
ndata = DataFrame(data)
最佳答案
我建议先解析为一个DataFrame，类似于已经进行的解析(有关实现，请参见下文)，然后根据需要进行调整。
然后，您正在寻找 pivot :
In [11]: df
Out[11]:
child Time grandchild
0 blah 1200 100
1 blah 1300 30
2 abc 1200 2
3 abc 1300 4
4 abc 1400 2
In [12]: df.pivot('Time', 'child', 'grandchild')
Out[12]:
child abc blah
Time
1200 2 100
1300 4 30
1400 2 NaN
我建议首先使用parse from a file并将所需的内容放入元组列表中:
from lxml import etree
root = etree.parse(file_name)
parents = root.getchildren()[0].getchildren()
In [21]: elems = [(p.attrib['name'], int(c.attrib['Time']), int(gc.text))
for p in parents
for c in p
for gc in c]
In [22]: elems
Out[22]:
[('blah', 1200, 100),
('blah', 1300, 30),
('blah', 1400, 70),
('abc', 1200, 2),
('abc', 1300, 4),
('abc', 1400, 2)]
对于多个文件，您甚至可以以更长的列表理解力对其进行重击。除非您有大量的xmls(这里
files是xmls的列表)，否则这应该不会太慢...elems = [(p.attrib['name'], int(c.attrib['Time']), int(gc.text))
for f in files
for p in etree.parse(f).getchildren()[0].getchildren()
for c in p
for gc in c]
将它们放在DataFrame中:
In [23]: pd.DataFrame(elems, columns=['child', 'Time', 'grandchild'])
Out[23]:
child Time grandchild
0 blah 1200 100
1 blah 1300 30
2 blah 1400 70
3 abc 1200 2
4 abc 1300 4
5 abc 1400 2
然后做枢轴。 :)