问题描述
我是R的新手.我昨天刮了一个要求登录的网站，该页面为xml格式，如下所示.
I am pretty new to R. I scraped a website that required login yesterday, the page is xml format like below.
<result status="success">
<code>1</code>
<note>success</note>
<teacherList>
<teacher id="D95">
<name>Mary</name>
<department id="420">
<name>Math</name>
</department>
<department id="421">
<name>Statistics</name>
</department>
</teacher>
<teacher id="D73">
<name>Adam</name>
<department id="412">
<name>English</name>
</department>
</teacher>
</teacherList>
</result>
最近，我刚刚将XML转换为列表.
Recently I just Converted an XML to a list.
library(XML)
library(rvest)
library(plyr)
library(dplyr)
library(httr)
library(pipeR)
library(xml2)
url.address <- "http://xxxxxxxxxxxxxxxxx"
session <-html_session(url.address)
form <-html_form(read_html(url.address))[[1]]
filled_form <- set_values(form,
"userid" = "id",
"Password" = "password")
s <- submit_form(session,filled_form)
z = read_xml(s$response)
z1 = as_list(z)
z2 <- z1$teacherList
现在，我需要从列表中提取数据并将其作为数据框.顺便说一句，有些人属于2个部门，但有些人仅属于1个部门.z2列表的一部分如下所示:
Now I need to extract data from a list and make it as a data frame. By the way, some people belong to 2 departments, but some only belong to 1. A part of the list z2 looks like below:
z2[[1]]
$name
$name[[1]]
[1] "Mary"
$department
$department$name
$department$name[[1]]
[1] "Math"
attr(,"id")
[1] "420"
$department
$department$name
$department$name[[1]]
[1] "statistics"
attr(,"id")
[1] "421"
attr(,"id")
[1] "D95236"
当我一一提取它们时，花费的时间太长:
When I extracted them one by one, it took too long:
attr(z2[[1]],"id")
"D95"
z2[[1]][[1]][[1]]
玛丽"
z2[[1]][[2]][[1]][[1]]
数学"
attr(z2[[1]][[2]], "id")
"420"
z2[[1]][[3]][[1]][[1]]
统计信息"
attr(z2[[1]][[3]], "id")
"421"
attr(z2[[2]],"id")
"D73"
z2[[2]][[1]][[1]]
亚当"
z2[[2]][[2]][[1]][[1]]
英语"
attr(z2[[2]][[2]],"id")
"412"
所以我试图写一个循环:
So I tried to write a loop:
for (x in 1:2){
for (y in 2:3){
a <- attr(z2[[x]],"id")
b <- z2[[x]][[1]][[1]]
d <- z2[[x]][[y]][[1]][[1]]
e <- attr(z2[[x]][[y]],"id")
g <- cbind(print(a),print(b),print(d),print(e))
}}
但是它根本不起作用，因为有些人只属于一个部门.我预期的结果:
but it doesn't work at all since some of the people only belong to one department. The result I expected:
任何建议将不胜感激！
Any advice would be appreciated!
dput(head(z2, 10))
structure(list(teacher = structure(list(name = list("Mary"),
department = structure(list(name = list("Math")), .Names = "name", id = "420"),
department = structure(list(name = list("statistics")), .Names = "name", id = "421")), .Names = c("name",
"department", "department"), id = "D95"), teacher = structure(list(
name = list("Adam"), department = structure(list(name = list(
"English")), .Names = "name", id = "412")), .Names = c("name",
"department"), id = "D73"), teacher = structure(list(name = list(
"Kevin"), department = structure(list(name = list("Chinese")), .Names = "name", id = "201")), .Names = c("name",
"department"), id = "D101"), teacher = structure(list(name = list(
"Nana"), department = structure(list(name = list("Science")), .Names = "name", id = "205")), .Names = c("name",
"department"), id = "D58"), teacher = structure(list(name = list(
"Nelson"), department = structure(list(name = list("Music")), .Names = "name", id = "370")), .Names = c("name",
"department"), id = "D14"), teacher = structure(list(name = list(
"Esther"), department = structure(list(name = list("Medicine")), .Names = "name", id = "361")), .Names = c("name",
"department"), id = "D28"), teacher = structure(list(name = list(
"Mia"), department = structure(list(name = list("Chemistry")), .Names = "name", id = "326")), .Names = c("name",
"department"), id = "D17"), teacher = structure(list(name = list(
"Jack"), department = structure(list(name = list("German")), .Names = "name", id = "306")), .Names = c("name",
"department"), id = "D80"), teacher = structure(list(name = list(
"Tom"), department = structure(list(name = list("French")), .Names = "name", id = "360")), .Names = c("name",
"department"), id = "D53"), teacher = structure(list(name = list(
"Allen"), department = structure(list(name = list("Spanish")), .Names = "name", id = "322")), .Names = c("name",
"department"), id = "D18")), .Names = c("teacher", "teacher",
"teacher", "teacher", "teacher", "teacher", "teacher", "teacher", "teacher",
"teacher"))
推荐答案
构造起来有点疯狂，但我认为它或多或少与该帖子的先前版本中发布的期望输出一致.我必须在lapply函数中使用sapply来提取第二个ID变量.
This was a bit crazy to construct, but I think it more or less conforms with the desired output posted in a previous version of the post. I had to use sapply within the lapply function to pull out the second ID variable.
do.call(rbind, # rbind list of data.frames output by lapply
lapply(unname(z2), # loop through list, first drop outer names
function(x) { # begin lapply function
temp <- unlist(x) # unlist inner elements to a vector
data.frame(name=temp[names(temp) == "name"], # subset on names
dept=temp[names(temp) == "department.name"], # subset on dept
id=attr(x, "id"), # extract one id
id2=unlist(sapply(x, attr, "id")), # extract other id
row.names=NULL) # end data.frame function, drop row.names
})) # end lapply function, lapply, and do.call
这将返回
name dept id id2
1 Mary Math D95 420
2 Mary statistics D95 421
3 Adam English D73 412
4 Kevin Chinese D101 201
5 Nana Science D58 205
6 Nelson Music D14 370
7 Esther Medicine D28 361
8 Mia Chemistry D17 326
9 Jack German D80 306
10 Tom French D53 360
11 Allen Spanish D18 322
第二个列表的结构与初始示例在许多方面有所不同.第一:移除一个巢.也就是说，新列表的深度比初始示例的深度小一.好像您为初始列表提供了z2 [[1]].其次，第二个示例缺少我最初所谓的id(诸如D95和D101之类的值).
The structure of the second list differs in a number of ways from the initial example. First: one nest is removed. That is, the depth of the new list is one less than that of the initial example. It would be as if you provided z2[[1]] for the initial list. Second, the second example is missing what I called id initially (values such as D95 and D101).
通过对原始代码的一些操作，我可以使用它
With a bit of manipulation of the original code, I got this to work with
lapply(list(z3), # loop through list, first drop outer names
function(x) { # begin lapply function
temp <- unlist(x) # unlist inner elements to a vector
data.frame(name=temp[names(temp) == "name"], # subset on names
dept=temp[names(temp) == "department.name"], # subset on dept
# id=attr(x, "id"), # extract one id
id2=unlist(sapply(x, attr, "id")), # extract other id
row.names=NULL) # end data.frame function, drop row.names
})
我在z2之前提到的对代码地址的更改被list(z3)替换为lapply的第一个参数，从而构造了所需的列表深度.另外，由于id2不存在，内部函数id=attr(x, "id"),的行已被注释掉.
The changes to the code address what I mentioned before z2 is replaced by list(z3) as the first argument to lapply, which constructs the needed list depth. Also, the line of the inner function id=attr(x, "id"), has been commented out as id2 does not exist.
这篇关于使用循环从R中的列表中提取数据的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持！