我的test.csv文件是(没有标题):

very good, very bad, you are great
very bad, good restaurent, nice place to visit


我想用,分隔我的语料库,以使最终的DocumentTermMatrix变为:

      terms
 docs       very good      very bad        you are great   good restaurent   nice place to visit
  doc1       tf-idf          tf-idf         tf-idf          0                    0
  doc2       0                tf-idf         0                tf-idf             tf-idf


如果不从DTM加载documents,则可以正确生成上述csv file,如下所示:

library(tm)
docs <- c(D1 = "very good, very bad, you are great",
    D2 = "very bad, good restaurent, nice place to visit")

dd <- Corpus(VectorSource(docs))
dd <- tm_map(dd, function(x) {
    PlainTextDocument(
       gsub("\\s+","~",strsplit(x,",\\s*")[[1]]),
       id=ID(x)
     )
})
inspect(dd)

# A corpus with 2 text documents
#
# The metadata consists of 2 tag-value pairs and a data frame
# Available tags are:
#   create_date creator
# Available variables in the data frame are:
#   MetaID

# $D1
# very~good
# very~bad
# you~are~great
#
# $D2
# very~bad
# good~restaurent
# nice~place~to~visit

dtm <- DocumentTermMatrix(dd, control = list(weighting = weightTfIdf))
as.matrix(dtm)


这将产生

# Docs good~restaurent nice~place~to~visit very~bad very~good you~are~great
#   D1       0.0000000           0.0000000        0 0.3333333     0.3333333
#   D2       0.3333333           0.3333333        0 0.0000000     0.0000000


如果我要从document文件加载csv,那么每个文档的第一项才被加入,如下所示:

> file_loc <- "testdata.csv"
> require(tm)
  Loading required package: tm
> x <- read.csv(file_loc, header = FALSE)
> x <- data.frame(lapply(x, as.character), stringsAsFactors=FALSE)
> dd <- Corpus(DataframeSource(x))
> dd <- tm_map(dd, stripWhitespace)
> dd <- tm_map(dd, tolower)
>  dd <- tm_map(dd, function(x) {
            PlainTextDocument(
            gsub("\\s+","~",strsplit(x,",\\s*")[[1]]),
            id=ID(x)
            )
          })
> inspect(dd)


仅加入以下第一项:

# $D1
# very~good

#
# $D2
# very~bad


我如何才能加入所有条款并像上面一样创建DocumentTermMatrix

最佳答案

您读取数据不正确。我使用scan进行阅读。以下作品:

docs <- scan("testdata.csv", "character", sep = "\n")

dd <- Corpus(VectorSource(x))
dd <- tm_map(dd, function(x) {
  PlainTextDocument(
    gsub("\\s+","~",strsplit(x,",\\s*")[[1]]),
    id=ID(x)
  )
})
inspect(dd)

dtm <- DocumentTermMatrix(dd, control = list(weighting = weightTfIdf))
as.matrix(dtm)

07-24 09:54
查看更多