从Python的NLTK中的自定义文本生成随机句子?

问题描述

我在Python下使用NLTK遇到麻烦，特别是.generate()方法.

I'm having trouble with the NLTK under Python, specifically the .generate() method.

打印使用Trigram语言模型生成的随机文本.

Print random text, generated using a trigram language model.

参数:

   * length (int) - The length of text to generate (default=100)

这是我正在尝试的简化版本.

Here is a simplified version of what I am attempting.

import nltk

words = 'The quick brown fox jumps over the lazy dog'
tokens = nltk.word_tokenize(words)
text = nltk.Text(tokens)
print text.generate(3)

这将始终生成

Building ngram index...
The quick brown
None

与从单词中构建随机短语相反.

As opposed to building a random phrase out of the words.

这是我的输出内容

print text.generate()

Building ngram index...
The quick brown fox jumps over the lazy dog fox jumps over the lazy
dog dog The quick brown fox jumps over the lazy dog dog brown fox
jumps over the lazy dog over the lazy dog The quick brown fox jumps
over the lazy dog fox jumps over the lazy dog lazy dog The quick brown
fox jumps over the lazy dog the lazy dog The quick brown fox jumps
over the lazy dog jumps over the lazy dog over the lazy dog brown fox
jumps over the lazy dog quick brown fox jumps over the lazy dog The
None

再次以相同的文本开始，但随后将其更改.我也尝试过使用Orwell 1984年的第一章.总是始终以前三个标记(在本例中是一个空格)开头，然后 then 继续随机生成文本.

Again starting out with the same text, but then varying it. I've also tried using the first chapter from Orwell's 1984. Again that always starts with the first 3 tokens (one of which is a space in this case) and then goes on to randomly generate text.

我在做什么错了?

推荐答案

要生成随机文本，U需要使用马氏链

To generate random text, U need to use Markov Chains

执行此操作的代码:从此处

import random

class Markov(object):

  def __init__(self, open_file):
    self.cache = {}
    self.open_file = open_file
    self.words = self.file_to_words()
    self.word_size = len(self.words)
    self.database()


  def file_to_words(self):
    self.open_file.seek(0)
    data = self.open_file.read()
    words = data.split()
    return words


  def triples(self):
    """ Generates triples from the given data string. So if our string were
    "What a lovely day", we'd generate (What, a, lovely) and then
    (a, lovely, day).
    """

    if len(self.words) < 3:
      return

    for i in range(len(self.words) - 2):
      yield (self.words[i], self.words[i+1], self.words[i+2])

  def database(self):
    for w1, w2, w3 in self.triples():
      key = (w1, w2)
      if key in self.cache:
    self.cache[key].append(w3)
      else:
    self.cache[key] = [w3]

  def generate_markov_text(self, size=25):
    seed = random.randint(0, self.word_size-3)
    seed_word, next_word = self.words[seed], self.words[seed+1]
    w1, w2 = seed_word, next_word
    gen_words = []
    for i in xrange(size):
      gen_words.append(w1)
      w1, w2 = w2, random.choice(self.cache[(w1, w2)])
    gen_words.append(w2)
    return ' '.join(gen_words)

说明:使用Python使用Markov链生成伪随机文本

Explaination: Generating pseudo random text with Markov chains using Python

这篇关于从Python的NLTK中的自定义文本生成随机句子?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持！