问题描述
我想要创建一个反转的文档索引,因此我需要知道一个集合中的所有独特的单词,已使用 answer按顺序二创建一个嵌套字典。提供的解决方案工作正常,但有一个问题。
首先,我打开文件,并制作一个唯一字的列表。这些独特的词我想要比较的原始文件。当有匹配时,频率计数器应该被更新并且它的值被存储在二维数组中。
输出应该最终看起来像这样:
word1,{doc1:freq},{doc2:freq}< br>
word2,{doc1:freq},{doc2:freq},{doc3:freq}
等....
问题是我不能更新字典变量。当尝试这样做时,我得到的错误:
文件scriptV3.py,第45行,在主
freq = dictionary [keyword] [filename] + 1
TypeError:不支持的操作数类型为+:'AutoVivification'和'int'
我想我需要以某种方式将AutoVivification的实例转换为int ....
如何去?
提前感谢
我的代码:
#!/ usr / bin / env python
#encoding:utf-8
import sys
import os
import re
import glob
import string
import sets
class AutoVivification(dict):
实现perl的自动更新特性。 b $ b def __getitem __(self,item):
try:
return dict .__ getitem __(self,item)
,除了KeyError:
value = self [item] = type self)()
返回值
def main():
pad ='temp /'
dictionary = AutoVivification()
docID = 0
用于glob.glob(os.path.join(pad,'* .html'))中的文件:#对于指定文件夹中的所有文件:
docID = docID + 1
filename = doc(doc)+ str(docID)
text = open(files,'r')read()以字符串的形式返回文件内容
text = extract(text,'< < / pre>')#call提取函数以从< pre>标签
text = text.lower()#所有小写字母
exclude = set(string.punctuation)#sets所有标点符号的列表
text =''.join如果字符不在排除)#使用创建排除列表从文件中删除字符
text = text.split()#creates从字符串列表(数组)
uniques = set(text)#make list
对于uniques中的关键字:#对于每个独特的单词do
用于文本中的单词:#对于doc中的每个单词:
if(word ==关键字和字典[关键字] [文件名]不是无):#if有关键字增量计数器发生
freq = dictionary [关键字] [文件名]#我们失败,实例为整数。
freq = dictionary [keyword] [filename] + 1
print(keyword,dictionary [keyword])
else:
dictionary [word] [filename] b
#extract子字符串1和2之间的文本
def extract(text,sub1,sub2):
return text.split(sub1,1)[ - 1] .split(sub2,1 )[0]
如果__name__ =='__main__':
main()
import collections
dictionary = collections.defaultdict(lambda:collections.defaultdict(int))
这将创建一个默认值为0的字典字典。当您希望增加一个条目时,使用:
dictionary [keyword] [filename] + = 1
i am trying to make an inversed document index, therefore i need to know from all unique words in a collection in which doc they occur and how often.
i have used this answer in order two create a nested dictionary. The provided solution works fine, with one problem though.
First i open the file and make a list of unique words. These unique words i than want to compare with the original file. When there is a match, the frequency counter should be updated and its value be stored in the two dimensional array.
output should eventually look like this:
word1, {doc1 : freq}, {doc2 : freq} <br>
word2, {doc1 : freq}, {doc2 : freq}, {doc3:freq}
etc....
Problem is that i cannot update the dictionary variable. When trying to do so i get the error:
File "scriptV3.py", line 45, in main
freq = dictionary[keyword][filename] + 1
TypeError: unsupported operand type(s) for +: 'AutoVivification' and 'int'
I think i need to cast in some way the instance of AutoVivification to int....
How to go?
thanks in advance
my code:
#!/usr/bin/env python
# encoding: utf-8
import sys
import os
import re
import glob
import string
import sets
class AutoVivification(dict):
"""Implementation of perl's autovivification feature."""
def __getitem__(self, item):
try:
return dict.__getitem__(self, item)
except KeyError:
value = self[item] = type(self)()
return value
def main():
pad = 'temp/'
dictionary = AutoVivification()
docID = 0
for files in glob.glob( os.path.join(pad, '*.html') ): #for all files in specified folder:
docID = docID + 1
filename = "doc_"+str(docID)
text = open(files, 'r').read() #returns content of file as string
text = extract(text, '<pre>', '</pre>') #call extract function to extract text from within <pre> tags
text = text.lower() #all words to lowercase
exclude = set(string.punctuation) #sets list of all punctuation characters
text = ''.join(char for char in text if char not in exclude) # use created exclude list to remove characters from files
text = text.split() #creates list (array) from string
uniques = set(text) #make list unique (is dat handig? we moeten nog tellen)
for keyword in uniques: #For every unique word do
for word in text: #for every word in doc:
if (word == keyword and dictionary[keyword][filename] is not None): #if there is an occurence of keyword increment counter
freq = dictionary[keyword][filename] #here we fail, cannot cast object instance to integer.
freq = dictionary[keyword][filename] + 1
print(keyword,dictionary[keyword])
else:
dictionary[word][filename] = 1
#extract text between substring 1 and 2
def extract(text, sub1, sub2):
return text.split(sub1, 1)[-1].split(sub2, 1)[0]
if __name__ == '__main__':
main()
One could use Python's collections.defaultdict instead of creating an AutoVivification class and then instantiating dictionary as an object of that type.
import collections
dictionary = collections.defaultdict(lambda: collections.defaultdict(int))
This will create a dictionary of dictionaries with a default value of 0. When you wish to increment an entry, use:
dictionary[keyword][filename] += 1
这篇关于Python:如何更新嵌套字典中键值对的值?的文章就介绍到这了,希望我们推荐的答案对大家有所帮助,也希望大家多多支持!