我先从字数列表开始:
julia> import Iterators: partition
julia> import StatsBase: countmap
julia> s = split("the lazy fox jumps over the brown dog");
julia> vocab_counter = countmap(s)
Dict{SubString{String},Int64} with 7 entries:
"brown" => 1
"lazy" => 1
"jumps" => 1
"the" => 2
"fox" => 1
"over" => 1
"dog" => 1
然后我要计算否。每个单词ngrams并将其存储在嵌套字典中。外键是ngram,内键是单词,最里面的值是给定单词的ngram计数。
我试过了:
ngram_word_counter = Dict{Tuple,Dict}()
for (word, count) in vocab_counter
for ng in ngram(word, 2) # bigrams.
if ! haskey(ngram_word_counter, ng)
ngram_word_counter[ng] = Dict{String,Int64}()
ngram_word_counter[ng][word] = 0
end
ngram_word_counter[ng][word] += 1
end
end
这给了我所需的数据结构:
julia> ngram_word_counter
Dict{Tuple,Dict} with 20 entries:
('b','r') => Dict("brown"=>1)
('t','h') => Dict("the"=>1)
('o','w') => Dict("brown"=>1)
('z','y') => Dict("lazy"=>1)
('o','g') => Dict("dog"=>1)
('u','m') => Dict("jumps"=>1)
('o','x') => Dict("fox"=>1)
('e','r') => Dict("over"=>1)
('a','z') => Dict("lazy"=>1)
('p','s') => Dict("jumps"=>1)
('h','e') => Dict("the"=>1)
('d','o') => Dict("dog"=>1)
('w','n') => Dict("brown"=>1)
('m','p') => Dict("jumps"=>1)
('l','a') => Dict("lazy"=>1)
('o','v') => Dict("over"=>1)
('v','e') => Dict("over"=>1)
('r','o') => Dict("brown"=>1)
('f','o') => Dict("fox"=>1)
('j','u') => Dict("jumps"=>1)
但是请注意,这些值是错误的:
('t','h') => Dict("the"=>1)
('h','e') => Dict("the"=>1)
本来应该:
('t','h') => Dict("the"=>2)
('h','e') => Dict("the"=>2)
自从这个词出现了两次。
仔细查看后,看来
haskey(ngram_word_counter, ng)
始终为false =(julia> ngram_word_counter = Dict{Tuple,Dict}()
for (word, count) in vocab_counter
for ng in ngram(word, 2) # bigrams.
println(haskey(ngram_word_counter, ng))
end
end
[出]:
false
false
false
false
false
false
false
false
false
false
false
false
false
false
false
false
false
false
false
false
为什么此
haskey()
条件始终为假? 最佳答案
TL; DR:应该为ngram_word_counter[ng][word] += count
而不是ngram_word_counter[ng][word] += 1
。
仅添加1
会忽略一个单词出现多次的多重贡献。单词出现的次数被编码为vocab_counter
值,这些值进入count
循环的变量for
中。因此,增量应为count
。
以后的调试检查是无效的,并且在通常情况下,调试代码的错误将问题弄混了。预期的检查可能是:
julia> ngram_word_counter = Dict{Tuple,Dict}()
for (word, count) in vocab_counter
for ng in ngram(word, 2) # bigrams.
println(haskey(ngram_word_counter, ng))
ngram_word_counter[ng] = 1
end
end
关于dictionary - 为什么此haskey()条件始终为false?,我们在Stack Overflow上找到一个类似的问题:https://stackoverflow.com/questions/43115581/