1、Itertools模块迭代器的种类
1.1 无限迭代器：
| 迭代器 | 参数 | 结果 | 示例 |
| count() | start, [step] | start, start+step, start+2*step, ... | count(10) --> 10 11 12 13 14 ... |
| cycle() | p | p0, p1, ... plast, p0, p1, ... | cycle('ABCD') --> A B C D A B C D ... |
| repeat() | elem [,n] | elem, elem, elem, ... endlessly or up to n times | repeat(10, 3) --> 10 10 10 |
1.2 终止于最短输入序列的迭代器：
| 迭代器 | 参数 | 结果 | 示例 |
| accumulate() | p [,func] | p0, p0+p1, p0+p1+p2, ... | accumulate([1,2,3,4,5]) --> 1 3 6 10 15 |
| chain() | p, q, ... | p0, p1, ... plast, q0, q1, ... | chain('ABC', 'DEF') --> A B C D E F |
| chain.from_iterable() | iterable | p0, p1, ... plast, q0, q1, ... | chain.from_iterable(['ABC', 'DEF']) --> A B C D E F |
| compress() | data, selectors | (d[0] if s[0]), (d[1] if s[1]), ... | compress('ABCDEF', [1,0,1,0,1,1]) --> A C E F |
| dropwhile() | pred, seq | seq[n], seq[n+1], starting when pred fails | dropwhile(lambda x: x<5, [1,4,6,4,1]) --> 6 4 1 |
| filterfalse() | pred, seq | elements of seq where pred(elem) is false | filterfalse(lambda x: x%2, range(10)) --> 0 2 4 6 8 |
| groupby() | iterable[, keyfunc] | sub-iterators grouped by value of keyfunc(v) | |
| islice() | seq, [start,] stop [, step] | elements from seq[start:stop:step] | islice('ABCDEFG', 2, None) --> C D E F G |
| starmap() | func, seq | func(*seq[0]), func(*seq[1]), ... | starmap(pow, [(2,5), (3,2), (10,3)]) --> 32 9 1000 |
| takewhile() | pred, seq | seq[0], seq[1], until pred fails | takewhile(lambda x: x<5, [1,4,6,4,1]) --> 1 4 |
| tee() | it, n | it1, it2, ... itn splits one iterator into n | |
| zip_longest() | p, q, ... | (p[0], q[0]), (p[1], q[1]), ... | zip_longest('ABCD', 'xy', fillvalue='-') --> Ax By C- D- |
1.3 组合产生器
| 迭代器 | 参数 | 结果 |
| product() | p, q, ...[repeat=1] | 笛卡尔乘积，等价于for循环嵌套（乘法原理） |
| permutations() | p[, r] | r长度元组，所有可能的排序，没有重复的元素（排列） |
| combinations() | p, r | r长度元组，按排序顺序，没有重复元素（组合） |
| combinations_with_replacement() | p, r | r长度元组，按排序顺序，存在重复元素 |
| product('ABCD', repeat=2) | AA AB AC AD BA BB BC BD CA CB CC CD DA DB DC DD | |
| permutations('ABCD', 2) | AB AC AD BA BC BD CA CB CD DA DB DC | |
| combinations('ABCD', 2) | AB AC AD BC BD CD | |
| combinations_with_replacement('ABCD', 2) | AA AB AC AD BB BC BD CC CD DD |
2、
repeat(object[, times])
创建一个迭代器，它重复返回object对象，无穷尽地运行，除非指定了times参数。用作map()的参数，将不变参数映射到被调用函数。同时，用zip()来创建元组记录的不变部分。
def repeat(object, times=None):
# repeat(10, 3) --> 10 10 10
if times is None:
while True:
yield object
else:
for i in range(times):
yield object
cycle(iterable)
创建一个迭代器，它返回可迭代对象中的元素，并且保存每个可迭代对象中元素的副本，当可迭代对象中的元素被耗尽时，返回保存在副本中的元素。无穷无尽地重复这一行为。近似等价于：
def cycle(iterable):
# cycle('ABCD') --> A B C D A B C D A B C D ...
saved = []
for element in iterable:
yield element
saved.append(element)
while saved:
for element in saved:
count(start=0, step=1)
创建一个迭代器，它返回以start开始的均匀间隔的值。通常用作map()参数产生连续性的数据点。另外，用zip()来添加序列号，近似等价于：
def count(start=0, step=1):
# count(10) --> 10 11 12 13 14 ...
# count(2.5, 0.5) -> 2.5 3.0 3.5 ...
n = start
while True:
yield n
n += step
compress(data, selectors)
创建一个迭代器，它过滤data的元素，返回仅当selecors为True时相应的data中的元素，当data或selectors可迭代对象中的元素被耗尽时停止。近似等价于：
def compress(data, selectors):
# compress('ABCDEF', [1,0,1,0,1,1]) --> A C E F
return (d for d, s in zip(data, selectors) if s)
dropwhile(predicate, iterable)
创建一个迭代器，只要predicate为True就从可迭代对象中移除元素；然后返回每个元素。请注意，迭代器不产生任何输出，直到predicate第一次变成False，所以它可能有很长的启动时间。近似等价于：
def dropwhile(predicate, iterable):
# dropwhile(lambda x: x<5, [1,4,6,4,1]) --> 6 4 1
iterable = iter(iterable)
for x in iterable:
if not predicate(x):
yield x
break
for x in iterable:
yield x
takewhile(predicate, iterable)
创建一个迭代器，只要predicate为True就返回可迭代对象中的元素。近似等价于：
def takewhile(predicate, iterable):
# takewhile(lambda x: x<5, [1,4,6,4,1]) --> 1 4
for x in iterable:
if predicate(x):
yield x
else:
break
tee(iterable, n=2)
从单个可迭代对象中返回n个独立的迭代器，近似等价于：
def tee(iterable, n=2):
it = iter(iterable)
deques = [collections.deque() for i in range(n)]
def gen(mydeque):
while True:
if not mydeque: # when the local deque is empty
try:
newval = next(it) # fetch a new value and
except StopIteration:
return
for d in deques: # load it to all the deques
d.append(newval)
yield mydeque.popleft()
return tuple(gen(d) for d in deques)
filterfalse(predicate, iterable) --->filter
创建一个迭代器，它过滤可迭代对象中的元素，返回仅当prediccate为False的元素，如果predicate为None，返回条目为False的元素。近似等价于：
def filterfalse(predicate, iterable):
# filterfalse(lambda x: x%2, range(10)) --> 0 2 4 6 8
if predicate is None:
predicate = bool
for x in iterable:
if not predicate(x):
yield x
starmap(function, iterable) --->map
创建一个迭代器，它使用从可迭代对象中获取的参数计算函数。当参数已经从单个可迭代对象（数据已经被预压缩）中分组到元组中时，而不是使用map()。map()和starmap()之间的区别与函数(a,b)和函数(* c)之间的区别相对应。 近似等价于：
def starmap(function, iterable):
# starmap(pow, [(2,5), (3,2), (10,3)]) --> 32 9 1000
for args in iterable:
yield function(*args)
zip_longest(*iterables, fillvalue=None) --->zip
创建一个迭代器，它聚合每个可迭代对象的元素，如果可迭代对象长度不均匀，那么缺失值将填充为fillvalue。迭代继续直到最长的可迭代对象被耗尽，近似等价于：
class ZipExhausted(Exception):
pass def zip_longest(*args, **kwds):
# zip_longest('ABCD', 'xy', fillvalue='-') --> Ax By C- D-
fillvalue = kwds.get('fillvalue')
counter = len(args) - 1
def sentinel():
nonlocal counter
if not counter:
raise ZipExhausted
counter -= 1
yield fillvalue
fillers = repeat(fillvalue)
iterators = [chain(it, sentinel(), fillers) for it in args]
try:
while iterators:
yield tuple(map(next, iterators))
except ZipExhausted:
pass
如果其中一个可迭代对象可能是无限的，那么zip_longest()函数应该使用限制调用次数的东西（例如islice()或takewhile()）来包装。如果未指定，则fillvalue默认为None。
3、Itertools模块的配方
def take(n, iterable):
"Return first n items of the iterable as a list"
return list(islice(iterable, n)) def tabulate(function, start=0):
"Return function(0), function(1), ..."
return map(function, count(start)) def tail(n, iterable):
"Return an iterator over the last n items"
# tail(3, 'ABCDEFG') --> E F G
return iter(collections.deque(iterable, maxlen=n)) def consume(iterator, n):
"Advance the iterator n-steps ahead. If n is none, consume entirely."
# Use functions that consume iterators at C speed.
if n is None:
# feed the entire iterator into a zero-length deque
collections.deque(iterator, maxlen=0)
else:
# advance to the empty slice starting at position n
next(islice(iterator, n, n), None) def nth(iterable, n, default=None):
"Returns the nth item or a default value"
return next(islice(iterable, n, None), default) def all_equal(iterable):
"Returns True if all the elements are equal to each other"
g = groupby(iterable)
return next(g, True) and not next(g, False) def quantify(iterable, pred=bool):
"Count how many times the predicate is true"
return sum(map(pred, iterable)) def padnone(iterable):
"""Returns the sequence elements and then returns None indefinitely. Useful for emulating the behavior of the built-in map() function.
"""
return chain(iterable, repeat(None)) def ncycles(iterable, n):
"Returns the sequence elements n times"
return chain.from_iterable(repeat(tuple(iterable), n)) def dotproduct(vec1, vec2):
return sum(map(operator.mul, vec1, vec2)) def flatten(listOfLists):
"Flatten one level of nesting"
return chain.from_iterable(listOfLists) def repeatfunc(func, times=None, *args):
"""Repeat calls to func with specified arguments. Example: repeatfunc(random.random)
"""
if times is None:
return starmap(func, repeat(args))
return starmap(func, repeat(args, times)) def pairwise(iterable):
"s -> (s0,s1), (s1,s2), (s2, s3), ..."
a, b = tee(iterable)
next(b, None)
return zip(a, b) def grouper(iterable, n, fillvalue=None):
"Collect data into fixed-length chunks or blocks"
# grouper('ABCDEFG', 3, 'x') --> ABC DEF Gxx"
args = [iter(iterable)] * n
return zip_longest(*args, fillvalue=fillvalue) def roundrobin(*iterables):
"roundrobin('ABC', 'D', 'EF') --> A D E B F C"
# Recipe credited to George Sakkis
pending = len(iterables)
nexts = cycle(iter(it).__next__ for it in iterables)
while pending:
try:
for next in nexts:
yield next()
except StopIteration:
pending -= 1
nexts = cycle(islice(nexts, pending)) def partition(pred, iterable):
'Use a predicate to partition entries into false entries and true entries'
# partition(is_odd, range(10)) --> 0 2 4 6 8 and 1 3 5 7 9
t1, t2 = tee(iterable)
return filterfalse(pred, t1), filter(pred, t2) def powerset(iterable):
"powerset([1,2,3]) --> () (1,) (2,) (3,) (1,2) (1,3) (2,3) (1,2,3)"
s = list(iterable)
return chain.from_iterable(combinations(s, r) for r in range(len(s)+1)) def unique_everseen(iterable, key=None):
"List unique elements, preserving order. Remember all elements ever seen."
# unique_everseen('AAAABBBCCDAABBB') --> A B C D
# unique_everseen('ABBCcAD', str.lower) --> A B C D
seen = set()
seen_add = seen.add
if key is None:
for element in filterfalse(seen.__contains__, iterable):
seen_add(element)
yield element
else:
for element in iterable:
k = key(element)
if k not in seen:
seen_add(k)
yield element def unique_justseen(iterable, key=None):
"List unique elements, preserving order. Remember only the element just seen."
# unique_justseen('AAAABBBCCDAABBB') --> A B C D A B
# unique_justseen('ABBCcAD', str.lower) --> A B C A D
return map(next, map(itemgetter(1), groupby(iterable, key))) def iter_except(func, exception, first=None):
""" Call a function repeatedly until an exception is raised. Converts a call-until-exception interface to an iterator interface.
Like builtins.iter(func, sentinel) but uses an exception instead
of a sentinel to end the loop. Examples:
iter_except(functools.partial(heappop, h), IndexError) # priority queue iterator
iter_except(d.popitem, KeyError) # non-blocking dict iterator
iter_except(d.popleft, IndexError) # non-blocking deque iterator
iter_except(q.get_nowait, Queue.Empty) # loop over a producer Queue
iter_except(s.pop, KeyError) # non-blocking set iterator """
try:
if first is not None:
yield first() # For database APIs needing an initial cast to db.first()
while True:
yield func()
except exception:
pass def first_true(iterable, default=False, pred=None):
"""Returns the first true value in the iterable. If no true value is found, returns *default* If *pred* is not None, returns the first item
for which pred(item) is true. """
# first_true([a,b,c], x) --> a or b or c or x
# first_true([a,b], x, f) --> a if f(a) else b if f(b) else x
return next(filter(pred, iterable), default) def random_product(*args, repeat=1):
"Random selection from itertools.product(*args, **kwds)"
pools = [tuple(pool) for pool in args] * repeat
return tuple(random.choice(pool) for pool in pools) def random_permutation(iterable, r=None):
"Random selection from itertools.permutations(iterable, r)"
pool = tuple(iterable)
r = len(pool) if r is None else r
return tuple(random.sample(pool, r)) def random_combination(iterable, r):
"Random selection from itertools.combinations(iterable, r)"
pool = tuple(iterable)
n = len(pool)
indices = sorted(random.sample(range(n), r))
return tuple(pool[i] for i in indices) def random_combination_with_replacement(iterable, r):
"Random selection from itertools.combinations_with_replacement(iterable, r)"
pool = tuple(iterable)
n = len(pool)
indices = sorted(random.randrange(n) for i in range(r))
return tuple(pool[i] for i in indices)