Python · iterators & generators

Iterators

The protocol behind every for loop: iter() makes a cursor, next() moves it, StopIteration says it's done.

The iterator protocol


          # get iterator
          numbers = [1, 2, 3]
          iterator = iter(numbers)
          # call next
          next(iterator)
          # ans: 1
          next(iterator)
          # ans: 2
          next(iterator)
          # ans: 3
          next(iterator)
          # ans: StopIteration
        

The iterator remembers its position; the list itself is unchanged.

A for loop uses iterators


          # for loop = iterator protocol
          for num in [1, 2, 3]:
              print(num)
        

          # equivalent to:
          iterator = iter([1, 2, 3])
          while True:
              try:
                  num = next(iterator)
                  print(num)
              except StopIteration:
                  break
        

A custom iterator


          # iterator class
          class Counter:
              def __init__(self, start, end):
                  self.current = start
                  self.end = end
              def __iter__(self):
                  return self
              def __next__(self):
                  if self.current >= self.end:
                      raise StopIteration
                  self.current += 1
                  return self.current - 1

          for num in Counter(0, 3):
              print(num)
          # ans: 0, 1, 2
        

Any object with __iter__ and __next__ works in a for loop.

Iterable vs iterator


          # iterable: has __iter__()
          # iterator: has __iter__() and __next__()

          # list is iterable but not iterator
          numbers = [1, 2, 3]
          iter(numbers)
          # ans: works
          next(numbers)
          # ans: TypeError
          # iterator is both
          iterator = iter(numbers)
          iter(iterator)
          # ans: returns itself
          next(iterator)
          # ans: 1
        
numbersiterable: __iter__ iterator__iter__ + __next__ iter() ✕next(numbers)TypeError iter() → itself next()1

Exercises

Predict the answer, then press → to check.

Exercise 1


        # what is iter()?
        lst = [1, 2, 3]
        it = iter(lst)
        type(it)
        # ans: list_iterator
      

Exercise 2


        # next() on list?
        lst = [1, 2, 3]
        next(lst)
        # ans: TypeError (a list is not an iterator)
      

Exercise 3


          # StopIteration when?
          it = iter([1, 2])
          next(it)
          # ans: 1
          next(it)
          # ans: 2
          next(it)
          # ans: StopIteration
        

Exercise 4


          # iterator exhaustion?
          it = iter([1, 2])
          list(it)
          # ans: [1, 2]
          list(it)
          # ans: [] (exhausted)
        

Exercise 5


        # iter on iterator?
        it = iter([1, 2, 3])
        iter(it) is it
        # ans: True (returns self)
      

Exercise 6


        # range is iterator?
        r = range(3)
        next(r)
        # ans: TypeError (range is iterable, not an iterator)
        iter(r)
        # ans: works (creates an iterator)
      

Exercise 7


          # string is iterable?
          s = "abc"
          for c in s:
              print(c)
          # ans: a, b, c
        

Exercise 8


          # dict iterator?
          d = {"a": 1, "b": 2}
          it = iter(d)
          next(it)
          # ans: "a" (iterates keys)
        

Exercise 9


        # custom __iter__ only?
        class MyIterable:
            def __iter__(self):
                return iter([1, 2, 3])

        for x in MyIterable():
            print(x)
        # ans: 1, 2, 3
      

An iterable only has to give out an iterator; here it borrows the list's.

Exercise 10


        # infinite iterator?
        class InfiniteCounter:
            def __init__(self):
                self.count = 0
            def __iter__(self):
                return self
            def __next__(self):
                self.count += 1
                return self.count
        # ans: never raises StopIteration
      

So a for loop over it needs a break.

Remember

Iterable

Can make an iterator: iter(x) works. Lists, strings, ranges, dicts.

Iterator

Hands out values with next() and remembers where it is.

StopIteration

Signals the end; a for loop catches it and stops.

One pass

An exhausted iterator stays empty; call iter() again for a fresh one.