This weeks PyPuzzle will test your knowledge of order of generator functions and the yield keyword.

  • Generator functions
  • yield keyword
  • Iterators and lazy evaluation

Feel free to use an online Python compiler and interpreter like [this] (https://www.online-python.com/) to try running the code yourself. The answer is supplied below the code.

Question

What is the expected output of the following code? How does the yield keyword affect the function’s behaviour and why doesn’t it behave like a typical return?

def my_generator(n):
    print("Generator started")
    for i in range(n):
        yield i
        print(f"Yielded {i}, pausing generator")

# Initialize the generator function
gen = my_generator(3)

# Step through the generator
print(next(gen))
print(next(gen))
print(next(gen))
print(next(gen))  # What happens here?

Hints

  1. What is the difference between yield and return? Consider how many times yield allows the function to pause and resume.
  2. Try to visualize what the function does with each call to next(gen). How does my_generator() know where to pick up after yielding a value?
  3. What might happen when next(gen) is called, but there are no more items to yield?

Answer

  • The first call to next(gen) starts the generator, which prints “Generator started", yields 0, and then pauses.
  • Each subsequent next(gen) call resumes from where it left off, yielding the next value in the sequence (1, then 2), and printing a message after each yield.
  • When next(gen) is called a fourth time, there are no more values to yield, so Python raises a StopIteration exception, indicating that the generator has been exhausted.

Expected output:

Generator started
0
Yielded 0, pausing generator
1
Yielded 1, pausing generator
2
Yielded 2, pausing generator

Learnings

  • Generator Functions and yield: Using yield in a function turns it into a generator function. Instead of returning a single result and terminating, it yields multiple values, pausing between each, and can resume execution each time it’s called.
  • Lazy Evaluation: Generators allow Python to produce values on the fly, making them memory-efficient for large data sequences, as values are generated only when needed. This is also beneficial for asynchronous tasks that are input/output (I/O)-bound as you can yield control back to an event loop once the required resources are ready.
  • StopIteration Exception: When a generator has no more values to yield, calling next() raises a StopIteration exception, signaling that iteration has completed. This is handled automatically when used in a loop, like a for loop.
  • Generator functions also make it possible to produce infinite sequences (like Fibonacci and prime numbers) without risking memory overload. You can produce as much as you like from an infinite sequence without actually storing the entire sequence.
  • An analogy is like streaming a TV show. You don’t have to load all the episodes from every season of the TV show when you start. You simply load the first episode and start watching that. If you need to pause it and leave for a while, that is fine, if you keep watching and want to watch the second episode, then simply load it once the first one finishes.