Possible differences between list and iterator


I have a weird problem with iterators, which I can't figure out. I have a complicated numerical routine returning a generator object (or after some changes to the code an islice). Afterwards I check, the results as I know that the results must have a negative imaginary part:

import numpy as np
threshold = 1e-8  # just check up to some numerical accuracy

results = result_generator(**inputs)
is_valid = [np.all(_result.imag < threshold) for _result in results]
print("Number of valid results: ", is_valid.count(True))

(Sorry for not giving an executable code, but I can't come up with a simple code at the moment.) The problem is now, that this returns one valid solution. If I change the code to

import numpy as np
threshold = 1e-8  # just check up to some numerical accuracy

results = list(result_generator(**inputs))
is_valid = [np.all(_result.imag < threshold) for _result in results]
print("Number of valid results: ", is_valid.count(True))

using a list instead of a generator, I get zero valid solution. I can however not wrap my head around what is different and thus have no idea how to debug the problem. If I go through the debugger and print out the result with the corresponding index the results are even different, the one of the generator is correct, the one of the list is wrong.


Here the numerical function:

def result_generator(z, iw, coeff, n_min, n_max):
    assert n_min >= 1
    assert n_min < n_max
    if n_min % 2:
        # index must be even
        n_min += 1

    id1 = np.ones_like(z, dtype=complex)
    A0, A1 = 0.*id1, coeff[0]*id1
    A2 = coeff[0] * id1
    B2 = 1. * id1

    multiplier = np.subtract.outer(z, iw[:-1])*coeff[1:]
    multiplier = np.moveaxis(multiplier, -1, 0).copy()

    def _iteration(multiplier_im):
        multiplier_im = multiplier_im/B2
        A2[:] = A1 + multiplier_im*A0
        B2[:] = 1. + multiplier_im

        A0[:] = A1
        A1[:] = A2 / B2
        return A1

    complete_iterations = (_iteration(multiplier_im) for multiplier_im in multiplier)
    return islice(complete_iterations, n_min, n_max, 2)

Answer

You're yielding the same array over and over instead of making new arrays. When you call list, you get a list of references to the same array, and that array is in its final state. When you don't call list, you examine the array in the state the generator yields it, each time it's yielded.

Stop reusing the same array over and over.