strange limitation/warning when using pydataset?

[ghsanti]

/usr/lib/python3.10/contextlib.py:153: UserWarning: Your input ran out of data; interrupting training. Make sure that your dataset or generator can generate at least steps_per_epoch * epochs batches.

However, it would be discounted that when steps_per_epoch batches have been seen, the generator would be called again, I guess it's called just once? But, why?

MWE:

import keras

# define our data loader
class MyDataset(keras.utils.PyDataset):
    def __getitem__(self, item):
        return keras.random.normal((32, 1)), keras.random.normal((32, 10))

    @property
    def num_batches(self):
        return 4

i = keras.Input((1,))
x = keras.layers.Dense(10)(i)
m = keras.models.Model(i, x)
m.compile(loss=keras.losses.MeanSquaredError(), optimizer="adam")


dataset = MyDataset(workers=1, use_multiprocessing=False)
m.fit(dataset, epochs=10, steps_per_epoch=3)

OFC, it won't happen using m.fit(dataset, epochs=2, steps_per_epoch=1)

[ghsanti]

Actually it does make sense as it is. You create just one iterator, once, and goes over all the data. So it needs to have more than the amount indicated by the msg.

Closing.