Skip to content

Loader.__len__ is not a pure function #163

@dpaleka

Description

@dpaleka

Counterintuitively, calling len() on a ffcv.loader.Loader instance modifies its internal state:

ffcv/ffcv/loader/loader.py

Lines 253 to 258 in bfd9b3d

def __len__(self):
next_order = self.next_traversal_order()
if self.drop_last:
return len(next_order) // self.batch_size
else:
return int(np.ceil(len(next_order) / self.batch_size))

This causes the bug in #140 when paired with OrderOption.QUASI_RANDOM.
I suppose the culprit is this line:

random.shuffle(page_to_samples_array[page_ix, :page_size])

Metadata

Metadata

Type

No type

Projects

No projects

Relationships

None yet

Development

No branches or pull requests

Issue actions