Very Large Datasets in PyTorch (opens in new tab)

In God we trust. All others must bring data. ~ W. Edwards Deming Datasets that fit in memory For simple machine learning problems, your PyTorch dataset class probably looks something like this: class SimpleDataset(Dataset): def __init__(self, features, targets): self.features = [] for feature in features: self.features.append(self._feature_transform(feature)) self.targets = targets def _feature_transform(self, feature): # Optional feature transformation function which # converts each feature ...