What is Epoch?

Epoch is one of the most basic units of measurement in machine learning training. When you train a model on a dataset with one million examples and a batch size of 100, each epoch consists of 10,000 parameter updates, one for each batch. After one epoch, every training example has been seen once. After ten epochs, each example has been seen ten times. The model uses all of these exposures to progressively refine its parameters through backpropagation and gradient descent.

The number of epochs is a critical hyperparameter that must be tuned carefully. Too few epochs and the model has not had enough exposure to the training data to learn the underlying patterns, a condition called underfitting. Too many epochs and the model begins to memorize the training data rather than generalizing from it, a condition called overfitting. The training and validation loss curves plotted over epochs are the primary diagnostic tool for identifying when to stop training: training loss decreases while validation loss begins to increase is the hallmark signal of overfitting onset.

Early stopping is the standard technique for avoiding overfitting during multi-epoch training. The trainer monitors the validation loss after each epoch and stops training when validation performance stops improving, even if training loss continues to decrease. This automatically selects the optimal number of epochs without requiring manual tuning, and the model weights from the epoch with the best validation performance are saved as the final model. Early stopping is a standard practice in both research and production training workflows.

For very large models and datasets, a single epoch may itself take days or weeks even on large GPU clusters. Frontier language models are sometimes trained for less than one epoch on their full pre-training corpus because the dataset is so enormous that a single pass provides sufficient learning signal. In this regime, other measures like tokens processed or compute FLOPs become more meaningful than epoch count as measures of training progress. This highlights how the practical meaning of 'epoch' depends heavily on the scale of the training job.