The Puzzle of Overparameterization

The classical picture

Start with the picture every statistics course draws, because the whole article is about the moment it stops being true.

You have a distribution D over pairs (x, y), and a sample S of n pairs drawn from it. You pick a predictor f from some class by minimizing the average loss on the sample. Two numbers matter, and only one of them is observable.

Training minimizes the first. You are judged on the second. Everything in learning theory is an attempt to bound the difference using quantities you can compute.

For squared loss there is an exact decomposition that makes the tension visible. Suppose the labels are a clean function plus noise, y = g(x) + ε with E[ε] = 0 and Var(ε) = σ². Let fS be the predictor your procedure returns when trained on sample S, and take expectations over the draw of S. At any fixed input x:

The first term is a floor nothing can beat. The second asks whether your class can express the truth on average. The third asks how much your fitted function jitters when the sample is redrawn. Bias is an error you make consistently; variance is an error you make differently every time.

Now the classical claim. As you add capacity, bias falls, because a richer class can track the truth more closely. Variance rises, because a richer class can also track the noise. Their sum has a minimum somewhere in the middle, and that minimum is the sweet spot.

Those curves are real least-squares fits, not sketches. The pattern is exactly what the textbook promises: degree 1 misses the shape identically every time, degree 12 wraps itself around whichever eleven noise realizations it happened to see, and the sum bottoms out around degree 4.

The theory backing this up is uniform convergence. Bound the worst-case gap over the entire class at once, and you get statements shaped like

where complexity is VC dimension, or Rademacher complexity, or a covering number. The shape is what matters: a class rich enough to fit anything has enormous complexity, so the bound is enormous, so it tells you nothing. That is a feature. It is supposed to be the mathematical statement of "a model flexible enough to fit anything has learned nothing".

Now put a number in it. The VC dimension of a ReLU network with W weights and L layers is Θ(W L log W). Take a small ResNet with 11 million parameters and 20 layers, trained on CIFAR-10's 50,000 images. The numerator is around 5×10⁹, the ratio is 10⁵, and the bound comes out near 300. Test error is a probability, so it cannot exceed 1. The bound is not wrong — it is vacuous: true, and carrying no information.

Language models are worse, not better, because the classical machinery does not even apply cleanly: cross-entropy is unbounded, so the concentration arguments behind these bounds need replacing before you can start. We come back to what does work in section 12.

You could reasonably say that this only shows the bound is loose, and that real networks are effectively much simpler than their parameter count suggests. That was the standard defence for years. Then someone checked.

Networks can memorize almost anything

The experiment is embarrassingly simple. Take CIFAR-10. Before training, replace every label with a uniformly random one. The images no longer carry any information about their targets: the best achievable test accuracy is 10%, chance. Now train a standard convolutional network on it, exactly as you otherwise would.

Zhang, Bengio, Hardt, Recht and Vinyals did this in 2016 and reported that ordinary image classifiers, trained by ordinary stochastic gradient methods, have no trouble at all fitting a randomly relabelled training set. Training error goes to zero. Not "low" — zero. Then they pushed further:

• Random pixels. Replace the images with pure Gaussian noise, keep random labels. Still fits.
• Explicit regularization. Weight decay, dropout and data augmentation are all present in the standard recipe. Removing them changes nothing qualitative about the result. Turning regularizers off costs a few points of test accuracy on real labels — it does not move the model from "generalizes" to "memorizes".
• Scale. On ImageNet with 1.2 million random labels across 1,000 classes, an Inception network still reached 95.20% top-1 accuracy on its own scrambled training set, and the authors report doing no hyperparameter tuning at all when they switched over from true labels.

And the cost of memorizing is mild. Fitting random labels takes a small constant factor more steps than fitting real ones, not an exponential blowup. The network is not straining.

The conceptual payload is one line, and it deserves to be stated on its own:

The same architecture, the same optimizer, the same number of parameters, the same sample size, the same regularizers. Change nothing but the labels and you move from 6% test error to 90% test error. Whatever explains good generalization on real data, it cannot be a property of the model class alone, because the model class is identical in both runs.

That kills a whole family of explanations at once. Any bound of the uniform-convergence shape — anything that holds for every function in the class — must be at least as large as the gap achieved by the worst function in the class. Random labels exhibit a function in the class with a gap near 0.9. So the bound is at least 0.9, whatever the real labels do. Nagarajan and Kolter later sharpened this into a formal argument that uniform convergence, even restricted to the algorithm's own output set, may be unable to explain generalization in these settings.

The interpolation threshold

Fix vocabulary. A model interpolates its training set when it drives training loss to (approximately) zero — when it reproduces every training label exactly. The interpolation threshold is the point, as you scale capacity up, where this first becomes possible.

For linear models the threshold is a counting fact. You have n constraints and p free parameters:

That middle line is where classical statistics expects catastrophe, and its intuition is sound. At p = n the system is square: there is exactly one function that threads every point, and it has no freedom left to be sensible. If two training inputs are nearly identical with different noise, the fit must swing violently between them. The matrix you invert is near-singular, the coefficients blow up, and the function between the data points goes wherever the algebra sends it.

The third line is where the classical intuition simply stops applying, and it is worth being precise about why. Once p > n, "fit the training data" no longer picks out a function. It picks out an infinite family of functions, all with identical training loss and wildly different test loss. The question which function does my model learn? is no longer answered by the loss at all.

Two refinements before moving on, because both get misused later.

Parameter counting is the wrong axis. A model that is not trained to convergence has not used all its parameters. Nakkiran and colleagues defined effective model complexity as the largest sample size on which the training procedure — architecture, optimizer, schedule, all of it — still reaches approximately zero training error. That is a property of the procedure, not of the architecture, and it is the axis along which the phenomena in the next section actually organize themselves.

Interpolation is a choice, not a destination. Large language models are typically trained for a single pass over a corpus far larger than the model. They do not interpolate their training set and they are not trying to. Almost everything in the next four sections is about a regime that LLM pretraining does not occupy. That does not make it useless — it is where our sharpest results live — but section 11 is where the two threads finally have to be reconciled.

Double descent

In 2019 Belkin, Hsu, Ma and Mandal pointed out that the textbook curve is not wrong, it is truncated. It stops drawing at the interpolation threshold. Keep going, and the test error comes back down — often below the classical sweet spot.

The shape has a name and a picture:

Before explaining anything, it helps to know exactly where the spike comes from, because in the linear case it is not mysterious at all. Consider fitting p random features to n points and taking the minimum-norm solution. The fitted coefficients involve inverting the feature Gram matrix, and its smallest eigenvalue controls everything.

At the threshold the model is forced to use every one of its degrees of freedom to hit every point exactly, and it has none left over to be small with. Past the threshold it has room to choose — and "choose the smallest solution" turns out to be a good choice. The peak is a conditioning failure, not a philosophical statement.

The following three figures are computed live in your browser: forty noisy samples from a fixed smooth function, random ReLU features, ridge-regularized least squares solved exactly. No curve here is drawn by hand.

Watch the fitted curve, not just the error plot. Just below the threshold the fit develops enormous excursions between data points — it is threading every sample with no slack. Just above, the excursions collapse.

Nakkiran et al. showed the same non-monotonicity appears along two more axes, which is what makes it interesting rather than a curiosity of counting features.

Epoch-wise is the one that should genuinely unsettle you, because nothing about the model changed. The same parameters, the same data, more steps. Gradient descent on a least-squares objective learns along the eigendirections of the feature covariance at rates proportional to their eigenvalues: big, signal-carrying directions are fitted in the first few steps, and the tiny directions — the ones where the noise lives — are fitted last. So "more training" is a capacity dial, and effective model complexity grows with time. That is exactly Nakkiran's point, and it is why early stopping and model size are two views of one knob.

Sample-wise non-monotonicity is the most counterintuitive of the three and the easiest to explain once you see the axis correctly. Adding data does not move the model; it moves the threshold. A model that was comfortably overparameterized at n = 30 is sitting right on the cliff at n = 60. More data is only unambiguously good if it does not drag you into the badly conditioned regime on the way.

What double descent does not explain

Double descent is real, reproducible, and much less explanatory than its fame suggests. Four things are worth saying plainly.

The peak is a fragility, not a feature

Nothing about the second descent requires the first ascent. Nakkiran, Venkat, Kakade and Ma proved that for linear regression with isotropic data, optimally tuned ridge regularization makes test error monotonic in both model size and sample size — the peak vanishes entirely — and demonstrated the same mitigation empirically on neural networks. The spike is what happens when a badly conditioned problem is solved without regularization. It is a symptom, and it has a cure.

Turn the ridge dial yourself:

Set noise to zero and the peak nearly disappears. Raise the ridge to its best value at each point and the curve becomes a clean, boring, monotone descent. Both facts should temper any story in which double descent is the reason large models work.

Some reported cases are an artifact of the x-axis

Curth, Jeffares and van der Schaar re-examined the classical (non-neural) examples — linear regression with increasing features, trees, boosting — and showed that in these cases the "second descent" appears because the horizontal axis silently switches from one notion of complexity to another at the interpolation threshold. Plot the same experiments against a smoothly varying effective complexity and you recover a single U-shape per axis, with two axes being traversed in sequence. Their conclusion is not that double descent is fake; it is that parameter counting was the problem, and that the classical theory survives if you stop counting raw parameters.

It leans on label noise

In the deep-learning experiments the peak is small or absent with clean labels and becomes dramatic at 10–20% label corruption. That is consistent with the conditioning story: the spike is the model straining to reproduce values that carry no signal. A dataset whose labels are mostly right does not put much stress on the threshold.

It describes a curve, not a mechanism

Even granting every experiment, double descent tells you that test error falls again past interpolation. It does not tell you why the interpolating solution it lands on is a good one. That is the actual question, and answering it requires knowing which interpolant the procedure selects — which is where the next two sections go.

Benign overfitting

Here is the sharpest version of the question. Forget curves. Can a predictor that fits noisy training data exactly have near-optimal risk, provably, in a setting we fully understand?

Yes. The setting is linear regression, and the analysis is worth walking through because the mechanism transfers as intuition even where the theorem does not.

Take y = ⟨β*, x⟩ + ε with x drawn from a Gaussian with covariance Σ, and let p be much larger than n so the system is underdetermined. Among the infinitely many β that fit the sample exactly, take the one of smallest norm:

The interpolation constraint means the noise must go somewhere. The predictor cannot average it away, because averaging it away would mean not fitting the training points. So the entire question of whether overfitting is harmful becomes: where does the absorbed noise land, and does anyone look there?

If the covariance has a few large eigenvalues and a long tail of tiny ones, the answer is encouraging. Test predictions are dominated by the large-eigenvalue directions, since that is where test inputs actually have mass. The min-norm solution, having many directions to work with, distributes each noise term across a huge number of tail directions, putting a vanishing amount into any one of them. The training points are hit exactly. Test predictions barely notice.

Bartlett, Long, Lugosi and Tsigler turned this into an exact characterization. They define two notions of effective rank of the covariance — roughly, how many directions carry meaningful variance, and how evenly the tail is spread — and prove that the minimum-norm interpolant is near-optimal precisely when the tail is large and flat enough. Their summary of the condition is the memorable part: overparameterization is not incidental but required, and the number of directions that are unimportant for prediction has to substantially exceed the sample size.

The spectrum has to be tuned surprisingly finely. A tail that decays too fast, like λk ∝ k−α with α > 1, does not have enough directions to hide noise in and overfitting is harmful. A tail that decays slowly enough, like λk ∝ k−1log−2(k+1), is benign. Benign overfitting is a property of the data distribution as much as the estimator.

The algorithm picks the solution

Past the interpolation threshold the training objective stops being a selection rule. Every point on the solution manifold has the same loss. Something else decides which one you get, and the only remaining candidate is the optimizer.

There is a beautiful, fully rigorous instance of this. Take logistic regression on linearly separable data, with no regularization at all, and run plain gradient descent.

Notice first that the problem is pathological in the classical sense: the loss has no minimizer. Any separating direction can be scaled up to drive the loss lower, so ||w|| grows without bound and the objective decreases forever. Textbook advice would be to add a penalty or stop early. Instead, ask what happens if you just keep going.

Soudry, Hoffer, Nacson, Gunasekar and Srebro proved that the direction converges, and converges to something specific:

Nobody wrote a margin term in the objective. Nobody added a norm penalty. Gradient descent, run on an unregularized loss, selects the max-margin interpolant on its own. The optimization algorithm is itself a form of regularization.

The rates make a practical prediction that is easy to check and frequently observed. Training loss reaches numerically-zero long before the direction has converged, because 1/log t is agonizingly slow: to halve the distance to the max-margin solution you square the number of steps. So a model can keep getting better on test data for a very long time after the training loss has flatlined, and it can do so while the validation loss is rising, because the growing norm inflates confidence on the points it gets wrong.

Now the part that makes this more than a cute fact about SVMs. The bias belongs to the specific algorithm, not to "optimization" in general. Run steepest descent with respect to a different norm and you converge to the max-margin solution in that norm: coordinate descent and sign-based updates head toward sparse, ℓ1-flavoured solutions rather than the ℓ2 one. Soudry and colleagues point this out explicitly for adaptive methods like Adam.

Which should give you pause. Every large model in production is trained with an adaptive optimizer and decoupled weight decay. If the implicit bias is the thing doing the regularizing, then the choice of optimizer is not a convergence-speed decision — it is a decision about which function you end up with.

You can watch a smaller version of that decision being made. The lab below runs the same gradient descent on the same separable data at seven weight-decay settings. All seven reach zero training error within a few hundred steps and stay there. They do not agree on the answer.

The results have been pushed beyond linear models — Lyu and Li showed that homogeneous deep networks trained on exponential-tailed losses converge in direction to a KKT point of a margin-maximization problem — but "homogeneous" excludes most of what a transformer contains, and nobody has an implicit-bias characterization for AdamW on a decoder-only network with layer norm. What we have is a proof of concept that the question which interpolant? has a real answer, in cases small enough to solve.

Parameter space is not function space

A recurring source of confusion is that "number of parameters" is being used as a proxy for "number of distinguishable behaviours", and the two are separated by enormous redundancy.

Two symmetries make this concrete. Permute the hidden units of any layer, permuting the corresponding weight columns and rows to match, and the network computes exactly the same function: a layer of width m contributes m! identical copies of every solution. For a width-4096 layer, that factor alone is beyond astronomical. Separately, in a ReLU network you can multiply one layer's weights by c > 0 and divide the next layer's by c, again with no change to the function — a continuous, unbounded family of parameter vectors sitting on one function.

This has a sharp consequence for one of the most popular folk explanations. "Flat minima generalize, sharp minima do not" is intuitive: a flat basin means the function is insensitive to its parameters, which sounds like robustness. But Dinh, Pascanu, Bengio and Bengio observed that the rescaling symmetry above lets you take any minimum and make it arbitrarily sharp or arbitrarily flat — while computing the identical function on every input. Sharpness measured in raw parameter coordinates is not a property of the learned function at all, so it cannot by itself explain a property of the learned function.

Flatness-based reasoning is not dead; it is repaired by measuring sharpness in a reparameterization-aware way, which is what modern versions do. The lesson is narrower and more useful: any proposed explanation of generalization should be invariant to the symmetries of the parameterization, and a surprising number of proposals are not.

Lazy and rich: what the infinite-width limit buys

If finite networks are hard, take a limit and hope it becomes tractable. Widen every layer toward infinity and something remarkable happens.

Write the first-order expansion of the network output around initialization:

Jacot, Gabriel and Hongler showed that under the standard parameterization, as width goes to infinity, K converges to a deterministic kernel that depends only on the architecture — and, crucially, stays constant throughout training. Each parameter moves an amount that shrinks with width, so the gradients never change, so the expansion above never stops being accurate.

The consequences are dramatic. Training becomes kernel gradient descent in a fixed feature space, the optimization problem becomes convex, the solution has a closed form, and the whole apparatus of kernel theory — including the benign-overfitting analysis from section 6 — becomes applicable. A genuinely nonlinear, non-convex object turned into a linear one.

It also throws away the thing everyone cares about. In the infinite-width limit, ∇θf(x; θ0) is the feature map, and it is fixed at initialization. The network does not learn representations. It fits a linear model on top of random features that happen to be arranged in a clever architecture-dependent way. Whatever a language model is doing when it develops an internal notion of syntax, or entity tracking, or arithmetic, is exactly the part the NTK limit removes.

The two regimes have names. Lazy training is the NTK picture: parameters barely move, features stay put, the model is effectively linear. Rich (or feature-learning) training is what practitioners want: the internal representations reorganize around the task. Chizat and Bach showed that laziness is not a consequence of width as such but of the scaling of the parameterization, and that a model can be pushed into either regime by choosing that scaling. Yang and Hu's maximal-update parameterization is the constructive version: a width scaling under which features continue to move even in the infinite-width limit — and, as a side effect, under which hyperparameters transfer across widths, which is why it appears in real training recipes.

Grokking

Everything so far has been about a static question: given a solution, is it good? Grokking is about a dynamic one, and it is strange enough to deserve its own section.

Train a small transformer on modular arithmetic — say (a + b) mod 113, with a fraction of the pairs held out. The training curve does the obvious thing: it memorizes the training pairs and hits 100% accuracy quickly. Validation accuracy sits at chance. Ordinarily you would stop here and report that the model overfit.

Power, Burda, Edwards, Babuschkin and Misra kept training. For orders of magnitude more steps, nothing happens. Then validation accuracy climbs from chance to nearly perfect, in a few hundred steps.

The jump is not as sudden as the accuracy curve suggests. Nanda, Chan, Lieberum, Smith and Steinhardt reverse-engineered exactly what a one-layer transformer learns on this task: it embeds the inputs onto rotations, and computes the answer using discrete Fourier components and trigonometric identities — genuine modular arithmetic, not a lookup table. Tracking the strength of that specific circuit directly, rather than watching accuracy, reveals it growing smoothly through the plateau. They split training into three phases: memorization, circuit formation, and cleanup, where the memorizing solution is finally discarded. Accuracy is a lagging indicator of a continuous process.

Varma, Shah, Kenton, Kramár and Kumar proposed the mechanism as a competition between two circuit families. Memorizing circuits are learned quickly but are inefficient: producing confident logits by memorization requires parameter norm that grows with the number of examples memorized. Generalizing circuits are slower to form but achieve the same logits at a fixed, dataset-size-independent norm. Weight decay applies constant downward pressure on norm, so once the dataset is large enough that memorization is the more expensive option, the generalizing circuit wins — and the visible transition is the cleanup that follows. The account makes two falsifiable predictions, both confirmed: ungrokking, where a grokked network sent back to a small dataset regresses to memorization, and semi-grokking, where a dataset near the critical size produces a transition to middling test accuracy.

It is not settled. Kumar, Bordelon, Gershman and Pehlevan exhibit grokking with no weight decay at all, in a setting where the weight norm increases throughout training, and argue the phenomenon is better understood as a delayed transition from lazy to rich dynamics — connecting it directly to section 9. Later work has produced grokking driven by numerical effects near the floating-point floor. Multiple mechanisms appear able to produce the same curve.

The regime language models actually train in

Time to reconcile. Almost every result above concerns a model trained to convergence on a fixed, modest dataset, often with deliberately corrupted labels. Pretraining looks nothing like that, in three specific ways.

Each example is seen roughly once

A frontier pretraining run makes approximately one pass over its corpus. This has an underappreciated consequence: under a single pass, the training loss is an unbiased estimate of the population loss. Each gradient step is computed on data the model has never seen, so the loss reported on the current batch is a fresh-sample estimate of exactly the quantity you care about. The classical generalization gap — the thing all of learning theory exists to bound — is close to zero by construction, and this is visible in practice: pretraining loss and held-out loss on the same distribution track each other closely.

The caveats are real but bounded. Corpora contain near-duplicates, so some sequences effectively get many passes. Repeating data deliberately is common when tokens are scarce, and Muennighoff and colleagues found that up to about four epochs the loss is nearly indistinguishable from training on fresh tokens, while past roughly sixteen epochs the returns go to essentially nothing. Somewhere between those two numbers, the classical overfitting story reappears.

The binding constraint is bits, not parameters

The interesting limit is not "parameters versus examples". It is "how much information can the weights hold, versus how much information is in the corpus".

Morris and colleagues measured this directly, by separating what a model stores about a specific dataset from what it stores about the data-generating process. Training hundreds of transformers from 500K to 1.5B parameters, they estimate the capacity of GPT-style models at approximately 3.6 bits per parameter, remarkably stable across scales. Doubling the weight precision from 16 to 32 bits moves it only to about 3.8, so the capacity is a property of the architecture and training rather than of the storage format.

Put that number against a real corpus:

The model is not choosing to generalize over memorizing. It is physically unable to memorize, by nearly three orders of magnitude, and the only way to reduce loss on data it cannot store is to find structure that predicts it. Generalization is not a happy accident here — it is the only strategy the budget permits.

Morris and colleagues also observe the transition directly: models memorize until their capacity fills, and past that point unintended memorization decreases as the model shifts to generalizing. They note that double descent shows up in their setting exactly when the dataset outgrows capacity — which places the interpolation peak of section 4 at the boundary between the two regimes, and firmly on the far side of it from any frontier model.

Drag the parameter count up and the token count down and you can reach the memorization regime — which is exactly what happens when you fine-tune a large model on a small dataset for many epochs, and exactly why that setting leaks training data while pretraining largely does not.

Memorization is real, but concentrated

None of this says memorization is absent. Carlini and colleagues showed that extractable memorization grows with model size, with how many times a sequence is duplicated in the corpus, and with how much context you use to prompt for it. The picture that fits both results: a small, high-entropy, heavily duplicated slice of the corpus gets stored more or less verbatim, while the overwhelming majority of it is compressed into structure. Deduplication is not a nicety; it is the lever that controls which slice.

And the question quietly changed

All of this concerns generalization within the pretraining distribution, which for a single-pass run is nearly a solved problem for uninteresting reasons. What people actually mean when they ask whether an LLM generalizes is something else: does it work on a distribution it was not trained on, does it compose skills it learned separately, does it follow a rule it has only seen instances of. Almost none of the theory in this article speaks to that. It is a genuinely different question, and it is where the later parts of this series live.

What can actually be certified

Given all the negative results, it is worth asking what we can prove about a specific trained model. The answer is more than you would expect, and it arrives from an unexpected direction.

First, the negative result that should calibrate expectations. Jiang and colleagues ran a large-scale controlled study, training thousands of models across systematically varied hyperparameters and evaluating more than forty proposed complexity measures against actual generalization. Many popular norm-based measures correlated poorly, and several correlated in the wrong direction once confounders were controlled for. Sharpness-flavoured and PAC-Bayes-flavoured measures did best, but nothing emerged as a reliable predictor.

What does work is compression. The argument is old and clean: if a model can be described in b bits and still fits the training data, then it was drawn from a set of at most 2b possible models, and a union bound over that finite set gives a real guarantee. The tighter you can compress the trained model without breaking it, the stronger the statement.

Lotfi and colleagues pushed this all the way to language models, which required handling three problems at once: token predictions within a sequence are not independent, the bits-per-dimension objective is unbounded so standard PAC-Bayes does not apply, and the models are enormous. Their solution combines a bound valid for unbounded log-likelihood via prediction smoothing, a subsampling extension that made the computation roughly 900× faster on a nine-billion-token corpus, and a heavily compressed training parameterization. The result was the first non-vacuous generalization bounds for pretrained LLMs.

Two findings from that work are worth carrying forward. Larger models had better bounds than smaller ones, which is the opposite of what parameter counting predicts and consistent with larger models being more compressible. And bounds computed on models trained with shuffled tokens came out worse — the bound can tell the difference between a model that found structure and one that could not.

Read the fine print, though. These bounds are proved for compressed or quantized variants of the model rather than the deployed weights, they require restricting to a finite hypothesis set, and non-vacuous is a much weaker word than tight — a bound of 0.9 on a quantity whose true value is 0.3 has cleared the bar. What they establish is a proof of existence: it is possible to make a rigorous, non-trivial statement about a billion-parameter model. That was not obviously true a few years ago.

And the fact that the successful route is compression is itself the most substantive hint in this article. A language model that predicts its corpus well is a good compressor of that corpus, and description length is exactly the quantity that turns into a generalization guarantee. That is not a full theory. It is a direction.

What we actually know

Assembling the pieces honestly:

Why the field moved this way. Capacity could not distinguish two zero-loss networks. The question shifted from counting parameters, to asking which solution optimization selects, to measuring the complexity of the learned function itself.

The pattern is consistent. Every result that is rigorous holds in a setting simple enough to analyze — linear models, kernels, fixed features, homogeneous networks, algorithmic datasets. Every result that is about real models is empirical, and describes a phenomenon rather than explaining it. The bridge between the two has not been built.

What survives, and what is worth carrying into the rest of this series, is smaller than a theory but not nothing:

• Counting parameters is the wrong measure of capacity. Counting bits is much better, and counting compressed description length is better still.
• The training procedure is part of the model. Two networks with identical architectures trained differently are different hypotheses, and the optimizer is doing selection work that the loss function is not.
• Memorizing and generalizing are competing ways to reduce the same loss. Which one wins is decided by how much data there is relative to how cheap memorizing is.
• For pretraining specifically, the memorization budget loses by roughly three orders of magnitude. That is the closest thing to a first-principles reason to expect a language model to generalize, and it is an argument about arithmetic rather than about deep learning.

That last point is what makes the next article possible. Once you accept that the model cannot store its corpus and must compress it instead, the natural question is how well it compresses as a function of size, data and compute — and that question, unlike this one, has a clean empirical answer.

References

Grouped by the section that leans on them. Every entry links to a primary source.