# Study proposes fractal basin geometry as a limit on deep learning predictability

> Ly and Gong propose that fractal basin geometry makes deep learning outcomes hard to predict and reproduce, because outcomes interleave at arbitrarily fine scales.

- Status: official (From the company, authors or authority themselves, with a link to their own page.)
- Topic: Research
- Published: 2026-11-05T00:00:00+00:00
- Updated: 2026-11-07T12:04:35.520230+00:00
- Web page: https://ai-news.ikangai.com/story/study-proposes-fractal-basin-geometry-as-a-limit-on-deep-learning-predictability/
- Seen in: Nature Machine Intelligence
- Change: research (Research)

Ly and Gong argue in Nature Machine Intelligence that a specific kind of geometry in the training landscape can make deep learning outcomes unpredictable, even when extrinsic sources of randomness are controlled. The mechanism they propose is riddled basin geometry. Every starting point that leads to one trained solution lies arbitrarily close to starting points that lead to a different solution. The authors call the resulting structure a fat fractal, meaning it has detail at arbitrarily fine scales. Because of this, making the starting point more precise buys little predictability. The paper was published online on 5 November 2026. An earlier arXiv version from October 2025 is linked from the publisher's page.

The motivation is an empirical puzzle the authors cite. Prior work attributes irreproducibility to extrinsic noise, such as random initialization, mini-batch ordering, data augmentation and GPU numerical errors. The paper notes, however, that when that non-determinism is removed, changing a single parameter by one bit can produce variability comparable to that arising from multiple noise sources combined. The authors' explanation rests on basins of attraction, the regions of parameter space whose starting points all flow to the same trained outcome. In a riddled basin, the basin of one outcome is interleaved with that of another, so no neighbourhood around a starting point is safely on one side. The authors distinguish this from previously studied trainability boundaries built on skinny fractals. Those have zero Lebesgue measure and become predictable once the initial uncertainty is reduced enough. A fat fractal is inseparable from its boundary, so shrinking the uncertainty does not restore prediction. The paper also separates this from chaos. Chaos makes trajectories diverge over time, whereas riddling produces an outcome-level unpredictability that persists at any scale.

To show the mechanism can arise in training, the authors use a minimal model: a two-layer network with tanh activation, fit by deterministic gradient descent to eight random data points. Its symmetries produce invariant planes. At a learning rate of 2.5, the training trajectory settles onto a chaotic attractor within one of them. The authors report positive maximal exponents and transverse Lyapunov exponents that are all negative, which makes the attractor a Milnor attractor with a basin of non-zero measure. The riddling itself is inferred indirectly. Positive fluctuations in finite-time transverse exponents appear in 25.3%, 9.4% and 0.8% of segments for T = 32, 128 and 512. The authors take these as a sign of transversely unstable periodic points. A diffusive scaling of those fluctuations is described as a hallmark of riddling. Mapping destinations over a 2,047 by 2,047 grid of initializations shows a butterfly-like pattern, and magnified views show the two outcomes interwoven across many scales.

The central quantitative claim is the uncertainty exponent. The authors define f(ε) as the probability that a random perturbation of size ε changes the basin, and they fit f(ε) to a power law in ε. For the minimal model, the fitted exponent is 0.0126 ± 0.0002. A power law with an exponent this close to zero means that cutting the perturbation size by a factor of ten reduces the chance of an unpredictable outcome only slightly. This is the sense in which the paper says more precision gives marginal gains. The same logic is applied to deep networks. There, the authors report intertwined destinations and near-zero exponents across the architectures, optimizers and training protocols they tested. They also add functional exponents, which measure similarity of the learned input-output functions rather than parameters. The paper's claims about deep networks rest on signatures described as consistent with riddling, not on a demonstrated riddled basin in those networks. The text says that direct stability analysis of high-dimensional training attractors is intractable, which is why the minimal model was chosen.

The paper's account of earlier observations, including irreducible model variability and performance-reproducibility trade-offs, is offered as a rationalization of those phenomena rather than a test of them. The arXiv abstract, revised in September 2026, contains the same core claims as the published version, though the wording on functional exponents differs slightly. Taken together, the work supports riddling as a candidate explanation for irreproducibility in deep learning.

## Why it matters

If the proposed geometry holds, teams cannot rely on tighter numerical precision alone to make training runs reproducible, so run-to-run variance has to be planned for.

## Original source

- [Fractal basin geometry and the limits to predictability and reproducibility in deep learning | Nature Machine Intelligence](https://www.nature.com/articles/s42256-026-01336-3?error=cookies_not_supported&code=456af9a3-35a6-449a-ba9a-769977e898a5)
- [(2510.05606) Fractal basin geometry and the limits to predictability and reproducibility in deep learning](https://arxiv.org/abs/2510.05606)

---

AI wrote this summary from the sources listed, and nobody reviewed it individually before publication. Check the original source before you rely on it.

Cite as: IKANGAI AI News, "Study proposes fractal basin geometry as a limit on deep learning predictability", 2026-11-05, https://ai-news.ikangai.com/story/study-proposes-fractal-basin-geometry-as-a-limit-on-deep-learning-predictability/
