Robust learning at low signal-to-noise
You can only measure how robust a model is to noise in a setting where you know the signal. This page covers how we built that setting, how a standard xLSTM behaves in it, and the changes that made it more robust.
At a glance
- DA-xLSTM is a noise-robust variant of the xLSTM architecture developed at ANC Labs. It combines a clipped training target, a smaller hidden state (16 → 4) and an average of five models.
- In ANC Labs' synthetic signal-to-noise lab, DA-xLSTM delivered 16–36% higher signal efficiency than a standard xLSTM in all 5 test cells.
- The lab's data generator is calibrated to 13 statistics of real crypto perpetual data, and all 13 fell within tolerances set in advance. 1,583 lab measurements were run in total.
- Across training panels, DA-xLSTM's results varied 20–42% less than the standard xLSTM's, with no degenerate solutions in any cell.
Models that memorise noise
Daily data from crypto perpetual markets is hard in three ways. First, the tails: the Student-t tail parameter of returns is about 2.8, so extreme days come far more often than a normal distribution would suggest. Second, co-movement: the average correlation between assets is around 0.58, which means a cross-section of a hundred assets carries much less independent information than it seems. Third, volatility clusters and persists (GARCH persistence 0.97).
Under these conditions a flexible model starts learning noise long before it runs out of data. When we looked at the epoch where a standard xLSTM hit its best validation score, it was within the first two epochs in 86% of 170 runs. The rest of training wasn't improving the model; it was fitting it to noise.
epoch
Building a market with a known signal
On real data there's no way to separate the signal a model picks up from the noise. That's why the lab exists: a data generator that mimics the statistics of the real market, but where we know how strong the planted signal is.
The generator combines three processes. On the price side, a two-state regime drives the market factor; assets fall into four clusters, each with GARCH-style volatility. On the signal side there is a hidden, persistent component that the model only sees through a noisy measurement. On the funding side there is a shared persistent component and an asset-specific one. Four of the nine features given to the model are pure noise on purpose, and a good model has to learn to ignore them.
Calibration used moments from real data through the end of 2025 only, and all thirteen targets below landed inside tolerances set in advance.
- 2-state regime
- market + 4 cluster factors
- GARCH volatility, Student-t shocks
- hidden persistent signal
- + measurement noise
- 9 features (4 pure noise)
- shared persistent component
- + asset-specific component
- carry return
- lab measurements (fit + evaluation)
- 1,583
- experiment cells (main grid)
- 26
- uninterrupted run time of the main grid
- 18.3 h
Calibration: real panel vs. generator
| Moment | Real | Synthetic | Tolerance | |
|---|---|---|---|---|
| Cross-sectional return correlation (ρ) | 0.581 | 0.553 | ±0.05 | within tolerance |
| Tail thickness (ν) | 2.78 | 3.08 | ±0.50 | within tolerance |
| Volatility persistence (GARCH) | 0.969 | 0.946 | ±0.05 | within tolerance |
| Funding persistence | 0.748 | 0.764 | ±0.10 | within tolerance |
| Correlation of funding and flow signal | −0.161 | −0.158 | ±0.10 | within tolerance |
| Funding vol / return vol | 0.034 | 0.032 | ±20% relative | within tolerance |
| Median volatility | 0.056 | 0.053 | ±15% relative | within tolerance |
| Flow signal persistence (10 days) | 0.534 | 0.456 | ±0.10 | within tolerance |
| Cross-sectional spread of volatility | 0.347 | 0.290 | ±0.15 | within tolerance |
| Dispersion of betas | 0.242 | 0.256 | ±0.10 | within tolerance |
| Share of common component in funding (daily) | 0.191 | 0.225 | ±0.05 | within tolerance |
| Share of common component in funding (10-day) | 0.320 | 0.380 | ±0.07 | within tolerance |
| Funding cross-sectional dispersion ratio | 0.287 | 0.316 | ±0.08 | within tolerance |
The weaker the signal, the larger the gap
Efficiency is the information coefficient (IC) between a model's forecasts and realised returns, divided by the best linear IC achievable on that data. 1 is the ceiling itself; 0.5 means half the available information was captured.
DA-xLSTM delivered higher efficiency than the standard xLSTM in every cell we tested: +0.118 to +0.212 in absolute terms, 16–36% in relative terms. In the persistent-signal regime the relative gain grows as the signal weakens: 16% at t = 4.5, 35% at t = 1.5. The changes help most exactly where noise dominates.
- DA-xLSTM
- standard xLSTM
- linear ceiling
The five test cells
| Signal regime | Signal strength t | Standard xLSTM | DA-xLSTM | Gain | Relative |
|---|---|---|---|---|---|
| a — memoryless signal | 2.5 | 0.582 | 0.794 | +0.212 | 36% |
| b — persistent signal | 1.5 | 0.518 | 0.698 | +0.180 | 35% |
| b — persistent signal | 2.5 | 0.596 | 0.762 | +0.166 | 28% |
| b — persistent signal | 4.5 | 0.729 | 0.847 | +0.118 | 16% |
| c — regime-dependent signal | 2.5 | 0.623 | 0.782 | +0.158 | 25% |
Three changes, each tested on its own
The starting point is an sLSTM with exponential gating, written in plain PyTorch. Because the exponential input gate grows linearly with the input, a single extreme observation in fat-tailed data can dominate the cell state for a long time. A wide hidden state also means too much freedom for low-signal data.
We tested each change on its own first, with a hypothesis, expected effect and pass criterion written before the run. Changes that passed and were compatible with each other were combined; target clipping and Huber loss do the same job, so only clipping went in.
- Input (cross-sectional rank)
- Exponential gates i, f
- Cell state + normaliser
- 2Hidden size 16 → 4
- Forecast
- 1Training target clipped: |y| ≤ 5
- 3Average of 5 independent models
- narrower spread across training panels; the model is more consistent across different training data
- 20–42%
- cells with no degenerate solution (no collapse to a constant or neutral forecast)
- 5/5
Three lessons from the lab
- 01
On noisy data, short memory generalises better
Long memory is xLSTM's strength. But at low signal-to-noise, feeding a single step instead of a 20-day input window raised efficiency by +0.118 (95% interval +0.052 to +0.183). The long window was giving the model more room to memorise noise.
- 02
Shrink towards a prior, not towards zero
Classic regularisation pulls coefficients towards zero. When the signal is weak, that wipes out small but real effects too. Shrinking towards a known structure, a prior, keeps that weak signal at the same regularisation strength, and it gives the model a sensible place to fall back to when the data says little.
- 03
A single extreme value can take over the memory
Because the exponential input gate grows linearly with the input, a single spike of size 3, 6, 10 stays in the hidden state for 0, 5, 12–13 steps respectively. Converting inputs to cross-sectional ranks and clipping the training target keeps this in check.