Flow-Corrected Thompson Sampling Tackles Concept Drift in Contextual Bandits

The challenge of non-stationary contextual bandits—where the relationship between context and reward shifts over time—has received a fresh treatment from a team of researchers. Their preprint, “Flow-Corrected Thompson Sampling for Non-Stationary Contextual Bandits,” proposes a Bayesian framework that, rather than discarding or down-weighting old observations, transports them to the present using an explicit drift model and assigns each a confidence weight reflecting transport reliability.

Standard methods such as discounting, sliding windows, or periodic restarts treat all historical data as uniformly less useful, which can discard valuable structure when drift follows predictable patterns. The team’s Flow-Corrected Thompson Sampling (fcTS) addresses this by preserving and correcting past experience through three specialized modes.

Por que importa

The problem with non-stationary bandits
Classical contextual bandit algorithms assume that reward parameters remain fixed. In practice, user preferences, market conditions, or system dynamics evolve, causing historical data to mislead current decisions. Existing heuristics—exponential discounting, fixed-length windows, or model resets—are simple but waste information when the drift is structured, such as gradual trends, seasonal cycles, or recurring regime changes.

The fcTS framework treats non-stationarity as a feature rather than a nuisance. It uses a linear Gaussian model that remains tractable for closed-form posterior updates. The method maintains truncated, incrementally updated sufficient statistics, keeping computational overhead manageable even over long deployments.

Contexto

How Flow-Corrected Thompson Sampling works
The algorithm requires an explicit model of how the reward-generating parameters evolve over time. The paper details three specializations:

• Linear parameter drift: An online slope estimator corrects past rewards to align with the current parameter value.
• Periodic variation: Phase-aware reuse identifies points from previous cycles that match the current phase, borrowing across cycles without naive averaging.
• Recurring regime switches: Changepoint detection partitions history into regimes, and the posterior memory retains only observations from the current regime after a switch is identified.

Each transported observation is weighted by an estimated reliability measure, so that poorly corrected points contribute less to the posterior. The result is a unified template that adapts to different drift types while reusing far more experience than forgetting-based baselines.

Key facts

Real-world applications and benchmarks
The authors tested fcTS on five synthetic scenarios covering linear drift, periodic patterns, and regime switches, plus a semi-synthetic portfolio-selection benchmark with multiple overlapping non-stationarities. In all cases, fcTS matched or exceeded the best forgetting-based baseline, with the largest gains in settings exhibiting recurring temporal structure—such as seasonal user behavior or changing market regimes.

For practitioners running large-scale recommendation systems, ad auctions, or algorithmic trading, the ability to retain and correct historical data could mean faster adaptation with less data waste. The paper provides detailed pseudocode and discusses implementation trade-offs, including the choice of drift model complexity.

Limitations and next checks
The method assumes that the drift model is known at least approximately. In practice, mismodeling the drift—for example, assuming linear drift when the true process is chaotic—could reduce the benefit. The paper acknowledges that the current template covers only three drift families; extending it to more general stochastic processes remains open work. Additionally, the experiments are simulated rather than deployed in a live system, so real-world latency and model selection challenges are not addressed.

Researchers interested in implementation can inspect the arXiv preprint for pseudocode and experimental setup. Practitioners working with non-stationary bandit problems can consider fcTS as a candidate for high-value applications where data efficiency is critical, but should first validate drift assumptions on their own historical logs.

Source: arXiv cs.LG preprint “Flow-Corrected Thompson Sampling for Non-Stationary Contextual Bandits” (arXiv:2606.23933, September 2026). https://arxiv.org/abs/2606.23933