State-Space Model (SSM)

A state-space model (SSM) is a class of neural network architecture that processes sequences by maintaining a compact, fixed-size hidden state that is updated incrementally as each new token arrives. Unlike transformer models, which compute attention across all previous tokens simultaneously, SSMs process input sequentially and carry forward only a compressed summary of prior context. This makes them significantly more memory-efficient at long sequence lengths and enables inference that scales linearly rather than quadratically with context length. The Mamba architecture is among the most widely adopted SSM variants and forms the basis for hybrid models such as AI21’s Jamba.

How do state-space models work?

State-space models are grounded in classical control theory, which describes systems whose internal state evolves over time in response to inputs. In deep learning, an SSM layer takes a sequence of tokens and maintains a hidden state vector that encodes a compressed representation of everything the model has processed so far. At each step, the hidden state is updated using a learned transition function, and an output is produced based on the current state and input.

Modern SSMs such as Mamba introduce selective state spaces, where the model learns to decide how much of each incoming token to incorporate into its state and how much prior context to retain or discard. This selectivity allows the model to capture relevant long-range dependencies without attending to every prior position, avoiding the quadratic computational cost of full self-attention.

Because the hidden state has a fixed size regardless of sequence length, SSMs do not require storing or reprocessing the entire history of tokens during generation. This contrasts with transformer-based models, which maintain a key-value cache that grows with context length and becomes a significant memory and latency constraint in long-context settings.

What are state-space models used for?

SSMs are used in settings where long-context efficiency and a low memory footprint are priorities. They are particularly well-suited to tasks involving very long documents, continuous data streams, or on-device inference where GPU memory is constrained.

In practice, SSMs are frequently deployed as a component within hybrid architectures rather than as standalone models. Hybrid Transformer-SSM models, such as Jamba, interleave attention layers with Mamba-based SSM layers, combining the global reasoning capability of attention with the memory efficiency of state-space processing. This makes them attractive for enterprise tasks that require processing of long documents, including contract analysis, financial report summarization, and multi-document question answering.

SSMs also create distinct training infrastructure challenges. Because they handle sequences differently from transformers, they do not natively support certain transformer-centric optimizations such as sequence packing, which requires architecture-specific handling of sequence boundaries. This has motivated research into model-agnostic alternatives – such as external padding minimization – that can recover training efficiency without requiring changes to the model architecture itself.