MIRAS: Google Framework Unifies Transformers, Mamba, and Titans Under One Model
Google researchers published MIRAS, a unifying framework arguing that every modern sequence model—Transformer (KV cache), Mamba (fixed-state compression), Titans (weight-update memory)—is solving the same four-choice optimization problem: how to map keys to values under an internal objective, with a regularizer that controls a "retention gate" balancing retention versus forgetting. The framework reframes Titans' "forgetting gate" as a retention gate, positioning all three architectures as instances of the same design space. The insight matters for architecture search: rather than treating these as competing paradigms, practitioners can now select along a four-dimensional design axis.
Why It Matters
A unifying framework across the three dominant sequence model families gives practitioners a principled way to mix and match memory mechanisms—rather than committing to one architectural family, teams can select the retention-forgetting tradeoff appropriate for their specific workload characteristics.