8 posts
A catatonic BitMamba 1B — broken tokenizer, 10^18 gradients, a midnight ENOMEM and a CUDA wedge that faked a dead GPU — closed 500 steps and the validation reproduced the eval loss to the fourth decimal. The diary of Phase 4B on a 12 GB RTX 3060.
The codec post ended with a dangling issue: the lm_head diverging in validation. Two weeks later, a methodical audit rescaled the model's head, generated a clean checkpoint, and became the foundation of Phase 4 — the Resurrection.
From the wave field theory paper to BitMamba-2 1B: how a study on wave propagation became the TatuEngine inference engine, and why the Ollama truncation loop (num_ctx 4096 vs 262K) became the Docs First rule that runs across the entire ecosystem.
Out of 78 posts, only 1 was about studies — the Ollama truncation loop. But it was in the studies that wave field theory, Mamba SSM and the BitMamba-2 1B were born. The story of what the blog's most neglected section actually produced.
Born from the absurd question 'what if we used RT Cores to process neural networks?', killed the dream on WSL2, and became a hybrid CPU/GPU engine with BitMamba-2 1B, radical compression, an agent with 8 subsystems and a security sandbox — 302 commits later.
298 commits, zero new features in 30 days, and an SSM inference engine that runs unsupervised. The TatuEngine doesn't need changes — it just works.
Exploring the current state of TatuEngine — lessons, challenges, and discoveries.
Exploring the beginnings of TatuEngine — lessons, challenges, and discoveries.