PSSA: A Non-Transformer Language Model Built From Scratch in Rust

A developer publishing under the handle Sparticle62ops has released PSSA, a language model implemented from scratch in Rust that does not use the transformer blocks that underpin almost every large model in production today. The code is available on GitHub, with a companion thread opened for technical questions. The release is notable less for its scale than for its approach: an attention-based sequence model built outside the standard Python and PyTorch stack.

Transformers have been the default building block for language models since the 2017 paper "Attention Is All You Need." That design now spans everything from small research checkpoints to the largest commercial systems, and the tooling around it — PyTorch, CUDA kernels, distributed training frameworks — assumes it. PSSA instead implements a different sequence-modeling scheme directly in Rust, a systems language chosen for memory safety and predictable performance without a garbage collector.

That combination puts PSSA in a small but growing camp of projects testing what can be built beyond the usual stack. Rust has become a more visible choice for machine-learning infrastructure over the past two years, with efforts like Candle and Burn aiming to bring model execution closer to the metal. Writing a full language model from scratch in Rust, rather than binding to existing C++ or CUDA libraries, is a heavier lift: it typically means hand-rolled tensor operations, gradient computation and training loops.

PSSA appears to be an independent effort rather than a corporate release, and key details are still missing. The initial post does not state the parameter count, training data, benchmark results or licensing terms, and the discussion thread has not yet drawn substantive technical replies. Without reported perplexity, downstream task scores or training compute, there is no way to place PSSA against even small open baselines.

For researchers tracking architecture diversity, the more interesting question is what PSSA actually changes about the attention mechanism and whether those changes hold up at any meaningful scale. From-scratch implementations are useful precisely because they strip away framework abstractions and make design choices legible. If the author publishes training curves, evaluation numbers or a paper alongside the code, PSSA could shift from curiosity to a reference point for non-transformer language modeling in Rust.

Until then, the project is a code release worth watching rather than a result worth citing. The open questions are straightforward: what the architecture replaces, how large the model is, and whether the Rust implementation can train at a scale where comparisons to transformer baselines mean anything. Watch the repository and the discussion thread for those numbers before drawing conclusions.