Lior Pachter Argues AI Alignment Research Should Borrow From Mathematics — But Not Its Incentives

Lior Pachter, a computational biologist at Caltech, published a blog post on September 12 arguing that current efforts to align AI systems with mathematics aim at the wrong target. The piece, "Align AI and Mathematics–To Something Else," contends that mathematics is not a value system that a model can be tuned toward, and that the norms making the discipline reliable originate elsewhere. His objection is not that AI systems are bad at math. It is that the field's trustworthiness comes from social practices no proof checker encodes.

The post lands in the middle of an active bet by major labs. Google DeepMind and OpenAI have both built systems that write proofs in Lean, the open-source proof assistant, on the premise that machine-checkable results give a cleaner training signal than human preference ratings. DeepMind's AlphaProof and AlphaGeometry performances at the 2024 International Mathematical Olympiad were widely read as validation of that premise. Pachter's counter is that a checker returning true or false captures a narrow slice of what mathematicians actually do.

What it leaves out, in his account, is the apparatus around the results: peer review, adversarial checking by rivals, professional credit for finding errors, and a culture that rewards refutation as much as proof. Mathematicians spent roughly a century assembling those institutions precisely because individual results and individual authors cannot be trusted on their own. A model optimized to satisfy Lean can still misjudge which theorems matter, apply a nonstandard definition, or state a result in a way that does not mean what its author intended.

The argument matters because every mainstream alignment technique needs some ground truth to optimize against, and each has documented failure modes. Constitutional AI, RLHF, and debate-based approaches all contend with reward hacking, sycophancy, and specification gaming. Mathematics stands out because its ground truth is verifiable by machine, which is exactly why labs keep returning to it. Pachter's claim is that checkability is a narrower property than correctness, and a much narrower one than the trustworthiness that makes a discipline worth depending on.

The post drew little immediate discussion, which cuts against the usual appetite for alignment arguments in technical forums. That silence may reflect how far the framing sits from the standard terms of the debate, where the question is usually which reward signal works rather than whether the signal captures the thing being rewarded. The unresolved question is what a working alternative would look like: if mathematics cannot supply the alignment target, which discipline can, and who gets to decide. Pachter does not answer that in the post.

For now the practical consequence is a sharper test for anyone building evaluation pipelines on formal math. Verifiable proof is a real signal, but it is a signal about one narrow output, not about whether a system reasons the way a careful researcher does. Teams treating Lean success as a proxy for alignment should expect that gap to show up in edge cases, and should be explicit about which norms they are not measuring.