---
ai_assisted: true
title: "Weight-Space Programming"
description: "How an LLM edited weights directly, and where it leads"
pubDate: 2026-11-04T00:00:00.000Z
category: AI
subcategory: Model
tags:
  - AI
  - LLM
  - Weight-Space Programming
  - Model Editing
  - Interpretability
  - Self-Improvement
  - LoRA
  - ICL
heroImage: ''
serial: 44
draft: false
published: true
---

Researchers at MIT CSAIL recently drew attention with a study, [Sparks of Weight-Space Programming: LLMs can natively read & write neural network weights](https://rikiyat.github.io/blog/weight-space-programming/), in which GPT-6 Astra edited the weights of Qwen3.5-0.8B directly and improved its math performance. They present the work as the starting point of a new research field, Weight-Space Programming (WSP). The headline result: by targeting about 0.016% of the weights, the edit raised MATH-500 accuracy from 7.9% to 48.1%. That figure, however, comes from the condition in which Astra was allowed 50 model runs. With 10 and 25 runs, the scores were 12.3% and 14.1%.

Having models modify themselves or other models is not new. Many studies have let models autonomously write and run fine-tuning code, or even improve architectures, often in the hope that this will become a cornerstone of recursive self-improvement (RSI). [PostTrainBench](https://posttrainbench.com/), for example, measures how well an agent can post-train small models with limited resources. Improving another model is not the same as recursively improving one's own ability to improve, but it is reasonable to expect the former to lay groundwork for the latter. Still, designing experiments and writing and running fine-tuning code is work human researchers have long done, and now that coding agents perform all kinds of feats, it no longer feels remarkable. Broadly, these approaches fall under LLM-as-Optimizer.

## What weight reading can and cannot do

What puzzled me about this study was this: individual parameters have no fixed semantics you can read off directly, so how could even Astra interpret and edit them? The experiments in the post offer a partial answer. In the first two, Astra identified which game a network was trained on and which shape a diffusion model generates. (For the Freeway network, its answer was "Frogger," which a keyword rule stated in the logs' README counted as correct.) In both cases, Astra mainly analyzed the weights of the input and output layers. Their coordinates already carry meaning, as grid cells or image pixels, so this is not the same as decoding an arbitrary concept formed inside a model from scratch.

The authors acknowledge this. In the third experiment, they removed a small language model's input and output layers and gave Astra only the middle Transformer blocks. Astra still identified the training language and found the copying heads (induction heads). This is better described as analyzing functional relationships that emerge from combinations of weight matrices than as reading the meaning of individual parameters. In contrast, when asked which of two Gemma 3 270M models had been trained to be sycophantic, Astra failed in all four attempts with the weights alone. It told all three pairs apart only after receiving the attention patterns the two models produced on texts it wrote itself. In short, the first two experiments were helped by input and output layers with fixed coordinates, and the third analyzed circuits in the middle layers, but weights alone could not reliably distinguish complex behavioral differences.

The same limitation applies to "auditing without evaluation," which the authors offer as a long-term prospect: finding backdoors or hidden behavioral traits by analyzing weights, without running inference. The current results do not show that this is feasible. If anything, the failure to detect sycophancy from weights alone shows that a gap remains between static weight analysis and functional analysis on real inputs.

## Directional editing and the Qwen experiment

One of the most active research areas today is model editing that finds an activation direction tied to a specific behavior and then locally modifies, at the weight level, how much that direction contributes to the output. The recipe: form a hypothesis from the architecture and prior work, analyze how internal activations relate to outputs to estimate the direction behind the behavior, then modify the relevant weights and verify the result. [Refusal in Language Models Is Mediated by a Single Direction](https://arxiv.org/abs/2406.11717) (Arditi et al., 2024), for instance, showed that refusal can be mediated by a particular direction and that intervening on that direction changes refusal behavior. This work belongs to a lineage that includes activation steering, representation engineering, and directional weight editing.

Astra improved Qwen's math performance in much the same way. It observed the model falling into repetitive reasoning loops, compared reasoning that progressed normally with reasoning stuck in repetition, and found a direction associated with the MLP outputs of layers 15 and 19. It then selected 64 columns in each of those two layers' weight matrices and intervened on their projection onto that direction. The authors liken this to removing a single direction, but an independent verification of the released checkpoint, [Weight-Space Programming: the edit decided from its bytes, the scores re-graded](https://carlostoledo.co/reports/weight-space-programming.html), found that the edit actually subtracts six times the projection, a much stronger adjustment than removing the direction. According to the same report, the factor of 6 was fixed after Astra tried 1, 2, 3, 4 and 6, based on 3,680 graded answers to practice problems, after its 42nd run. No MATH-500 problem was used along the way.

So Astra did not edit the weights in one shot without feedback. It had up to 50 model runs, and each run could evaluate many problems at once. The strength selection above shows what that search looked like in practice: Astra did not know the answer from the start, but tuned the strength of the edit while grading thousands of practice answers. Still, the 10-, 25- and 50-run conditions were one run each, and the three runs reached different edits. The 10-run run subtracted a repetition direction at 1.5x from 30 matrices across 15 layers, the 25-run run switched off one attention head in layer 15, and only the 50-run run edited the MLPs of two layers at 6x. The gap between 14.1% at 25 runs and 48.1% at 50 cannot be read as a budget effect alone. The 50-run run spent more than 30 runs raising the strength from 1 to 6 and checking each step, which shows the budget helped, but that cannot be separated from the runs taking different paths.

The large gain also looks less like new reasoning ability than like relief from a problem that kept the model from finishing its answers, which let its existing ability show. Before the edit, 93.0% of Qwen's MATH-500 outputs hit the token limit; afterward, 33.1% did. According to the analysis Astra reported on practice problems during the work, 576 of the 579 answers that became correct after the edit had originally hit the token limit. On MATH-500, according to Toledo's re-grading, all 35 first samples in which the original model gave a boxed answer were correct, while 241 of the 303 boxed answers from the edited model (79.5%) were. The two sets differ and cannot be compared directly, but this fits the reading that the edit made the model finish its answers rather than made it more accurate. Gemma 3 270M, which rarely loops, supports the same reading: a similar procedure moved its MATH-500 accuracy only from 6.8% to at most 7.7%. The authors themselves state that stopping loops is different from adding math ability the model lacked.

What this work most lacks is a baseline. Qwen's official model card warns that the small model is prone to thinking loops, so the first question is how much performance can be recovered just by adjusting decoding settings or switching to non-thinking mode. An independent re-evaluation, [Weight-Space Programming: the edit decided from its bytes, the scores re-graded](https://carlostoledo.co/reports/weight-space-programming.html), answers part of it. Under the model card's recommended settings, the unedited Qwen scored 22.3% on MATH-500 in thinking mode (with an 81,920-token limit) and 44.0% in non-thinking mode (a floor that counts refused answers as wrong; 49.7% counting only decided answers). The recommended thinking-mode settings also differ from the post's evaluation (32,768-token limit, temperature 0.6) in both limit and temperature. Non-thinking mode is a different mode of reasoning, so it cannot simply be ranked against the edited model's 48.1%. Still, the impression of a "sixfold improvement" clearly depends heavily on the choice of baseline. The same report's rerun of the unedited model under the post's settings scored 7.2%, close to the post's 7.9%. The post reports internal controls such as random-direction edits, but no comparison against these edit-free inference settings.

On the other hand, the experiment logs and the edited model are public, and Ahsen Khaliq at Hugging Face rebuilt the model from the logs and re-ran the evaluation, so the results are open to scrutiny. The logs repository, [RikiyaT/lm_understand-logs](https://github.com/RikiyaT/lm_understand-logs), however, says it includes only the runs reported in the post and leaves out trial runs of earlier prompt versions. Even so, the repository's results table keeps some runs the post does not mention. The 0-run condition (an earlier prompt), the 5-run condition, and a 10-run condition with another prompt version submitted no edit and scored the same as the original; that version's 25-run condition scored 14.1%. The 48.1% comes from a single 50-run run among these. How well it reproduces under repeated runs is still unknown.

## Turning existing model-editing methods into agent work

Seen this way, the significance of this work, at least for the Qwen experiment, lies in combining LLM-as-Optimizer with directional weight editing so that a general-purpose LLM carries out existing model interpretation and editing on its own. It does not discover a new optimization principle; it brings established methods from mechanistic interpretability and model editing into LLM-based optimization. This account does not cover the experiments that write small networks' weights from scratch. Even those, though, are closer to implementing known game rules and algorithms as neural network computations. The car-control network that could not get over a ramp, which Astra fixed in one try without tools, was fixed the same way: by inserting a separate path that reads the car's position and speed and makes it back up before accelerating. In the process, the largest absolute weight grew from 2.37 to 876 and 13 values exceeded 100. The evaluation used a single level, and according to the prompt every episode started from the same state, differing only in sampled actions. These experiments are therefore not evidence of semantics that hold universally across weight space. The contribution of this work lies less in having edited weights directly than in showing that a general-purpose LLM can analyze a problem, design an editing procedure, and actually improve a model.

## Internalizing the optimization procedure and zero-shot editing

Writing weights without feedback was, in fact, already partly demonstrated in this study. In the MinAtar experiment, Astra received only the game rules and source code and the network layout, wrote the weights without ever running the environment, and in Breakout outscored a PPO policy trained for 500 million steps (in Seaquest and Asterix it fell short of that PPO). In the Kinetix experiment, given descriptions of the scene, rules and observation layout, the existing policy's 256 failed episodes, one recorded episode, and the weights, it fixed the policy in one try, and the edited policy succeeded in all 1,024 evaluation episodes. In the Go experiment, the zero-forward-pass condition ran Python once to turn a hand-designed circuit into weights, and won just over half (54%) of its games against smaller PPO networks. These are not zero-shot in the sense of having no information, though; they are generation and editing without iteration or training, based on the structure and problem description given. And the ability showed up in small networks whose coordinates carry fixed meaning or whose rules are known. On Qwen, a real language model, Astra needed repeated runs and observation. The 10- and 25-run runs nearly doubled GSM8K, but MATH-500 rose only a few points and the share of answers hitting the token limit barely fell. In Go, only the 0- and 50-run conditions played a PPO network of the same size; the 0-run network lost and the 50-run network won. The internalization meant here is something else. Astra's zero-shot editing came from general knowledge acquired in pre-training. If enough model-editing experiments accumulated to learn general editing rules that do not depend on a particular checkpoint, the loop of hypothesis, execution and observation that large models still require might itself be internalized into parameters. It might then become possible to edit another model's weights on the spot, zero-shot, toward a goal stated in natural language, without separate gradient descent or iterative feedback.

This possibility is already realized in limited form in Sakana AI's [Text-to-LoRA: Instant Transformer Adaption](https://proceedings.mlr.press/v267/charakorn25a.html) (ICML 2025) and [Doc-to-LoRA: Learning to Instantly Internalize Contexts](https://proceedings.mlr.press/v306/charakorn26a.html) (ICML 2026). The former is a hypernetwork pre-trained to generate LoRA adapters from a natural-language task description; the latter meta-learns how to compress the information in a document or context into an adapter. Both generate an adapter for a new input in a single forward pass, without separately fine-tuning the target model. But the former learns to generate task-specific adapters while the latter approximates context distillation, so the transformations they learn do not serve the same purpose.

WSP constructs an editing method on the fly while analyzing a problem; Text-to-LoRA and Doc-to-LoRA apply pre-learned transformation rules to new inputs. Still, if the experience accumulated through WSP's analysis and editing could be used to train such meta-models, a more general weight-editing ability might emerge where the two approaches meet. The key question will be whether it can go beyond zero-shot generalization to particular tasks and handle different model architectures and weight coordinate systems. Current work has not reached that level; this connection is a hypothesis about future research.

## The computational relationship between ICL and explicit weight updates

More fundamentally, some studies show that in-context learning (ICL) itself can implicitly perform computations similar to parameter learning. [Linear Transformers Are Secretly Fast Weight Programmers](https://proceedings.mlr.press/v139/schlag21a.html) (ICML 2021) showed a mathematical equivalence between linear attention and fast weight programming, and [Transformers Learn In-Context by Gradient Descent](https://proceedings.mlr.press/v202/von-oswald23a.html) (ICML 2023) showed that, in a particular linear regression setting, a Transformer's forward pass can implement a computation equivalent to gradient descent. These results do not mean that ICL in every real LLM works exactly like gradient descent. They do show that context-based inference on fixed parameters and explicit parameter updates are not computationally foreign to each other.

If so, ICL and explicit weight editing may not be separate principles of adaptation so much as two ways of implementing similar computations in different representations: activations or context state on one side, explicit parameters on the other. From this angle, Text-to-LoRA and Doc-to-LoRA are interesting because they learned a computation that turns a context or task description into an explicit parameter update. Real differences remain between the two, in persistence of state, memory use, cost of repeated inference, and the range of functions they can implement. But if they can achieve functionally similar adaptation under some conditions, perhaps the line we have drawn between learning and ICL is partly less about the underlying computation than about how state is represented and stored; that is, closer to a question of abstraction boundaries.

If WSP develops along these lines, the model analysis and optimization that agents now perform from the outside might be internalized into a single learnable operation. Unlike training a LoRA adapter on production data each time, this could lead to online adaptation that applies pre-learned adaptation rules at inference time. If new experience could also be folded in continuously while accumulated edits stay stable, it might grow into online continual learning. Training cost would not disappear, though. Part of the optimization cost of each edit would shift into prior meta-learning, and the preservation of existing abilities and the stability of accumulated edits would still have to be demonstrated separately.

## Deploying and controlling weight-editing ability

Technical feasibility alone will not decide the impact of this technology. How actively frontier model companies such as OpenAI and Anthropic build this ability into their models, and how much editing authority they give users, will also matter. Given safety controls and model integrity, intellectual property, and service stability, it is doubtful they would want to let users freely change a commercial model's weights through natural-language instructions alone. They may use such technology internally for model development while offering outsiders only limited fine-tuning or managed adapter generation. If so, general zero-shot weight editing may develop first in research labs and the open-weight ecosystem.

Even without letting users edit the original weights, though, providers could offer the ability to update internal state from context or to generate temporary adapters in other forms. In the long run, then, what matters is not only whether a model can edit weights, but where learned adaptation runs and who controls the resulting edits. As WSP establishes itself as a research field, the deciding variables will likely be whether weight-editing rules generalize, and in what form real systems permit and use such abilities.
