The Task in the Vector: What Ten Examples Leave Behind

Give GPT-J ten antonym pairs and it will answer the eleventh. Copy one 4,096-number vector out of the middle of that forward pass, paste it into a prompt that has no examples at all, and the model gets back a bit over half of what the ten examples bought it.

That trick has two names, task vector and function vector, and the first one already meant something else. For a reproducibility paper at BlackboxNLP this year, I checked whether it survives a change of model. On word pairs it mostly does. On sentiment, in 18 of 24 runs the model gave every movie review the same answer.

This is the story of what that vector carries, and of what it turns out not to.

TL;DR

"Task vector" means two unrelated things: a difference between two sets of weights in model merging, and a single hidden state in in-context learning research. Function vectors are a third construction, built from a handful of attention heads. Both activation-space methods move simple tasks into zero-shot prompts, and I reproduced that on GPT-J, Llama-3.1 and Gemma-2. The catch: a vector built from deliberately wrong labels often does nearly as well, and on sentiment classification a single vector carries almost nothing. Beating zero-shot isn't evidence until you've beaten that control.

Two things called a task vector

The idea underneath all of this is older than transformers. In 2013, Mikolov, Yih and Zweig showed that word embeddings encode relations as directions: king − man + woman lands near queen. A relation, in other words, is something you can add.

This result needed a footnote. Nissim, van Noord and van der Goot pointed out in 2019 that the standard analogy test quietly excludes the input words from the candidate answers, and that several famous examples look much less tidy without that rule. Hold onto that. The question of what you compare against comes back later, and it decides more than the vector does.

In December 2022, Ilharco et al. took the "directions encode skills" idea to weight space. Fine-tune a model, subtract the original weights, and the difference is a task vector. Add it to give a model the skill, subtract it to remove one.

In October 2023, two papers took the same idea to activations. Hendel, Geva and Globerson pulled a single hidden state out of an in-context prompt and also called it a task vector. The same month, Todd et al. built a function vector from the outputs of a few attention heads. So the field gave two unrelated objects the same name ten months apart, and every paper in the second family now spends a sentence of related work explaining that it doesn't mean the first.

The practical difference fits in one row of the table below. The weight-space version needs a fine-tune and changes the model permanently. The activation-space versions need only a prompt, and they last exactly one forward pass.

The last two columns are cousins rather than the subject of this post. In-context vectors and steering vectors (ActAdd, CAA) make the same "add a direction to the activations" move, aimed at style and behavior rather than at reproducing a task.

The weight-space version: subtract the base

Ilharco's task vector is the simplest object here. Take a pre-trained model, fine-tune it on a task, and subtract:

The paper shows three operations, and all three work better than they have any right to. Negation removes a skill: subtracting a task vector cut target-task accuracy by 45.8 points on CLIP ViT-L/14 with little change on ImageNet, and subtracting a "toxic text" vector from GPT-2 dropped toxic generations from 4.8% to 0.8% while WikiText-103 perplexity moved by less than half a point. Addition combines skills: adding all eight vision task vectors produced one model with 91.2% of the individually fine-tuned accuracy. Analogies work too, in the form ${\tau }_C+({\tau }_B-{\tau }_A)$, which is Mikolov's parallelogram again with fine-tunes in place of words.

Why should adding weight differences work at all? Ortiz-Jimenez, Favero and Frossard gave the cleanest answer: pre-training leaves the network with weight directions that control separate regions of function space, which they call weight disentanglement. Fine-tuning in the linearized tangent space strengthens it; eight-task addition on ViT-L/14 rose from 88.8 to 93.5 normalized accuracy that way. TIES-Merging and DARE followed as practical fixes for what happens when several deltas fight over the same parameters.

The constraint is strict: same architecture, same starting weights. You can't add a Llama delta to a Gemma. The same algebra is what made it possible to prove that a "homegrown" model was a merge, which I wrote about in The Math Remembers.

The weight-space story is mostly settled. The activation-space one isn't.

In-context learning compresses the prompt

Hendel's paper makes a specific claim about how in-context learning works. The model reads the demonstrations, compresses them into a single vector $\theta$, and then applies a function to the query using it, roughly $f(x;\theta )$. If that's true, $\theta$ should be a real object you can find and move.

So they moved it. Run the ten-shot prompt, but end it with a dummy query instead of the real one, so $\theta$ can't encode the real answer. Read the state of the residual stream at the last token after layer L. Then run a prompt that contains only the real query, and overwrite the state at the same layer and position with $\theta$.

Across 18 single-token tasks and seven models (LLaMA 7B, 13B and 30B, GPT-J, and three Pythia sizes), the patched prompt kept "around 80-90%" of ICL accuracy, against 10–20% with no demonstrations. It worked best when $\theta$ came from an intermediate layer. The most convincing test was a conflict: patch task B's vector into a prompt full of task A's examples, and the model mostly does task B (0.77 when a Next Letter prompt received a To Upper vector). The examples were still sitting in the prompt. The vector won anyway.

Todd's function vectors take a different route to a similar place. Here are the two pipelines next to each other.

In code, the difference between the two interventions is one operator. These lines are from my reproduction, written with NNsight; h is the residual stream at the chosen layer, and -1 is the last token.

# Task vector: overwrite the state (src/fvtv/tv.py)h = blk.output[0] if is_tuple else blk.outputif additive:    h[:, -1, :] = h[:, -1, :] + theta.to(h)else:    h[:, -1, :] = theta.to(h)# Function vector: add to the state (src/fvtv/fv.py)h = blk.output[0] if is_tuple else blk.outputh[:, -1, :] += fv_vector.to(h)

Replacement throws away whatever the zero-shot prompt had computed at that position; addition keeps it and nudges it. (The additive branch is a control I ran for task vectors. It mostly did worse than replacement.) Keep the addition in mind, because it comes up again with Gemma-2.

Function vectors: which heads carry the task

Hendel's $\theta$ is the whole residual state, everything the model knows at that point. Todd et al. asked a narrower question: which attention heads are responsible for the task, and what happens if you take only those?

Finding them takes a causal test rather than a correlation, in four steps.

The headline numbers are large. Averaged over six tasks with no examples in the prompt, the function vector lifted GPT-J from 5.5% to 57.5%, GPT-NeoX from 6.7% to 57.1%, and Llama 2 70B from 8.2% to 83.8%. One detail matters for reading those numbers: accuracy was measured on queries the model already answered correctly with ten-shot ICL. So they tell you how much of the ICL behavior the vector carries, not how often the model is right.

The vector also carries more than a list of task words. Decode the function vector to vocabulary, rebuild a vector from its top 100 tokens, and Antonym accuracy drops from 48.2% to 4.8%. Function vectors even compose, sometimes: adding and subtracting the right ones produced a "capital of the last country in the list" vector that scored 0.60 against 0.32 for plain ICL, while the matching English-to-French composition managed 0.06.

A reasonable guess is that these are just induction heads, the copy-what-came-before heads that earlier work tied to in-context learning. Yin and Steinhardt checked across 12 models and found they're mostly different heads. Few-shot performance depends more on the function-vector heads as models grow, although many of those heads start out as induction heads early in training.

What happened when I moved models

Both original papers did most of their work on GPT-J-era models. Interpretability results can depend on metric and intervention choices and vary from model to model, so for my BlackboxNLP 2026 reproducibility paper I reran both methods on checkpoints neither paper had tested. The code, task data and raw outputs are on GitHub.

The setup was deliberately plain. GPT-J-6B came first, because both papers used it and I wanted to reproduce before extending. Then Llama-3.1-8B, Gemma-2-9b-it, and Llama-3.1-70B for task vectors only, because the function-vector head search was too slow at 70B on the compute I had. I used 16 word-pair tasks from Todd's collection (antonyms, translation, capitals, capitalization, next item in a sequence) and scored the first token of each answer.

To compare across models with very different baselines, I measured how much of the gap between zero-shot and ICL accuracy each intervention closed:

A recovery of 1 means the vector did everything the ten examples did; 0 means it did nothing. Here's how the three original claims came out, with each model's best-layer recovery in brackets:

The layer result replicated cleanly. On every model, task-vector recovery climbs through the early layers, peaks somewhere in the middle (layer 12 of 28 on GPT-J, 14 of 32 on Llama-8B, 24 of 42 on Gemma-2, 30 of 80 at 70B), and falls to almost nothing in the last third. Whatever $\theta$ is, it exists in the middle of the network and is gone by the end.

Close

The function vector on Gemma-2 is the result I spent the longest on. It recovered 0.01 of the gap. In a later two-task follow-up on both the base and instruction-tuned Gemma checkpoints, it scored 0% on antonyms and country capitals while ICL reached 68.0–98.7%. I scaled it up to eight times its size, injected it before Gemma's extra normalization step instead of after (Gemma-2 normalizes the attention output before adding it to the residual stream), and tried selecting up to 40 heads shared across tasks. None of it produced a consistent advantage over random controls.

The other surprise was format. On GPT-J, extracting the task vector from x -> y lines instead of Q:/A: lines lifted capitalization recovery from 0.47 to 0.97. Some of what looks like a task the vector can't carry is a template it doesn't like.

The control that changes the verdict

So far, every comparison has been against zero-shot accuracy. That's a low bar, and the most useful thing my reproduction did was add a harder one.

The harder control is a task vector built from shuffled labels: the same demonstration inputs and outputs, paired wrongly. If the vector carries the input-to-output rule, the shuffled version should carry nothing. It carries a lot.

Close

Across the four models, shuffled-label vectors recovered 0.28–0.48 of the gap, against 0.56–0.83 for the real ones. A vector from a sibling task (synonyms instead of antonyms, say) recovered almost nothing, between −0.01 and 0.06. So the vector strongly encodes what kind of prompt this is, and less strongly encodes what the mapping is. Pan et al. separate these as task recognition and task learning. It varies a lot by task: on GPT-J's synonym task, shuffling dropped recovery from 0.81 to zero, while on 70B country-capital the shuffled vector kept 0.88 against 1.03. The model already knows capitals; it mostly needs to be told that capitals are the question.

There was a second problem with the first round: I had picked each vector's best layer on the same data I evaluated it on, which flatters the real vector. So I reran the main comparisons more carefully. In the follow-up, each task's inputs were split three ways: one set to build the vector, one to choose the layer, one to test. Every control also chose its own best layer on the middle set, instead of borrowing the layer that suited the real vector.

On Llama-3.1-8B, that changed the answer. Flip the baseline and watch the verdict move:

Same vector, same test set, three verdicts: +45.9 points against zero-shot, +15.3 against a shuffled control run at the real vector's layer, and −4.2 against a shuffled control at its own best layer. Only the last asks whether the vector carries the task, and it can't tell the two apart.

Elsewhere the advantage held up under the stricter test. Task vectors beat their shuffled controls by 16.1 points on GPT-J, 19.1 on base Gemma-2 and 32.3 on instruction-tuned Gemma-2. Function vectors beat random controls by 35.9 points on GPT-J and 24.3 on Llama-3.1-8B.

If I had to hand reviewers one sentence from the paper, it would be the one from its discussion: gains over zero-shot accuracy alone can overstate transfer.

Where one vector runs out

Word pairs have a weakness as a test. The answers are usually things the model already knows, so a vector only has to point at the right corner of memory. To see whether a vector can carry a rule the model doesn't already hold, I used short movie-review fragments from the sentiment data Todd et al. released, with arbitrary labels: positive reviews map to A and negative to B, and in a second condition, the reverse.

In-context learning handled it fine. With ten examples, accuracy was 73.5% on GPT-J, 90.8% on Llama-3.1-8B, 94.0% on base Gemma-2 and 93.3% on the instruction-tuned version. Task vectors scored 52.7%, 50.0%, 49.6% and 50.0%, against a 50% baseline you'd get by always picking one letter.

Close

In 18 of 24 arbitrary-label runs, the patched model gave the same letter to every review in the test set. A model that answers "A" to everything is, technically, a classifier. Switching back to natural labels, positive and negative, didn't help: 50.0% on three models, 49.2% on the fourth.

I'm not the only one hitting this wall, and the explanations converge. Dong et al. argue that a task vector behaves like a single synthetic demonstration, a linear combination of the real ones, which predicts that it can't represent higher-rank mappings. Tikhonov, Oseledets and Tutubalina built 3,096 few-shot tasks to test task vectors at scale and put their conclusion in the title, at this year's ACL Student Research Workshop: "One Task Vector is not Enough". And Xiong et al., this month, replace the fixed vector with a per-head linear operator, because a vector that ignores the input can't handle tasks where the answer depends on reading it.

Which is where my paper ends up too: a vector that works on word pairs isn't, by itself, evidence that it carries tasks in general.

What my reproduction doesn't show

It would be easy to read the last three sections as "the original papers were wrong." They weren't, and I want to be careful about that.

Both originals were explicit about their scope: single-token answers, simple tasks. On exactly those tasks, task-vector transfer replicated on three model families neither paper tested, and function vectors replicated on two of them. The layer structure replicated everywhere. That's the main result, and it's a positive one.

Several differences are mine, not theirs. My function-vector pipeline chose ten heads per task, where Todd et al. average the causal scores across tasks, and I fixed the demonstrations within each evaluation split instead of resampling them per query. The Gemma failure could be the method, the architecture, or my port of it. The paper doesn't claim to know which, and neither does this post.

Two scoring choices differ from the originals, deliberately. I kept every query rather than only those ICL already answers correctly; the recovery ratio already divides by ICL's effect, so the filter would only shrink the pool. And I scored the first token of each answer, which lets "Buenos" stand in for "Buenos Aires." That's lenient, but every condition (zero-shot, ICL, real vector, shuffled control) is scored the same way, so the comparisons the paper rests on aren't affected much, and only 2.4% of answers in the task set are more than one word. What it does mean is that my absolute accuracies and Todd's headline numbers aren't directly comparable.

And the statistics are modest. The held-out comparison rests on eight tasks; the exploratory grid on 5 to 16 per model. I checked whether function-vector and task-vector recovery correlate across tasks and found ρ = 0.05 on GPT-J and −0.05 on Llama-3.1-8B, which with 14 and 8 tasks is too little data to say anything either way.

Compute set the edges of the study. The exploratory grid took roughly 21,000 remote jobs over three days on NDIF's shared deployments, and the function-vector head search on Gemma-2 needed about 1,400 jobs for a single task and seed. That's why the 70B arm only tests task vectors.

Where this leaves you

Ten examples leave a real, movable signal in the middle of the network. On simple tasks it transfers to prompts that never saw the examples. Much of what it carries is "what kind of question is this" rather than "what's the rule," and for tasks where the answer depends on the input, one vector isn't enough.

If you work with any of these methods, or review papers that do:

• Say which task vector you mean. Weight delta or hidden state. They don't share evidence, and they don't share failure modes.
• Compare against a shuffled-label vector, not just zero-shot. It's the control that separates task recognition from task learning, and it's cheap to build.
• Let every control pick its own layer on held-out data. Borrowing the real vector's best layer tilts the comparison before it starts.
• Check the write path when you port to a new architecture. Normalization placement and head width differ between model families. Compare the vector's norm to the residual stream's before you read a null result as a property of the method.
• Treat word-pair success as the floor. Test a task with unfamiliar labels before claiming the vector carries the task.
• Reach for multiple vectors or operators when the mapping depends on the input. Xiong et al.'s task operators are a reasonable place to start.

The vector knows what kind of question it's looking at. Whether it knows the answer is the part you have to control for.

Sources

• My paper: Rudraraju, Do Function Vectors and Task Vectors Generalize? A Reproducibility Study Across Model Families and Scale (BlackboxNLP 2026): every reproduction number in this post
• Hendel, Geva, Globerson, In-Context Learning Creates Task Vectors (Findings of EMNLP 2023): the activation-space task vector and the conflict test
• Todd et al., Function Vectors in Large Language Models (ICLR 2024): function vectors, AIE head selection, composition
• Ilharco et al., Editing Models with Task Arithmetic (ICLR 2023): the weight-space task vector
• Ortiz-Jimenez, Favero, Frossard, Task Arithmetic in the Tangent Space (NeurIPS 2023): weight disentanglement as the reason task arithmetic works
• Mikolov, Yih, Zweig, Linguistic Regularities in Continuous Space Word Representations (NAACL 2013): relations as directions
• Nissim, van Noord, van der Goot, Fair is Better than Sensational (2019): the analogy test's hidden exclusion rule
• Yin, Steinhardt, Which Attention Heads Matter for In-Context Learning? (ICML 2025): function-vector heads vs induction heads
• Dong, Jiang, Zhu, Ning, Understanding Task Vectors in In-Context Learning: Emergence, Functionality, and Limitations (2025): the linear combination conjecture
• Tikhonov, Oseledets, Tutubalina, One Task Vector is not Enough: A Large-Scale Study for In-Context Learning (ACL 2026 SRW): 3,096 tasks, same limit
• Xiong et al., Capturing In-Context Learning Dynamics with Task Operators (2026): per-head operators in place of fixed vectors
• Pan, Gao, Chen, Chen, What In-Context Learning "Learns" In-Context (Findings of ACL 2023): task recognition vs task learning
• Zhang, Nanda, Towards Best Practices of Activation Patching in Language Models (ICLR 2024): why intervention choices change interpretability results
• Liu et al., In-context Vectors (ICML 2024): a neighbor in the comparison table
• Turner et al., Steering Language Models With Activation Engineering (2023): activation addition for steering
• Panickssery et al., Steering Llama 2 via Contrastive Activation Addition (2023): contrastive steering vectors
• Fiotto-Kaufman et al., NNsight and NDIF (ICLR 2025): the tooling the reproduction ran on

Companion code: fv-tv-repro has the code, task data and raw outputs behind every number above.