What is caveman-steer?

rkique/caveman-steer — explained in plain English

Analysis updated 2026-05-18

★ 2PythonAudience · researcherComplexity · 4/5Setup · hard

Mindmap

mindmap
  root((caveman-steer))
    What it does
      Shortens AI code explanations
      Combines prompt and steering
      Measures token savings
    Method
      Diff-in-means steering vector
      Four test conditions
      Qwen2.5-Coder-7B-Instruct
    Findings
      Prompt plus steer shortest
      Steering alone weak
      Accuracy stays similar
    Tech stack
      Python
      AST parsing
      gpt-4o-mini scoring

mindmap root((caveman-steer)) What it does Shortens AI code explanations Combines prompt and steering Measures token savings Method Diff-in-means steering vector Four test conditions Qwen2.5-Coder-7B-Instruct Findings Prompt plus steer shortest Steering alone weak Accuracy stays similar Tech stack Python AST parsing gpt-4o-mini scoring

What do people build with it?

USE CASE 1

Study how activation steering combines with prompt instructions to shorten model output.

USE CASE 2

Reproduce the four-condition benchmark comparing base, prompt, steer, and prompt plus steer.

USE CASE 3

Learn a working example of building a diff-in-means steering vector.

USE CASE 4

Reference the token savings data when deciding whether to try steering for concise AI agent output.

What is it built with?

PythonPyTorchAST

How does it compare?

	rkique/caveman-steer	0-bingwu-0/live-interpreter	010zx00x1/faresnipe
Stars	2	2	2
Language	Python	Python	Python
Setup difficulty	hard	moderate	easy
Complexity	4/5	2/5	2/5
Audience	researcher	general	general

Figures from each repo's GitHub metadata at analysis time.

How do you get it running?

Difficulty · hard Time to first run · 1h+

Requires running a 7B parameter language model locally plus GPU access to compute and apply the steering vector.

Not stated in the README, no license information is given.

So what is it?

This project is a research experiment about making AI coding assistants give shorter answers without losing accuracy. It builds on a separate popular tool called caveman, which tells an AI model, through its prompt instructions, to drop small words like the and a, and to answer in short fragments rather than full sentences, cutting the length of its answers by roughly two thirds while keeping the technical content correct. The author wanted to know whether a technique called activation steering could shrink answers even further. Activation steering works by nudging the internal numerical state of the model while it is generating text, rather than only changing the text of the instructions given to it. The steering direction used here was built by comparing the model's internal activity when it produces short, caveman style answers against its activity when producing normal, longer answers, and then using the difference between the two as a push in the desired direction. The experiments compare four setups: the model with no special instructions at all, the model given only the caveman style prompt, the model given only the steering nudge with no special prompt, and the model given both the prompt and the steering nudge together. The tests use a language model called Qwen2.5-Coder-7B-Instruct and ask it to explain what a piece of code does in plain language, using a standard benchmark dataset built for that exact task. The results show that using the caveman prompt by itself already shortens answers a great deal, and adding steering on top of the prompt shortens them further still, by about another 13 percent, while a separate check found that the combined approach used far fewer of the discouraged small words like the and a than the prompt alone. Steering by itself, without the prompt, made little difference. Correctness stayed roughly the same, around 90 to 95 percent, across all four setups, based on scoring done by a separate AI model. The repository is written in Python and includes the code used to prepare the data, run the experiments, and generate the charts and example outputs shown in the results.

Copy-paste prompts

Prompt 1

Explain how this project builds its diff-in-means steering vector from concise versus verbose examples.

Prompt 2

Walk me through how the caveman prompt and activation steering are combined at inference time.

Prompt 3

Show me how the four conditions, base, prompt, steer, and prompt plus steer, are evaluated for correctness.

Prompt 4

Help me understand why steering alone had little effect while combining it with the prompt worked better.

Frequently asked questions