Compressed-output skills have moved from novelty to a recognized lever for reducing inference cost and latency in LLM-driven agents. The Caveman digest introduced several variants of the technique, but it deliberately stopped short of recommending one over another, instead urging readers to run each on a real task. This article compares the documented Caveman variants against the nearest external alternatives and frames the trade-offs around workflow, model choice, and stack constraints. It is written for engineering leads and prompt designers who already understand token economics and are evaluating which compression skill to standardize on. No original benchmark numbers are claimed; all performance references are tied to the cited sources so readers can reproduce or update them.
The Caveman digest deliberately avoids declaring a winning variant because the headline token-savings figures it cites are not portable. Percentages reported in one model, prompt template, or tool stack rarely transfer cleanly to another, and treating them as universal has historically led teams to standardize on a skill that quietly degrades once the surrounding context shifts.
This article treats a different question than the sibling benchmark piece. The benchmark article asks does compression work? and measures aggregate output reduction against a reference task. Here the question is narrower: given that compression is on the table, which documented variant fits the workflow, model, and stack you actually run? Answering that requires evaluating on a representative task, not on synthetic prompts.
The digest recommends three axes that any honest evaluation needs to cover:
A fourth, version-sensitive axis is worth flagging explicitly. Compressed-output skills often rewrite tool descriptions, example blocks, or schema hints. Behavior that holds against a given model snapshot can drift when the underlying model is upgraded, even within the same family, because the new snapshot may weight brevity, schema strictness, or hallucination differently. Anything that touches tool schemas should be re-tested on the new model version before the skill is promoted.
In short, the digest's guidance reduces to a single discipline: pick a real task, instrument the three axes above, and re-run the comparison whenever the model or stack changes. The remaining sections walk through the documented variants and the closest external alternatives against that evaluation frame.
The Caveman skill exposes its compression behavior through a small ladder of intensity levels rather than a single monolithic prompt. Secondary write-ups of the SKILL.md document four named levels — Lite, Full, Ultra, and Wenyan (文言文) — each tuned to a different reader and a different cost target (skillsllm.com, 8labs.id).
The strictest setting maps onto the "telegraphic" category described in the blueprint: maximum token reduction, but proportionally higher risk that any structured payload in the response — tool-call arguments, JSON envelopes, schema-bound fields — drifts away from a parser-valid form. The default Full level maps onto the permissive "compressed-natural" mode: still readable, still aggressive on filler, and a safer baseline for mixed prose-and-structure output.
The hybrid variant the digest describes — compressing surrounding prose while preserving tool-call JSON byte-for-byte — is not surfaced as a named level in the retrieved documentation. It is best treated as a composition pattern on top of the documented levels, typically implemented by pairing a low-level mode (Lite or Full) with an explicit "do not touch fenced code or JSON" rule. Headline savings shrink versus Ultra, but downstream parsers stay safe.
Finally, the SKILL.md wording is version-sensitive: published examples differ slightly across tutorials and forks. Readers reproducing token numbers should pin a known commit of the upstream repository rather than re-measuring against an unpinned copy.
Provider-native prompt caching stores the KV state of stable prompt prefixes server-side, so repeat requests pay a fraction of the input price. Anthropic exposes it through a cache_control marker on stable blocks, with cache reads billed at roughly 10% of the input rate (a 90% discount) and a 1.25× write premium that breaks even after about 1.4 reads; OpenAI offers automatic caching on prompts of 1,024 tokens or more, with 50% off cached input on older models and 90% matching on newer flagships (ProgramStrategyHQ, PointFive).
It optimizes input cost on repeated prefixes and is widely cited as the highest-ROI move for production traffic, with one case study reporting a 59% cumulative cost drop climbing past 90% on fully optimized paths (ProgramStrategyHQ, NeuralTrust). It does not reduce output volume: every generated token is still billed at the full output rate, so it is complementary to a Caveman-style output skill rather than a substitute. Provider prompt caching is covered in detail in the sibling caching article.
All major chat-completions-style APIs accept a max_tokens (or max_completion_tokens on newer OpenAI models) parameter that hard-caps token generation, plus a stop parameter that halts generation on chosen sequences. OpenAI's documentation is explicit that this is a hard cutoff, not a length controller: the model normally stops when finished or when it hits a stop sequence (OpenAI Help Center). Practitioners pair the cap with prompt-level length guidance ("reply in three sentences or fewer") to save an estimated 20–40% on output cost (NeuralTrust).
The trade-off is bluntness: a token cap will truncate tool calls, JSON payloads, and structured fields mid-schema, producing invalid output the parser has to recover from. Stop sequences give finer control but require knowing the exact terminator ahead of time, which is rarely true for free-form agent prose.
Libraries such as Microsoft's LLMLingua and the newer open-source Headroom proxy operate on the prompt itself. LLMLingua uses a small model to score tokens by predictability and drops low-entropy filler, reportedly saving 30–45% on input tokens with quality held roughly constant on representative queries (NeuralTrust, SurePrompts). Headroom goes further with content-aware engines — JSON, AST-aware code, log, RAG-dedup, conversation-history, and an optional ML model — claiming 60–95% reduction on JSON and 15–20% on coding-agent traffic, plus a "verbosity steering" hook that appends a terseness note to the end of the system prompt so the cache still hits (Headroom on GitHub, Dev.to).
These tools target input length and, in Headroom's case, "effort routing" on reasoning models (downshifting reasoning_effort or thinking.budget_tokens when a turn is just a tool-result continuation). They do not change the model surface area, so the output still reflects the model's natural verbosity unless a verbosity-steering directive is layered on. Fidelity is harder to predict than summarization, especially for code and literal quotes, and token-level deletion can leave prompts that look ungrammatical.
A separate lever is to make the model's output shape the constraint rather than its length. Structured-output and JSON-mode APIs guarantee schema-valid responses (JSON Schema, regex, or context-free grammar) with effectively no inference-time overhead, as documented for the Outlines library (blog.dottxt.ai, PromptQuorum). Combined with template prompts that lock fields and forbid preamble, the model is steered to skip conversational filler and emit only the fields a downstream tool consumes.
This is a workflow change, not a parameter change: the agent must already have a fixed schema for its next step, and human-readable summaries still need a separate call. Compared with the Caveman skills, structured output minimizes free-form prose by design rather than by instruction, which is more reliable but only applies when the next hop in the agent is a parser, a row, or a UI field — not when the deliverable is a report or an email.
Compressed-output behavior is not uniform across model families. The same telegraphic prompt can land very differently depending on the model's scale, training regimen, and tool-calling conventions, so stack fit is one of the more concrete axes on which to compare the documented Caveman variants against external alternatives.
Smaller open-weight chat models are generally more sensitive to aggressive compression than frontier-scale systems. When a prompt is reduced to a near-telegraphic form, models in the 7B–13B class tend to follow it more literally and may drop nuance, whereas larger frontier models can often reconstruct intent from sparse tokens. This asymmetry matters when selecting between the more aggressive Caveman variants and softer external approaches. If the target runtime is a local open-weight model, a heavily compressed skill can degrade task accuracy even when token savings look attractive on paper.
Frontier systems from Anthropic and OpenAI are more forgiving of sparse prompts because they have the latent capacity to fill in missing connective tissue. The plain Caveman digest leans on this behavior, and it is also why hybrid variants that preserve tooling cues work well on these stacks. By contrast, an external alternative that relies on a verbose, prompt-engineered wrapper may be redundant on a frontier runtime and unnecessarily long on a small local model.
Tool-calling formats diverge across providers. Anthropic, OpenAI, and open-weight runtimes each define their own schema for function calls, structured outputs, and tool descriptions. The hybrid Caveman variant, which preserves tool-related tokens while compressing narrative text, is therefore stack-specific. On stacks where the tool schema is rigidly tokenized, over-compressing the tool block can cause format errors; on stacks where the tool layer is loosely defined, more aggressive compression tends to be safe. Matching the variant to the runtime's tool conventions is often a more reliable lever than further prompt tuning.
Model upgrades can change how strictly a compressed prompt is followed, particularly for instruction-tuned chat models. A prompt that produced consistent output under one revision may be interpreted more loosely after a major version bump, and vice versa. Because of this, any version-sensitive claim about compressed-output performance should be re-checked against the specific model revision in production rather than against older evaluations.
A fair comparison between compressed-output skills depends on a harness that isolates the prompt technique from the surrounding stack. Without that, token counts drift between runs and any "winning" variant may simply reflect a model or runtime change rather than the skill itself.
The harness below is adapted from the methodology used by the 65% benchmark article that underpins the Caveman digest. Readers are encouraged to reuse that setup directly rather than designing a new one, since the digest's own numbers are only reproducible against it.
Cover at least three task archetypes that exercise different parts of the compression surface:
A fourth task — open-ended summarization — is optional but recommended if the skill will be used in agent loops where free-form text is common.
Each task should be backed by a held-out evaluation set that is versioned alongside the harness. Mixing train and eval prompts, or quietly updating one side of the split, is the most common reason compressed-output benchmarks disagree across publications. Pin the exact prompt IDs and the exact few-shot examples in a config file committed to the repository.
Where the runtime exposes them, freeze:
temperature = 0 (or the lowest available setting) for tasks where a single deterministic answer is expected.top_p = 1 to avoid silent truncation of compressed vocabularies.seed where supported, to make stochastic variants reproducible.Determinism cannot be guaranteed on every provider, but the configuration above minimises drift between runs.
The scorer should record two classes of metric:
A single CSV or JSONL output that joins these two columns per example makes downstream comparison straightforward.
The harness should be re-executed end-to-end whenever any of the following change:
SKILL.md or any system prompt embedded in the skill.All three are version-sensitive: a skill that wins on one model can lose ground when a provider ships a new checkpoint, and a runtime upgrade can change how compressed characters are tokenised. Treating the harness as a regression suite, rather than a one-off benchmark, is what keeps the comparison honest over time.
The variants and external alternatives covered earlier resolve into a small number of recurring decision points. Use the rubric below as a first filter, then validate the surviving candidate on a task that mirrors production traffic.
Choose the strict telegraphic variant. It assumes a sufficiently capable model that can reconstruct meaning from sparse tokens and a downstream consumer (a human reader or a chat surface) that tolerates reduced punctuation, dropped articles, and minimal formatting. Because the variant commits the hardest to token reduction, it tends to produce the lowest output token counts among the Caveman family, which is the right trade when the model is large enough to compensate. If the model is smaller or less instruction-tuned, the same compressed input can degrade task accuracy enough to negate the savings.
Choose the hybrid variant. The hybrid approach keeps the prose body compressed while preserving literal, well-formed values for structured fields such as JSON, SQL, IDs, or enum-like tokens. When a code path rather than a human interprets the output, literal correctness on those fields matters more than headline compression ratios. A malformed bracket or a truncated identifier that still "reads" to a human can silently break a parser; the hybrid variant explicitly protects against that class of failure.
Avoid Caveman-style compression and instead use provider prompt caching, output token caps, or upstream truncation. Compressed-output skills only reduce tokens emitted by the model; they do not address input-side cost, which is usually the dominant driver when context windows are large or when retrieval-augmented prompts reuse long passages. They also help little when the artifact must be long-form or richly formatted (reports, rendered markup, code with extensive comments), because the consumer needs the structure back and the model still spends output budget reconstructing it.
Whichever variant the rubric selects, treat it as a hypothesis. Run each surviving candidate through an evaluation harness that measures both token reduction and end-task quality on representative traffic, and pick the one whose trade-off curve still holds at your real prompt lengths and model version.