memoryproductivity / token optimizer

Token Optimizer

token_optimizer

Compresses code or text to reduce token count before sending to a remote LLM — removes comments, docstrings, blank lines, and optionally produces a concise structural summary. Use when: a file or snippet is too large for a prompt, you need to trim context before an LLM call, or you want to estimate compression savings. Scope: local text processing only. No network access. Works on any language. Not for: permanent file modification (source is unchanged), context window management of files-in-session (use context_pruner), or reading files from disk (combine with get_file).

How to use token_optimizer

Provide only the required inputs below. Start with the smallest scope that answers the task, then review the risk properties before an agent executes the call.

Risk and execution properties

Read only: yes

Destructive: no

Idempotent: yes

Open world: no — operates locally or through configured services

Inputs

textstringrequired

The code or text to compress.

modestring

'compress': remove comments, docstrings, blank lines, leading whitespace (preserves structure). 'summarize': return a compact structural outline (functions, classes, key logic). 'both': compress first, then append summary.

Allowed: "compress""summarize""both"

Default: "compress"

languagestring

Language hint for comment stripping: 'python', 'javascript', 'typescript', 'go', 'rust', 'java', 'c', 'cpp', 'shell', 'auto'. Default: 'auto' (detected from content).

Default: "auto"

preserve_structureboolean

If true, keep one blank line between top-level definitions (classes, functions) for readability. Default: true.

Default: true

Example input

{
  "text": "<string>",
  "mode": "compress",
  "language": "auto",
  "preserve_structure": true
}

Browse or search the full 74-tool reference to choose the smallest suitable capability.

Browse all tools →