Harel Lidar

Paper summary · Sept 2026

AdaptiVocab - Enhancing LLM efficiency in focused domains through lightweight vocabulary adaptation

Summary of AdaptiVocab - Enhancing LLM efficiency in focused domains through lightweight vocabulary adaptation by Itai Nakash, Roi Reichart · paper ↗

Provenance

Claim

When focusing on narrow domains, one can achieve significant speedups by removing rare tokens and replacing them with common N-Grams

Techniques and evidence

  1. Identify highest scoring N-grams
  2. Identify lowest scoring tokens
  3. Replace set from 1 with set from 2.
  4. Apply tokenizer to new text -
    1. Tokenize text with original tokenizer
    2. Decompose tokens that were removed from the original vocab into smaller chunks
    3. Merge existing tokens into the new n-tokens where possible
      1. This is done greedily, otherwise it is too computationally expensive to be worthwhile
  5. Exponential embedding initialization 1.

Generalizable vs specific

Cherry-picking

Limitations and flaws

Why is this a one<>one replacement of rare and new

Concrete output

Anki