Paper summary · Sept 2026
Speculative Probing - LLM Monitoring at Speculative-Decoding Cost
They repurpose the speculative decoding head, be that a MTP or EAGLE head to work as a sequence classifier (for prompt safety for instance). They achieve this by performing Soft Prompt Tuning on the draft heads.
Pipeline
Given a sequence of n tokens with precomputed base model hidden states and token IDs, one can constuct the following:
- Single step probing - replace the hidden state of the next predicted token (What’s usually concatenated for draft heads) with a learned soft prompt. These are fed into the draft model and subsequently routed to the classification head.
- They use a single layer weight vector for classification. i.e
- Recursive probing - Same thing, but recursively chain multiple soft prompts.