Harel Lidar

Paper summary · Sept 2026

Speculative Probing - LLM Monitoring at Speculative-Decoding Cost

Summary of Speculative Probing - LLM Monitoring at Speculative-Decoding Cost · paper ↗

They repurpose the speculative decoding head, be that a MTP or EAGLE head to work as a sequence classifier (for prompt safety for instance). They achieve this by performing Soft Prompt Tuning on the draft heads.

Pipeline

Given a sequence of n tokens with precomputed base model hidden states and token IDs, one can constuct the following:

  1. Single step probing - replace the hidden state of the next predicted token (What’s usually concatenated for draft heads) with a learned soft prompt. These are fed into the draft model and subsequently routed to the classification head.
    • They use a single layer weight vector for classification. i.e out=σ(wT⋅a)out = \sigma ( w^T \cdot a)
  2. Recursive probing - Same thing, but recursively chain multiple soft prompts.