LLM SECURITY TELEMETRY · SOURCE GUIDE

03. Tool-definition poisoning

Where this concern meets the GGUF execution graph

The highlighted annotations locate attack entry, possible influence, constraints and observable effects for this concern. They are source-based hypotheses, not a trace of a demonstrated attack. GGUF supplies weights and metadata; llama.cpp builds and executes the computation graph.

Source snapshot: llama.cpp 5266f24da75dc449bd56cbed7addb9c8e4a6a73e. Concrete Llama architecture, ordinary generation path, non-fused attention as configured by this driver. Other architectures and fused kernels can use different graphs. All code below is embedded and expandable; no network or inference is required.

STAGE 01

GGUF → model weights, vocabulary and architecture

GGUF is a model file. The loader reads its metadata and tensors; C++ architecture code constructs the executable graph.

MODEL BASELINE: keep the GGUF identity and inference revision fixed. This concern does not demonstrate an attack on the GGUF loader or a modification of model weights.

llama.cpp implementation

llama.cpp/src/llama-model-loader.cpp:562–580View code

562    if (!fname.empty()) {563        // Load the main GGUF564        struct ggml_context * ctx = NULL;565        struct gguf_init_params params = {566            /*.no_alloc = */ true,567            /*.ctx      = */ &ctx,568        };569570        metadata_ptr.reset(gguf_init_from_file(fname.c_str(), params));571        metadata = metadata_ptr.get();572        if (metadata == nullptr) {573            throw std::runtime_error(format("%s: failed to load model from %s", __func__, fname.c_str()));574        }575576        get_key(llm_kv(LLM_KV_GENERAL_ARCHITECTURE), arch_name, false);577        llm_kv = LLM_KV(llm_arch_from_string(arch_name));578579        files.emplace_back(new llama_file(fname.c_str(), "rb", use_direct_io));580        contexts.emplace_back(ctx);

llama.cpp implementation

llama.cpp/src/llama.cpp:343–371View code

343344        model->hparams.vocab_only = params.vocab_only;345        model->hparams.no_alloc   = params.no_alloc;346347        try {348            model->load_hparams(ml);349        } catch(const std::exception & e) {350            throw std::runtime_error("error loading model hyperparameters: " + std::string(e.what()));351        }352        if (model->arch == LLM_ARCH_CLIP) {353            throw std::runtime_error("CLIP cannot be used as main model, use it with --mmproj instead");354        }355        try {356            model->load_vocab(ml);357        } catch(const std::exception & e) {358            throw std::runtime_error("error loading model vocabulary: " + std::string(e.what()));359        }360361        model->load_stats(ml);362        model->print_info();363364        if (params.vocab_only) {365            LLAMA_LOG_INFO("%s: vocab only - skipping tensors\n", __func__);366            return {0, model_ptr.release()};367        }368369        if (!model->load_tensors(ml)) {370            return {-2, nullptr};371        }

STAGE 02

Rendered text → token IDs

The caller supplies the final prompt. llama_tokenize delegates to the loaded vocabulary; tokenization itself does not execute the transformer.

ATTACK ENTRY: instructions are inserted into a model-visible tool or parameter description. This modifies rendered text, not an MCP server or GGUF tensor.

llama.cpp implementation

llama.cpp/src/llama-vocab.cpp:4410–4419View code

4410int32_t llama_tokenize(4411    const struct llama_vocab * vocab,4412                  const char * text,4413                     int32_t   text_len,4414                 llama_token * tokens,4415                     int32_t   n_tokens_max,4416                        bool   add_special,4417                        bool   parse_special) {4418    return vocab->tokenize(text, text_len, tokens, n_tokens_max, add_special, parse_special);4419}

Driver: read the rendered prompt

src/cst_capture.cpp:83–91View code

83int main(int argc,char ** argv){84    if(argc!=2){std::cerr<<"Usage: cst-capture config.json\n";return 2;}85    try {86        json j=json::parse(read_file(argv[1]));87        std::string model_path=j.at("model"),prompt=read_file(j.at("prompt_file"));88        std::string log=j.value("log_path",std::string("trial.cstbin")),jsonl=j.value("jsonl_path",std::string("trial.telemetry.jsonl"));89        std::string run=j.value("run_id",std::string("local"));90        int max_tokens=j.value("max_tokens",32),nctx=j.value("context_size",4096),stride=j.value("score_stride",1);91        if(max_tokens<=0 || nctx<=0 || stride<=0)throw std::runtime_error("invalid token/context/score limits");

STAGE 03

Decode → build or reuse graph → set current inputs

process_ubatch applies memory state, builds or reuses the graph, sets new inputs and requests graph computation. Reusing topology still computes the new input.

EXECUTION ENTRY: the current prefix and context state reach the model here. Building or reusing topology is not itself an attack; the relevant influence is identified in the stages below.

llama.cpp implementation

llama.cpp/src/llama-context.cpp:1334–1403View code

1334llm_graph_result * llama_context::process_ubatch(const llama_ubatch & ubatch, llm_graph_type gtype, llama_memory_context_i * mctx, ggml_status & ret) {1335    if (mctx && !mctx->apply()) {1336        LLAMA_LOG_ERROR("%s: failed to apply memory context\n", __func__);1337        ret = GGML_STATUS_FAILED;1338        return nullptr;1339    }13401341    auto * res = gf_res_prev.get();1342    auto * gf  = res->get_gf();13431344    // the new graph parameters1345    // in order to correctly reuse a graph, it's full topology has to be uniquely determined by these parameters1346    const auto gparams = graph_params(res, ubatch, mctx, gtype);13471348    if (!graph_reuse_disable && res->can_reuse(gparams)) {1349        //LLAMA_LOG_DEBUG("%s: reusing previous graph\n", __func__);13501351        // with pipeline parallelism, the previous graph_compute_async may still be running1352        // on the GPU. we must synchronize before set_inputs to avoid overwriting input tensors1353        // that the previous compute is still reading.1354        if (cparams.pipeline_parallel) {1355            ggml_backend_sched_synchronize(sched.get());1356        }13571358        n_reused++;1359    } else {1360        res->reset();13611362        ggml_backend_sched_reset(sched.get());1363        ggml_backend_sched_set_eval_callback(sched.get(), cparams.cb_eval, cparams.cb_eval_user_data);13641365        //const auto t_start_us = ggml_time_us();13661367        gf = model.build_graph(gparams);13681369        //LLAMA_LOG_INFO("graph build time: %.3f ms\n", (ggml_time_us() - t_start_us)/1000.0);13701371        if (!gf) {1372            LLAMA_LOG_ERROR("%s: failed to initialize graph\n", __func__);1373            ret = GGML_STATUS_FAILED;1374            return nullptr;1375        }13761377        if (!ggml_backend_sched_alloc_graph(sched.get(), gf)) {1378            LLAMA_LOG_ERROR("%s: failed to allocate graph\n", __func__);1379            ret = GGML_STATUS_ALLOC_FAILED;1380            return nullptr;1381        }1382    }13831384    // set the input data for the input tensors1385    {1386        //const auto t_start_us = ggml_time_us();13871388        // FIXME this call causes a crash if any model inputs were not used in the graph and were therefore not allocated1389        res->set_inputs(&ubatch);13901391        //LLAMA_LOG_INFO("graph set inputs time: %.3f ms\n", (ggml_time_us() - t_start_us)/1000.0);1392    }13931394    const auto status = graph_compute(res->get_gf(), ubatch.n_tokens > 1);1395    if (status != GGML_STATUS_SUCCESS) {1396        LLAMA_LOG_ERROR("%s: failed to compute graph, compute status: %d\n", __func__, status);1397        ret = status;1398        return nullptr;1399    }14001401    ret = GGML_STATUS_SUCCESS;14021403    return res;

llama.cpp implementation

llama.cpp/src/llama-model.cpp:2673–2690View code

2673ggml_cgraph * llama_model::build_graph(const llm_graph_params & params) const {2674    std::unique_ptr<llm_graph_context> llm = build_arch_graph(params);26752676    // add on pooling layer2677    llm->build_pooling(cls, cls_b, cls_out, cls_out_b, cls_norm);26782679    // add backend sampling layers (if any)2680    llm->build_sampling();26812682    // if the gguf model was converted with --sentence-transformers-dense-modules2683    // there will be two additional dense projection layers2684    // dense linear projections are applied after pooling2685    // TODO: move reranking logic here and generalize2686    llm->build_dense_out(dense_2_out_layers, dense_2_out_layers_b, dense_3_out_layers);26872688    llm->res->set_outputs(params);26892690    return llm->res->get_gf();

Driver: decode each prompt token

src/cst_capture.cpp:176–182View code

176        cst_set_position(0,0,0,0,"prefill");cst_log_prompt(input.data(),(int)input.size(),prompt.c_str());177        auto trial_start=Clock::now(),start=trial_start;178        std::vector<int32_t> keys;179        for(size_t i=0;i<input.size();++i){180            cst_set_position(0,0,(int)i,0,"prefill");keys.push_back((int)i);cst_set_attention_key_positions(keys.data(),(int)keys.size());181            decode_one(ctx.get(),input[i]);182        }

Inside the Llama forward graph

Tensor dependency order: embeddings → per-layer attention and FFN → final logits. K/V, attention and FFN repeat for each layer. The backend executes these operations during decode; graph construction alone does not compute their values.

STAGE 04

Token embeddings + positions

Token IDs select rows of the GGUF embedding matrix. The Llama graph supplies positions and enters its transformer-layer loop.

INPUT REPRESENTATION: tokens from the source described above become embedding vectors and positional inputs. The embedding lookup does not enforce source trust or label a token as an attack.

llama.cpp implementation

llama.cpp/src/llama-graph.cpp:2303–2328View code

2303ggml_tensor * llm_graph_context::build_inp_embd(ggml_tensor * tok_embd) const {2304    const int64_t n_embd_inp = hparams.n_embd_inp();2305    const int64_t n_embd     = hparams.n_embd;23062307    assert(n_embd_inp >= n_embd);23082309    auto inp = std::make_unique<llm_graph_input_embd>(n_embd_inp);23102311    inp->tokens = ggml_new_tensor_1d(ctx0, GGML_TYPE_I32, ubatch.n_tokens);2312    cb(inp->tokens, "inp_tokens", -1);2313    ggml_set_input(inp->tokens);2314    res->t_inp_tokens = inp->tokens;23152316    inp->embd = ggml_new_tensor_2d(ctx0, GGML_TYPE_F32, n_embd_inp, ubatch.n_tokens);2317    cb(inp->embd, "inp_embd", -1);2318    ggml_set_input(inp->embd);23192320    // select one of the 2 inputs, based on the batch contents2321    // ref: https://github.com/ggml-org/llama.cpp/pull/185502322    std::array<ggml_tensor *, 2> inps;23232324    // token embeddings path (ubatch.token != nullptr)2325    {2326        auto & cur = inps[0];23272328        cur = ggml_get_rows(ctx0, tok_embd, inp->tokens);

llama.cpp implementation

llama.cpp/src/models/llama.cpp:108–131View code

108    inpL = build_inp_embd(model.tok_embd);109110    // inp_pos - contains the positions111    ggml_tensor * inp_pos = build_inp_pos();112113    using inp_attn_type = std::conditional_t<embed, llm_graph_input_attn_no_cache, llm_graph_input_attn_kv>;114115    inp_attn_type * inp_attn = nullptr;116    if constexpr (embed) {117        inp_attn = build_attn_inp_no_cache();118    } else {119        inp_attn = build_attn_inp_kv();120    }121122    const float kq_scale = hparams.f_attention_scale == 0.0f ? 1.0f/sqrtf(float(n_embd_head)) : hparams.f_attention_scale;123124    ggml_tensor * inp_out_ids = build_inp_out_ids();125126    for (int il = 0; il < n_layer; ++il) {127        res->t_layer_inp[il] = inpL;128129        ggml_tensor * inpSA = inpL;130131        // norm

STAGE 05

Per layer: Q/K/V + position encoding → KV cache

The graph computes Q/K/V, applies RoPE and adds K/V cache writes. Attention reads the available cache with a position mask.

Tokens from the poisoned definition enter K/V context alongside the user task and other definitions.

llama.cpp implementation

llama.cpp/src/models/llama.cpp:139–161View code

139            // rope freq factors for llama3; may return nullptr for llama2 and other models140            ggml_tensor * rope_factors = model.get_rope_factors(cparams, il);141142            // compute Q and K and RoPE them143            auto [Qcur, Kcur, Vcur] = build_qkv(model.layers[il], cur,144                    n_embd_head, n_head, n_head_kv, il);145146            Qcur = ggml_rope_ext(147                    ctx0, Qcur, inp_pos, rope_factors,148                    n_rot, rope_type, n_ctx_orig, freq_base, freq_scale,149                    ext_factor, attn_factor, beta_fast, beta_slow150                    );151152            Kcur = ggml_rope_ext(153                    ctx0, Kcur, inp_pos, rope_factors,154                    n_rot, rope_type, n_ctx_orig, freq_base, freq_scale,155                    ext_factor, attn_factor, beta_fast, beta_slow156                    );157158            cb(Qcur, "Qcur", il);159            cb(Kcur, "Kcur", il);160            cb(Vcur, "Vcur", il);161

llama.cpp implementation

llama.cpp/src/llama-graph.cpp:2815–2833View code

2815    const auto * mctx_cur = inp->mctx;28162817    // store to KV cache2818    {2819        const auto & k_idxs = inp->get_k_idxs();2820        const auto & v_idxs = inp->get_v_idxs();28212822        ggml_build_forward_expand(gf, mctx_cur->cpy_k(ctx0, k_cur, k_idxs, il));2823        ggml_build_forward_expand(gf, mctx_cur->cpy_v(ctx0, v_cur, v_idxs, il));2824    }28252826    ggml_tensor * kq_mask = inp->get_kq_mask();28272828    ggml_tensor * q = q_cur;2829    ggml_tensor * k = mctx_cur->get_k(ctx0, il);2830    ggml_tensor * v = mctx_cur->get_v(ctx0, il);28312832    ggml_tensor * cur = build_attn_mha(q, k, v, kq_b, kq_mask, sinks, v_mla, 0, kq_scale, il);2833    cb(cur, "kqv_out", il);

STAGE 06

Per layer: QK scores → masked softmax → weighted V

On the non-fused path, kq_soft_max contains attention probabilities. Those weights mix value vectors into the attention output.

POSSIBLE INFLUENCE: the tool-choice query attends to the poisoned description. Compare definition-region attention at an aligned choice boundary.

llama.cpp implementation

llama.cpp/src/llama-graph.cpp:2605–2610View code

2605        ggml_tensor * kq = ggml_mul_mat(ctx0, k, q);2606        cb(kq, "kq", il);26072608        // note: this op tends to require high floating point range2609        //       while for some models F16 is enough, for others it is not, so we default to F32 here2610        ggml_mul_mat_set_prec(kq, GGML_PREC_F32);

llama.cpp implementation

llama.cpp/src/llama-graph.cpp:2633–2650View code

26332634        if (kq_b) {2635            kq = ggml_add(ctx0, kq, kq_b);2636            cb(kq, "kq_plus_kq_b", il);2637        }26382639        kq = ggml_soft_max_ext(ctx0, kq, kq_mask, kq_scale, hparams.f_max_alibi_bias);2640        ggml_soft_max_add_sinks(kq, sinks);2641        cb(kq, "kq_soft_max", il);26422643        if (!v_trans) {2644            // note: avoid this branch2645            v = ggml_cont(ctx0, ggml_transpose(ctx0, v));2646            cb(v, "v_cont", il);2647        }26482649        ggml_tensor * kqv = ggml_mul_mat(ctx0, v, kq);2650        cb(kqv, "kqv", il);

STAGE 07

Per layer: attention residual → FFN → layer output

Attention output is added to the residual, transformed by the FFN, then added again. l_out feeds the next layer; the layer loop repeats.

POSSIBLE INFLUENCE: the description can alter representations used for planning and name/argument generation.

llama.cpp implementation

llama.cpp/src/models/llama.cpp:173–195View code

173        }174        if (il == n_layer - 1 && inp_out_ids) {175            cur   = ggml_get_rows(ctx0,   cur, inp_out_ids);176            inpSA = ggml_get_rows(ctx0, inpSA, inp_out_ids);177        }178        ggml_tensor * ffn_inp = ggml_add(ctx0, cur, inpSA);179        cb(ffn_inp, "ffn_inp", il);180181        // feed-forward network (non-MoE)182        if (model.layers[il].ffn_gate_inp == nullptr) {183184            cur = build_norm(ffn_inp,185                    model.layers[il].ffn_norm, NULL,186                    LLM_NORM_RMS, il);187            cb(cur, "ffn_norm", il);188189            cur = build_ffn(cur,190                    model.layers[il].ffn_up,   model.layers[il].ffn_up_b,   model.layers[il].ffn_up_s,191                    model.layers[il].ffn_gate, model.layers[il].ffn_gate_b, model.layers[il].ffn_gate_s,192                    model.layers[il].ffn_down, model.layers[il].ffn_down_b, model.layers[il].ffn_down_s,193                    NULL,194                    LLM_FFN_SILU, LLM_FFN_PAR, il);195            cb(cur, "ffn_out", il);

llama.cpp implementation

llama.cpp/src/models/llama.cpp:220–228View code

220        cur = ggml_add(ctx0, cur, ffn_inp);221        cb(cur, "ffn_out", il);222223        cur = build_cvec(cur, il);224        cb(cur, "l_out", il);225226        // input for next layer227        inpL = cur;228    }

STAGE 08

Final normalization → vocabulary logits

After the final layer, output normalization and the GGUF output projection produce result_output / t_logits.

OBSERVE: compare likelihoods of supplied serialized tool names. Auxiliary teacher forcing repeats this graph path with a separate context.

llama.cpp implementation

llama.cpp/src/models/llama.cpp:230–246View code

230231    cur = build_norm(cur,232            model.output_norm, NULL,233            LLM_NORM_RMS, -1);234235    cb(cur, "result_norm", -1);236    res->t_embd = cur;237238    if constexpr (!embed) {239        // lm_head240        cur = build_lora_mm(model.output, cur, model.output_s);241242        cb(cur, "result_output", -1);243        res->t_logits = cur;244    }245246    ggml_build_forward_expand(gf, cur);

STAGE 09

Backend executes these graph operations; callbacks observe tensors

The preceding boxes describe tensor dependencies. Numerics execute through this scheduler, which computes graph ranges, synchronizes and invokes requested observation callbacks during execution.

Telemetry observes primary graph tensors. Auxiliary scoring disables the callback; it is an additional computation, not a tool execution path.

llama.cpp implementation

llama.cpp/src/llama-context.cpp:2491–2518View code

2491ggml_status llama_context::graph_compute(2492            ggml_cgraph * gf,2493                   bool   batched) {2494    int n_threads        = batched ? cparams.n_threads_batch : cparams.n_threads;2495    ggml_threadpool_t tp = batched ? threadpool_batch        : threadpool;24962497    if (backend_cpu != nullptr) {2498        auto * reg = ggml_backend_dev_backend_reg(ggml_backend_get_device(backend_cpu));2499        auto * set_threadpool_fn = (decltype(ggml_backend_cpu_set_threadpool) *) ggml_backend_reg_get_proc_address(reg, "ggml_backend_cpu_set_threadpool");2500        if (set_threadpool_fn) {2501            set_threadpool_fn(backend_cpu, tp);2502        }2503    }25042505    // set the number of threads for all the backends2506    for (const auto & set_n_threads_fn : set_n_threads_fns) {2507        set_n_threads_fn.second(set_n_threads_fn.first, n_threads);2508    }25092510    auto status = ggml_backend_sched_graph_compute_async(sched.get(), gf);2511    if (status != GGML_STATUS_SUCCESS) {2512        LLAMA_LOG_ERROR("%s: ggml_backend_sched_graph_compute_async failed with error %d\n", __func__, status);2513    }25142515    // fprintf(stderr, "splits: %d\n", ggml_backend_sched_get_n_splits(sched));25162517    return status;2518}

llama.cpp implementation

llama.cpp/ggml/src/ggml-backend.cpp:1802–1838View code

1802        if (!sched->callback_eval) {1803            enum ggml_status ec = ggml_backend_graph_compute_async(split_backend, &split->graph);1804            if (ec != GGML_STATUS_SUCCESS) {1805                return ec;1806            }1807        } else {1808            // similar to ggml_backend_compare_graph_backend1809            for (int j0 = 0; j0 < split->graph.n_nodes; j0++) {1810                struct ggml_tensor * t = split->graph.nodes[j0];18111812                // check if the user needs data from this node1813                bool need = sched->callback_eval(t, true, sched->callback_eval_user_data);18141815                int j1 = j0;18161817                // determine the range [j0, j1] of nodes that can be computed together1818                while (!need && j1 < split->graph.n_nodes - 1) {1819                    t = split->graph.nodes[++j1];1820                    need = sched->callback_eval(t, true, sched->callback_eval_user_data);1821                }18221823                struct ggml_cgraph gv = ggml_graph_view(&split->graph, j0, j1 + 1);18241825                enum ggml_status ec = ggml_backend_graph_compute_async(split_backend, &gv);1826                if (ec != GGML_STATUS_SUCCESS) {1827                    return ec;1828                }18291830                // TODO: pass backend to the callback, then the user can decide if they want to synchronize1831                ggml_backend_synchronize(split_backend);18321833                if (need && !sched->callback_eval(t, false, sched->callback_eval_user_data)) {1834                    break;1835                }18361837                j0 = j1;1838            }

llama.cpp implementation

llama.cpp/ggml/src/ggml-backend.cpp:461–464View code

461enum ggml_status ggml_backend_graph_compute_async(ggml_backend_t backend, struct ggml_cgraph * cgraph) {462    GGML_ASSERT(backend);463    return backend->iface.graph_compute(backend, cgraph);464}

STAGE 10

Raw logits → grammar / sampler stages → selected token

This driver applies its configured samplers after decode. llama_sampler_apply dispatches each stage; a grammar stage filters the candidate distribution.

A tool-call grammar can constrain structure while still allowing the wrong tool or arguments. Candidate scores are separate from primary sampling.

llama.cpp implementation

llama.cpp/src/llama-sampler.cpp:382–389View code

382void llama_sampler_apply(struct llama_sampler * smpl, struct llama_token_data_array * cur_p) {383    if (!smpl) {384        return;385    }386387    GGML_ASSERT(smpl->iface->apply);388    smpl->iface->apply(smpl, cur_p);389}

llama.cpp implementation

llama.cpp/src/llama-sampler.cpp:2680–2685View code

2680static void llama_sampler_grammar_apply(struct llama_sampler * smpl, llama_token_data_array * cur_p) {2681    auto * ctx = (llama_sampler_grammar *) smpl->ctx;2682    if (ctx->grammar) {2683        llama_grammar_apply_impl(*ctx->grammar, cur_p);2684    }2685}

STAGE 11

Selected token → output bytes → next decode ↺

llama_token_to_piece returns the token bytes. The caller records them and feeds the selected token into the next decode when another prediction is needed.

BEHAVIORAL EVIDENCE: read the actual tool name and argument bytes. A name emitted by the model does not establish tool invocation.

llama.cpp implementation

llama.cpp/src/llama-vocab.cpp:4421–4429View code

4421int32_t llama_token_to_piece(4422    const struct llama_vocab * vocab,4423                 llama_token   token,4424                        char * buf,4425                     int32_t   length,4426                     int32_t   lstrip,4427                        bool   special) {4428    return vocab->token_to_piece(token, buf, length, lstrip, special);4429}

Driver: record bytes and feed back the selected token

src/cst_capture.cpp:219–240View code

219            for(auto & s:samplers)llama_sampler_accept(s.get(),id);220            std::string text=piece(vocab,id);cst_log_token_bytes(text.data(),text.size());221            event("special_token",{{"token_id",id},{"is_eog",llama_vocab_is_eog(vocab,id)},222                  {"eos_raw_logprob",llama_vocab_eos(vocab)>=0?json(selected_logprob(raw,nv,llama_vocab_eos(vocab))):json(nullptr)}});223            if(t==0){double ttft=ms(trial_start);cst_log_timing(CST_TIMING_FIRST_TOKEN,ttft,1,ttft>0?1000.0/ttft:0);}224            cst_log_generated_token(id,raw,nv);++generated;prefix.push_back(id);225            if(llama_vocab_is_eog(vocab,id)){stop="eog";break;}226            std::cout<<text<<std::flush;227            if(t+1==max_tokens)break;228            cst_set_position(0,0,(int)prefix.size()-1,t+1,"decode");229            keys.push_back((int)prefix.size()-1);cst_set_attention_key_positions(keys.data(),(int)keys.size());230            start=Clock::now();decode_one(ctx.get(),id);elapsed=ms(start);231            cst_log_timing(CST_TIMING_DECODE,elapsed,1,elapsed>0?1000.0/elapsed:0);232            auto memory=llama_get_memory(ctx.get());233            event("cache_lifecycle",{{"operation","append"},{"min_position",llama_memory_seq_pos_min(memory,0)},234                  {"max_position",llama_memory_seq_pos_max(memory,0)},{"logical_tokens",keys.size()},235                  {"allocated_context_capacity",llama_n_ctx(ctx.get())}});236        }237        elapsed=ms(trial_start);cst_log_timing(CST_TIMING_TOTAL_TRIAL,elapsed,generated,elapsed>0?generated*1000.0/elapsed:0);238        event("generation_end",{{"stop_reason",stop},{"generated_tokens",generated},{"prompt_tokens",input.size()}});239        cst_close();std::cout<<"\n";return cst_last_error()?1:0;240    }catch(const std::exception & e){event("generation_end",{{"stop_reason","error"},{"message",e.what()}});cst_close();std::cerr<<e.what()<<"\n";return 1;}

Generation feedback: selected token → next decode → the same forward graph with updated context. The final EOG or token-limit token is recorded without another decode. Attack success must be established from the guide’s behavioral criterion; tensor differences alone do not establish it.

Guide index · Implementation and schema

LLM-facing idea

A model-visible tool description or parameter description embeds instructions that redirect planning. This guide concerns the text consumed by the model, not modification of an MCP server.

Controlled experiment

Render a benign tool definition and a modified definition into otherwise equivalent prompts. Keep the declared tool capability and user task fixed where possible. Mark the definition with region type 5 (untrusted tool schema); an embedded passage may also receive region type 11 (attack payload).

Keep the exact rendered definition in the prompt artifact. If source-definition identity matters, supply its identifier or hash as region metadata; the executable does not discover or hash individual tool definitions automatically.

Relevant telemetry

Event	What to inspect
`prompt`, `context_region`	The definition that was actually visible to inference, with supplied source identity.
`candidate_likelihood`	Likelihood of selected tool-name or argument strings at a configured boundary.
`attention`, `hidden_state`	Changes near the definition and decision boundary.
`token_bytes`	Actual generated name/argument bytes.

Capture with this codebase

The following is a configuration overlay, not a standalone configuration. Merge it into examples/capture.json, supply a real model and prompt, and compute any region spans from that prompt.

{
  "capture_prompt_text": true,
  "candidates": [
    {
      "name": "inspect_file",
      "category": "tool",
      "anchor": "\"name\":\"",
      "text": "inspect_file"
    },
    {
      "name": "write_file",
      "category": "tool",
      "anchor": "\"name\":\"",
      "text": "write_file"
    }
  ],
  "score_stride": 1
}

Reading the evidence

Compare each candidate’s log likelihood across conditions at a corresponding tool-name boundary. Retain its token IDs and the actual generated output. A change in normalized candidate score is only a change within the supplied list; it does not quantify every possible tool call.

Output example

Illustrative event excerpt: omitted envelope and data fields are intentional. These values are not experiment results or detection thresholds. Complete event envelopes are documented in the implementation guide.

{
  "kind": "candidate_likelihood",
  "data": {
    "name": "write_file",
    "method": "teacher_forced_sequence",
    "sequence_logprob": -4.2,
    "sequence_probability": 0.0149955768
  }
}

Implementation source

Expand a file-and-line reference to read the actual source. These excerpts are embedded snapshots from this checkout. They show the measurement implementation; the experiments and output examples above remain illustrative.

Register patterns and candidate strings; create auxiliary context

src/cst_capture.cpp:138–146View code

138        for(const auto & p:j.value("patterns",json::array())) {std::string name=p.at("name"),cat=p.value("category",std::string("canary")),s=p.at("text");cst_register_pattern(name.c_str(),cat.c_str(),s.data(),s.size());}139        std::vector<Candidate> candidates;140        for(const auto & c:j.value("candidates",json::array())) {141            Candidate item{c.at("name"),c.value("category",std::string("tool")),tokenize(vocab,c.at("text"),false),tokenize(vocab,c.value("anchor",std::string("")),false)};142            if(item.tokens.empty() || input.size()+max_tokens+item.tokens.size()>(size_t)nctx)throw std::runtime_error("candidate empty or exceeds auxiliary context capacity");143            candidates.push_back(std::move(item));144        }145        Context auxiliary(nullptr,llama_free);146        if(!candidates.empty()){auto acp=cp;acp.cb_eval=nullptr;auxiliary.reset(llama_init_from_model(model.get(),acp));if(!auxiliary)throw std::runtime_error("auxiliary context init failed");}

Auxiliary candidate replay and relative scores

src/cst_capture.cpp:59–82View code

59static void score_candidates(llama_context * auxiliary,int nv,const std::vector<llama_token>& prefix,const std::vector<Candidate>& cs){60    json relative=json::array();double total=0;61    for(const auto & c:cs){62        if(!suffix_match(prefix,c.anchor))continue;63        auto start=Clock::now();llama_memory_clear(llama_get_memory(auxiliary),true);64        for(auto id:prefix)decode_one(auxiliary,id);65        std::vector<double> lp;66        for(size_t i=0;i<c.tokens.size();++i){67            lp.push_back(selected_logprob(llama_get_logits_ith(auxiliary,-1),nv,c.tokens[i]));68            if(i+1<c.tokens.size())decode_one(auxiliary,c.tokens[i]);69        }70        cst_log_candidate_likelihood(c.name.c_str(),c.tokens.data(),lp.data(),(int)lp.size());71        double joint=std::accumulate(lp.begin(),lp.end(),0.0);72        relative.push_back({{"name",c.name},{"category",c.category},{"sequence_logprob",joint}});73        total+=ms(start);74    }75    // These are scores over the supplied list, not disjoint semantic event probabilities.76    if(!relative.empty()){77        double m=-INFINITY;for(auto & x:relative)m=std::max(m,x["sequence_logprob"].get<double>());78        double z=0;for(auto & x:relative)z+=std::exp(x["sequence_logprob"].get<double>()-m);79        if(std::isfinite(m))for(auto & x:relative)x["relative_score"]=std::exp(x["sequence_logprob"].get<double>()-m)/z;80        event("candidate_relative_scores",{{"entries",relative},{"interpretation","normalized_over_supplied_strings"},{"auxiliary_ms",total}});81    }82}

Hidden-state summaries and projections

src/cst_core.cpp:309–331View code

309extern "C" void cst_log_hidden(uint16_t layer,const float * x,int n) {310    if(!valid_vector(x,n)){invalid("invalid hidden vector");return;}311    auto j=stats(x,n);j["layer"]=layer;j["location"]="residual_layer_output";312    auto it=references.find(layer);313    if(it!=references.end() && (int)it->second.values.size()==n) {314        double dot=0,na=0,nb=0,dist=0;315        for(int i=0;i<n;++i){double a=x[i],b=it->second.values[i];dot+=a*b;na+=a*a;nb+=b*b;dist+=(a-b)*(a-b);}316        j["reference_id"]=it->second.id;j["reference_l2_distance"]=std::sqrt(dist);317        j["reference_cosine"]=(na>0 && nb>0)?json(dot/std::sqrt(na*nb)):json(nullptr);318    }319    if(config.hidden_projection_dims) {320        std::vector<double> proj(config.hidden_projection_dims,0);321        // Counter-derived signs, independent of the process RNG. Algorithm ID is part of the record.322        for(int p=0;p<config.hidden_projection_dims;++p) for(int i=0;i<n;++i) {323            uint64_t v=((uint64_t)(p+1)<<32)|(uint32_t)i;324            v+=0x9e3779b97f4a7c15ULL;v=(v^(v>>30))*0xbf58476d1ce4e5b9ULL;325            v=(v^(v>>27))*0x94d049bb133111ebULL;v^=v>>31;326            proj[p]+=(v&1?1.0:-1.0)*x[i]/std::sqrt((double)config.hidden_projection_dims);327        }328        j["projection"]=proj;j["projection_method"]="splitmix64_rademacher_v1";329    }330    if(config.capture_hidden_vectors)j["values"]=std::vector<float>(x,x+n);331    emit("hidden_state",j);

Implementation status and limits

Candidate evaluation is implemented in an auxiliary llama context. Anchor matching is token-based and depends on the model’s actual serialization. No semantic tool-call parser or tool execution is included. A emitted tool name establishes output text, not invocation or external effect.

All tensor-based interpretations depend on actual tensor_observed coverage and the tested model/backend. See validation results. There is no universal security score or calibrated threshold in this implementation.

Offline HTML with embedded source excerpts. Rebuild from the repository root with python3 scripts/build_security_docs.py after changing the Markdown or source files. Install scripts/requirements-docs.txt first; pass --llama-cpp-dir /path/to/llama.cpp for a different checkout location. The builder requires the pinned source revision. Review excerpt ranges if source lines move. No inference runs in this page.