LLM SECURITY TELEMETRY · SOURCE GUIDE

12. Resource exhaustion and context pressure

Where this concern meets the GGUF execution graph

The highlighted annotations locate attack entry, possible influence, constraints and observable effects for this concern. They are source-based hypotheses, not a trace of a demonstrated attack. GGUF supplies weights and metadata; llama.cpp builds and executes the computation graph.

Source snapshot: llama.cpp 5266f24da75dc449bd56cbed7addb9c8e4a6a73e. Concrete Llama architecture, ordinary generation path, non-fused attention as configured by this driver. Other architectures and fused kernels can use different graphs. All code below is embedded and expandable; no network or inference is required.

STAGE 01

GGUF → model weights, vocabulary and architecture

GGUF is a model file. The loader reads its metadata and tensors; C++ architecture code constructs the executable graph.

MODEL BASELINE: keep the GGUF identity and inference revision fixed. This concern does not demonstrate an attack on the GGUF loader or a modification of model weights.

llama.cpp implementation

llama.cpp/src/llama-model-loader.cpp:562–580View code

562    if (!fname.empty()) {563        // Load the main GGUF564        struct ggml_context * ctx = NULL;565        struct gguf_init_params params = {566            /*.no_alloc = */ true,567            /*.ctx      = */ &ctx,568        };569570        metadata_ptr.reset(gguf_init_from_file(fname.c_str(), params));571        metadata = metadata_ptr.get();572        if (metadata == nullptr) {573            throw std::runtime_error(format("%s: failed to load model from %s", __func__, fname.c_str()));574        }575576        get_key(llm_kv(LLM_KV_GENERAL_ARCHITECTURE), arch_name, false);577        llm_kv = LLM_KV(llm_arch_from_string(arch_name));578579        files.emplace_back(new llama_file(fname.c_str(), "rb", use_direct_io));580        contexts.emplace_back(ctx);

llama.cpp implementation

llama.cpp/src/llama.cpp:343–371View code

343344        model->hparams.vocab_only = params.vocab_only;345        model->hparams.no_alloc   = params.no_alloc;346347        try {348            model->load_hparams(ml);349        } catch(const std::exception & e) {350            throw std::runtime_error("error loading model hyperparameters: " + std::string(e.what()));351        }352        if (model->arch == LLM_ARCH_CLIP) {353            throw std::runtime_error("CLIP cannot be used as main model, use it with --mmproj instead");354        }355        try {356            model->load_vocab(ml);357        } catch(const std::exception & e) {358            throw std::runtime_error("error loading model vocabulary: " + std::string(e.what()));359        }360361        model->load_stats(ml);362        model->print_info();363364        if (params.vocab_only) {365            LLAMA_LOG_INFO("%s: vocab only - skipping tensors\n", __func__);366            return {0, model_ptr.release()};367        }368369        if (!model->load_tensors(ml)) {370            return {-2, nullptr};371        }

STAGE 02

Rendered text → token IDs

The caller supplies the final prompt. llama_tokenize delegates to the loaded vocabulary; tokenization itself does not execute the transformer.

ATTACK ENTRY: an excessive rendered prompt or workload can increase context and generation demands. The driver rejects prompt-plus-budget overflow before graph execution.

llama.cpp implementation

llama.cpp/src/llama-vocab.cpp:4410–4419View code

4410int32_t llama_tokenize(4411    const struct llama_vocab * vocab,4412                  const char * text,4413                     int32_t   text_len,4414                 llama_token * tokens,4415                     int32_t   n_tokens_max,4416                        bool   add_special,4417                        bool   parse_special) {4418    return vocab->tokenize(text, text_len, tokens, n_tokens_max, add_special, parse_special);4419}

Driver: read the rendered prompt

src/cst_capture.cpp:83–91View code

83int main(int argc,char ** argv){84    if(argc!=2){std::cerr<<"Usage: cst-capture config.json\n";return 2;}85    try {86        json j=json::parse(read_file(argv[1]));87        std::string model_path=j.at("model"),prompt=read_file(j.at("prompt_file"));88        std::string log=j.value("log_path",std::string("trial.cstbin")),jsonl=j.value("jsonl_path",std::string("trial.telemetry.jsonl"));89        std::string run=j.value("run_id",std::string("local"));90        int max_tokens=j.value("max_tokens",32),nctx=j.value("context_size",4096),stride=j.value("score_stride",1);91        if(max_tokens<=0 || nctx<=0 || stride<=0)throw std::runtime_error("invalid token/context/score limits");

STAGE 03

Decode → build or reuse graph → set current inputs

process_ubatch applies memory state, builds or reuses the graph, sets new inputs and requests graph computation. Reusing topology still computes the new input.

EXECUTION ENTRY: the current prefix and context state reach the model here. Building or reusing topology is not itself an attack; the relevant influence is identified in the stages below.

llama.cpp implementation

llama.cpp/src/llama-context.cpp:1334–1403View code

1334llm_graph_result * llama_context::process_ubatch(const llama_ubatch & ubatch, llm_graph_type gtype, llama_memory_context_i * mctx, ggml_status & ret) {1335    if (mctx && !mctx->apply()) {1336        LLAMA_LOG_ERROR("%s: failed to apply memory context\n", __func__);1337        ret = GGML_STATUS_FAILED;1338        return nullptr;1339    }13401341    auto * res = gf_res_prev.get();1342    auto * gf  = res->get_gf();13431344    // the new graph parameters1345    // in order to correctly reuse a graph, it's full topology has to be uniquely determined by these parameters1346    const auto gparams = graph_params(res, ubatch, mctx, gtype);13471348    if (!graph_reuse_disable && res->can_reuse(gparams)) {1349        //LLAMA_LOG_DEBUG("%s: reusing previous graph\n", __func__);13501351        // with pipeline parallelism, the previous graph_compute_async may still be running1352        // on the GPU. we must synchronize before set_inputs to avoid overwriting input tensors1353        // that the previous compute is still reading.1354        if (cparams.pipeline_parallel) {1355            ggml_backend_sched_synchronize(sched.get());1356        }13571358        n_reused++;1359    } else {1360        res->reset();13611362        ggml_backend_sched_reset(sched.get());1363        ggml_backend_sched_set_eval_callback(sched.get(), cparams.cb_eval, cparams.cb_eval_user_data);13641365        //const auto t_start_us = ggml_time_us();13661367        gf = model.build_graph(gparams);13681369        //LLAMA_LOG_INFO("graph build time: %.3f ms\n", (ggml_time_us() - t_start_us)/1000.0);13701371        if (!gf) {1372            LLAMA_LOG_ERROR("%s: failed to initialize graph\n", __func__);1373            ret = GGML_STATUS_FAILED;1374            return nullptr;1375        }13761377        if (!ggml_backend_sched_alloc_graph(sched.get(), gf)) {1378            LLAMA_LOG_ERROR("%s: failed to allocate graph\n", __func__);1379            ret = GGML_STATUS_ALLOC_FAILED;1380            return nullptr;1381        }1382    }13831384    // set the input data for the input tensors1385    {1386        //const auto t_start_us = ggml_time_us();13871388        // FIXME this call causes a crash if any model inputs were not used in the graph and were therefore not allocated1389        res->set_inputs(&ubatch);13901391        //LLAMA_LOG_INFO("graph set inputs time: %.3f ms\n", (ggml_time_us() - t_start_us)/1000.0);1392    }13931394    const auto status = graph_compute(res->get_gf(), ubatch.n_tokens > 1);1395    if (status != GGML_STATUS_SUCCESS) {1396        LLAMA_LOG_ERROR("%s: failed to compute graph, compute status: %d\n", __func__, status);1397        ret = status;1398        return nullptr;1399    }14001401    ret = GGML_STATUS_SUCCESS;14021403    return res;

llama.cpp implementation

llama.cpp/src/llama-model.cpp:2673–2690View code

2673ggml_cgraph * llama_model::build_graph(const llm_graph_params & params) const {2674    std::unique_ptr<llm_graph_context> llm = build_arch_graph(params);26752676    // add on pooling layer2677    llm->build_pooling(cls, cls_b, cls_out, cls_out_b, cls_norm);26782679    // add backend sampling layers (if any)2680    llm->build_sampling();26812682    // if the gguf model was converted with --sentence-transformers-dense-modules2683    // there will be two additional dense projection layers2684    // dense linear projections are applied after pooling2685    // TODO: move reranking logic here and generalize2686    llm->build_dense_out(dense_2_out_layers, dense_2_out_layers_b, dense_3_out_layers);26872688    llm->res->set_outputs(params);26892690    return llm->res->get_gf();

Driver: decode each prompt token

src/cst_capture.cpp:176–182View code

176        cst_set_position(0,0,0,0,"prefill");cst_log_prompt(input.data(),(int)input.size(),prompt.c_str());177        auto trial_start=Clock::now(),start=trial_start;178        std::vector<int32_t> keys;179        for(size_t i=0;i<input.size();++i){180            cst_set_position(0,0,(int)i,0,"prefill");keys.push_back((int)i);cst_set_attention_key_positions(keys.data(),(int)keys.size());181            decode_one(ctx.get(),input[i]);182        }

Inside the Llama forward graph

Tensor dependency order: embeddings → per-layer attention and FFN → final logits. K/V, attention and FFN repeat for each layer. The backend executes these operations during decode; graph construction alone does not compute their values.

STAGE 04

Token embeddings + positions

Token IDs select rows of the GGUF embedding matrix. The Llama graph supplies positions and enters its transformer-layer loop.

INPUT REPRESENTATION: tokens from the source described above become embedding vectors and positional inputs. The embedding lookup does not enforce source trust or label a token as an attack.

llama.cpp implementation

llama.cpp/src/llama-graph.cpp:2303–2328View code

2303ggml_tensor * llm_graph_context::build_inp_embd(ggml_tensor * tok_embd) const {2304    const int64_t n_embd_inp = hparams.n_embd_inp();2305    const int64_t n_embd     = hparams.n_embd;23062307    assert(n_embd_inp >= n_embd);23082309    auto inp = std::make_unique<llm_graph_input_embd>(n_embd_inp);23102311    inp->tokens = ggml_new_tensor_1d(ctx0, GGML_TYPE_I32, ubatch.n_tokens);2312    cb(inp->tokens, "inp_tokens", -1);2313    ggml_set_input(inp->tokens);2314    res->t_inp_tokens = inp->tokens;23152316    inp->embd = ggml_new_tensor_2d(ctx0, GGML_TYPE_F32, n_embd_inp, ubatch.n_tokens);2317    cb(inp->embd, "inp_embd", -1);2318    ggml_set_input(inp->embd);23192320    // select one of the 2 inputs, based on the batch contents2321    // ref: https://github.com/ggml-org/llama.cpp/pull/185502322    std::array<ggml_tensor *, 2> inps;23232324    // token embeddings path (ubatch.token != nullptr)2325    {2326        auto & cur = inps[0];23272328        cur = ggml_get_rows(ctx0, tok_embd, inp->tokens);

llama.cpp implementation

llama.cpp/src/models/llama.cpp:108–131View code

108    inpL = build_inp_embd(model.tok_embd);109110    // inp_pos - contains the positions111    ggml_tensor * inp_pos = build_inp_pos();112113    using inp_attn_type = std::conditional_t<embed, llm_graph_input_attn_no_cache, llm_graph_input_attn_kv>;114115    inp_attn_type * inp_attn = nullptr;116    if constexpr (embed) {117        inp_attn = build_attn_inp_no_cache();118    } else {119        inp_attn = build_attn_inp_kv();120    }121122    const float kq_scale = hparams.f_attention_scale == 0.0f ? 1.0f/sqrtf(float(n_embd_head)) : hparams.f_attention_scale;123124    ggml_tensor * inp_out_ids = build_inp_out_ids();125126    for (int il = 0; il < n_layer; ++il) {127        res->t_layer_inp[il] = inpL;128129        ggml_tensor * inpSA = inpL;130131        // norm

STAGE 05

Per layer: Q/K/V + position encoding → KV cache

The graph computes Q/K/V, applies RoPE and adds K/V cache writes. Attention reads the available cache with a position mask.

RESOURCE LOCATION: retained token positions and configured context capacity affect cache requirements. Capacity and logical token counts are not measured memory bytes.

llama.cpp implementation

llama.cpp/src/models/llama.cpp:139–161View code

139            // rope freq factors for llama3; may return nullptr for llama2 and other models140            ggml_tensor * rope_factors = model.get_rope_factors(cparams, il);141142            // compute Q and K and RoPE them143            auto [Qcur, Kcur, Vcur] = build_qkv(model.layers[il], cur,144                    n_embd_head, n_head, n_head_kv, il);145146            Qcur = ggml_rope_ext(147                    ctx0, Qcur, inp_pos, rope_factors,148                    n_rot, rope_type, n_ctx_orig, freq_base, freq_scale,149                    ext_factor, attn_factor, beta_fast, beta_slow150                    );151152            Kcur = ggml_rope_ext(153                    ctx0, Kcur, inp_pos, rope_factors,154                    n_rot, rope_type, n_ctx_orig, freq_base, freq_scale,155                    ext_factor, attn_factor, beta_fast, beta_slow156                    );157158            cb(Qcur, "Qcur", il);159            cb(Kcur, "Kcur", il);160            cb(Vcur, "Vcur", il);161

llama.cpp implementation

llama.cpp/src/llama-graph.cpp:2815–2833View code

2815    const auto * mctx_cur = inp->mctx;28162817    // store to KV cache2818    {2819        const auto & k_idxs = inp->get_k_idxs();2820        const auto & v_idxs = inp->get_v_idxs();28212822        ggml_build_forward_expand(gf, mctx_cur->cpy_k(ctx0, k_cur, k_idxs, il));2823        ggml_build_forward_expand(gf, mctx_cur->cpy_v(ctx0, v_cur, v_idxs, il));2824    }28252826    ggml_tensor * kq_mask = inp->get_kq_mask();28272828    ggml_tensor * q = q_cur;2829    ggml_tensor * k = mctx_cur->get_k(ctx0, il);2830    ggml_tensor * v = mctx_cur->get_v(ctx0, il);28312832    ggml_tensor * cur = build_attn_mha(q, k, v, kq_b, kq_mask, sinks, v_mla, 0, kq_scale, il);2833    cb(cur, "kqv_out", il);

STAGE 06

Per layer: QK scores → masked softmax → weighted V

On the non-fused path, kq_soft_max contains attention probabilities. Those weights mix value vectors into the attention output.

RESOURCE LOCATION: attention uses the visible key context; larger workloads can increase work. Timing depends on backend, layout and capture settings.

llama.cpp implementation

llama.cpp/src/llama-graph.cpp:2605–2610View code

2605        ggml_tensor * kq = ggml_mul_mat(ctx0, k, q);2606        cb(kq, "kq", il);26072608        // note: this op tends to require high floating point range2609        //       while for some models F16 is enough, for others it is not, so we default to F32 here2610        ggml_mul_mat_set_prec(kq, GGML_PREC_F32);

llama.cpp implementation

llama.cpp/src/llama-graph.cpp:2633–2650View code

26332634        if (kq_b) {2635            kq = ggml_add(ctx0, kq, kq_b);2636            cb(kq, "kq_plus_kq_b", il);2637        }26382639        kq = ggml_soft_max_ext(ctx0, kq, kq_mask, kq_scale, hparams.f_max_alibi_bias);2640        ggml_soft_max_add_sinks(kq, sinks);2641        cb(kq, "kq_soft_max", il);26422643        if (!v_trans) {2644            // note: avoid this branch2645            v = ggml_cont(ctx0, ggml_transpose(ctx0, v));2646            cb(v, "v_cont", il);2647        }26482649        ggml_tensor * kqv = ggml_mul_mat(ctx0, v, kq);2650        cb(kqv, "kqv", il);

STAGE 07

Per layer: attention residual → FFN → layer output

Attention output is added to the residual, transformed by the FFN, then added again. l_out feeds the next layer; the layer loop repeats.

RESOURCE LOCATION: every evaluated token traverses transformer layers. FFN and hidden-vector capture contribute computation or recording work.

llama.cpp implementation

llama.cpp/src/models/llama.cpp:173–195View code

173        }174        if (il == n_layer - 1 && inp_out_ids) {175            cur   = ggml_get_rows(ctx0,   cur, inp_out_ids);176            inpSA = ggml_get_rows(ctx0, inpSA, inp_out_ids);177        }178        ggml_tensor * ffn_inp = ggml_add(ctx0, cur, inpSA);179        cb(ffn_inp, "ffn_inp", il);180181        // feed-forward network (non-MoE)182        if (model.layers[il].ffn_gate_inp == nullptr) {183184            cur = build_norm(ffn_inp,185                    model.layers[il].ffn_norm, NULL,186                    LLM_NORM_RMS, il);187            cb(cur, "ffn_norm", il);188189            cur = build_ffn(cur,190                    model.layers[il].ffn_up,   model.layers[il].ffn_up_b,   model.layers[il].ffn_up_s,191                    model.layers[il].ffn_gate, model.layers[il].ffn_gate_b, model.layers[il].ffn_gate_s,192                    model.layers[il].ffn_down, model.layers[il].ffn_down_b, model.layers[il].ffn_down_s,193                    NULL,194                    LLM_FFN_SILU, LLM_FFN_PAR, il);195            cb(cur, "ffn_out", il);

llama.cpp implementation

llama.cpp/src/models/llama.cpp:220–228View code

220        cur = ggml_add(ctx0, cur, ffn_inp);221        cb(cur, "ffn_out", il);222223        cur = build_cvec(cur, il);224        cb(cur, "l_out", il);225226        // input for next layer227        inpL = cur;228    }

STAGE 08

Final normalization → vocabulary logits

After the final layer, output normalization and the GGUF output projection produce result_output / t_logits.

RESOURCE LOCATION: vocabulary projection and optional full-distribution recording add cost. Separate telemetry overhead from model execution cost.

llama.cpp implementation

llama.cpp/src/models/llama.cpp:230–246View code

230231    cur = build_norm(cur,232            model.output_norm, NULL,233            LLM_NORM_RMS, -1);234235    cb(cur, "result_norm", -1);236    res->t_embd = cur;237238    if constexpr (!embed) {239        // lm_head240        cur = build_lora_mm(model.output, cur, model.output_s);241242        cb(cur, "result_output", -1);243        res->t_logits = cur;244    }245246    ggml_build_forward_expand(gf, cur);

STAGE 09

Backend executes these graph operations; callbacks observe tensors

The preceding boxes describe tensor dependencies. Numerics execute through this scheduler, which computes graph ranges, synchronizes and invokes requested observation callbacks during execution.

PRIMARY COST LOCATION: backend graph execution, allocation and synchronization. The driver records diagnostic timing but does not measure RSS, GPU utilization or enforce wall-time cancellation.

llama.cpp implementation

llama.cpp/src/llama-context.cpp:2491–2518View code

2491ggml_status llama_context::graph_compute(2492            ggml_cgraph * gf,2493                   bool   batched) {2494    int n_threads        = batched ? cparams.n_threads_batch : cparams.n_threads;2495    ggml_threadpool_t tp = batched ? threadpool_batch        : threadpool;24962497    if (backend_cpu != nullptr) {2498        auto * reg = ggml_backend_dev_backend_reg(ggml_backend_get_device(backend_cpu));2499        auto * set_threadpool_fn = (decltype(ggml_backend_cpu_set_threadpool) *) ggml_backend_reg_get_proc_address(reg, "ggml_backend_cpu_set_threadpool");2500        if (set_threadpool_fn) {2501            set_threadpool_fn(backend_cpu, tp);2502        }2503    }25042505    // set the number of threads for all the backends2506    for (const auto & set_n_threads_fn : set_n_threads_fns) {2507        set_n_threads_fn.second(set_n_threads_fn.first, n_threads);2508    }25092510    auto status = ggml_backend_sched_graph_compute_async(sched.get(), gf);2511    if (status != GGML_STATUS_SUCCESS) {2512        LLAMA_LOG_ERROR("%s: ggml_backend_sched_graph_compute_async failed with error %d\n", __func__, status);2513    }25142515    // fprintf(stderr, "splits: %d\n", ggml_backend_sched_get_n_splits(sched));25162517    return status;2518}

llama.cpp implementation

llama.cpp/ggml/src/ggml-backend.cpp:1802–1838View code

1802        if (!sched->callback_eval) {1803            enum ggml_status ec = ggml_backend_graph_compute_async(split_backend, &split->graph);1804            if (ec != GGML_STATUS_SUCCESS) {1805                return ec;1806            }1807        } else {1808            // similar to ggml_backend_compare_graph_backend1809            for (int j0 = 0; j0 < split->graph.n_nodes; j0++) {1810                struct ggml_tensor * t = split->graph.nodes[j0];18111812                // check if the user needs data from this node1813                bool need = sched->callback_eval(t, true, sched->callback_eval_user_data);18141815                int j1 = j0;18161817                // determine the range [j0, j1] of nodes that can be computed together1818                while (!need && j1 < split->graph.n_nodes - 1) {1819                    t = split->graph.nodes[++j1];1820                    need = sched->callback_eval(t, true, sched->callback_eval_user_data);1821                }18221823                struct ggml_cgraph gv = ggml_graph_view(&split->graph, j0, j1 + 1);18241825                enum ggml_status ec = ggml_backend_graph_compute_async(split_backend, &gv);1826                if (ec != GGML_STATUS_SUCCESS) {1827                    return ec;1828                }18291830                // TODO: pass backend to the callback, then the user can decide if they want to synchronize1831                ggml_backend_synchronize(split_backend);18321833                if (need && !sched->callback_eval(t, false, sched->callback_eval_user_data)) {1834                    break;1835                }18361837                j0 = j1;1838            }

llama.cpp implementation

llama.cpp/ggml/src/ggml-backend.cpp:461–464View code

461enum ggml_status ggml_backend_graph_compute_async(ggml_backend_t backend, struct ggml_cgraph * cgraph) {462    GGML_ASSERT(backend);463    return backend->iface.graph_compute(backend, cgraph);464}

STAGE 10

Raw logits → grammar / sampler stages → selected token

This driver applies its configured samplers after decode. llama_sampler_apply dispatches each stage; a grammar stage filters the candidate distribution.

Sampler-stage work and auxiliary candidate replay add overhead. Disable candidate scoring for a baseline or report its time separately.

llama.cpp implementation

llama.cpp/src/llama-sampler.cpp:382–389View code

382void llama_sampler_apply(struct llama_sampler * smpl, struct llama_token_data_array * cur_p) {383    if (!smpl) {384        return;385    }386387    GGML_ASSERT(smpl->iface->apply);388    smpl->iface->apply(smpl, cur_p);389}

llama.cpp implementation

llama.cpp/src/llama-sampler.cpp:2680–2685View code

2680static void llama_sampler_grammar_apply(struct llama_sampler * smpl, llama_token_data_array * cur_p) {2681    auto * ctx = (llama_sampler_grammar *) smpl->ctx;2682    if (ctx->grammar) {2683        llama_grammar_apply_impl(*ctx->grammar, cur_p);2684    }2685}

STAGE 11

Selected token → output bytes → next decode ↺

llama_token_to_piece returns the token bytes. The caller records them and feeds the selected token into the next decode when another prediction is needed.

BEHAVIORAL EVIDENCE: record generated length, stop reason and timing. A long valid answer is not automatically resource abuse.

llama.cpp implementation

llama.cpp/src/llama-vocab.cpp:4421–4429View code

4421int32_t llama_token_to_piece(4422    const struct llama_vocab * vocab,4423                 llama_token   token,4424                        char * buf,4425                     int32_t   length,4426                     int32_t   lstrip,4427                        bool   special) {4428    return vocab->token_to_piece(token, buf, length, lstrip, special);4429}

Driver: record bytes and feed back the selected token

src/cst_capture.cpp:219–240View code

219            for(auto & s:samplers)llama_sampler_accept(s.get(),id);220            std::string text=piece(vocab,id);cst_log_token_bytes(text.data(),text.size());221            event("special_token",{{"token_id",id},{"is_eog",llama_vocab_is_eog(vocab,id)},222                  {"eos_raw_logprob",llama_vocab_eos(vocab)>=0?json(selected_logprob(raw,nv,llama_vocab_eos(vocab))):json(nullptr)}});223            if(t==0){double ttft=ms(trial_start);cst_log_timing(CST_TIMING_FIRST_TOKEN,ttft,1,ttft>0?1000.0/ttft:0);}224            cst_log_generated_token(id,raw,nv);++generated;prefix.push_back(id);225            if(llama_vocab_is_eog(vocab,id)){stop="eog";break;}226            std::cout<<text<<std::flush;227            if(t+1==max_tokens)break;228            cst_set_position(0,0,(int)prefix.size()-1,t+1,"decode");229            keys.push_back((int)prefix.size()-1);cst_set_attention_key_positions(keys.data(),(int)keys.size());230            start=Clock::now();decode_one(ctx.get(),id);elapsed=ms(start);231            cst_log_timing(CST_TIMING_DECODE,elapsed,1,elapsed>0?1000.0/elapsed:0);232            auto memory=llama_get_memory(ctx.get());233            event("cache_lifecycle",{{"operation","append"},{"min_position",llama_memory_seq_pos_min(memory,0)},234                  {"max_position",llama_memory_seq_pos_max(memory,0)},{"logical_tokens",keys.size()},235                  {"allocated_context_capacity",llama_n_ctx(ctx.get())}});236        }237        elapsed=ms(trial_start);cst_log_timing(CST_TIMING_TOTAL_TRIAL,elapsed,generated,elapsed>0?generated*1000.0/elapsed:0);238        event("generation_end",{{"stop_reason",stop},{"generated_tokens",generated},{"prompt_tokens",input.size()}});239        cst_close();std::cout<<"\n";return cst_last_error()?1:0;240    }catch(const std::exception & e){event("generation_end",{{"stop_reason","error"},{"message",e.what()}});cst_close();std::cerr<<e.what()<<"\n";return 1;}

Generation feedback: selected token → next decode → the same forward graph with updated context. The final EOG or token-limit token is recorded without another decode. Attack success must be established from the guide’s behavioral criterion; tensor differences alone do not establish it.

Guide index · Implementation and schema

LLM-facing idea

An input condition may consume more context, generate longer output or require more inference time. The telemetry characterizes model execution cost, with capture overhead explicitly considered.

Controlled experiment

Compare workloads under the same model, backend, thread count, context capacity and capture settings. Measure prompt tokens and generated tokens separately. Disable candidate scoring for a baseline timing run or report its auxiliary time independently.

Increase prompt length only within the configured capacity. The driver rejects prompt-plus-generation overflow rather than silently truncating or sliding the context. An oversized request rejected before capture initialization may only produce a nonzero exit and stderr, not a telemetry log.

Relevant telemetry

Event	What to inspect
`timing`	Prefill, first-token, decode and total-trial elapsed time.
`prompt`, `generation_end`	Input and generated token counts, stop reason.
`cache_lifecycle`, `capture_capabilities`	Logical position range and allocated context capacity.
`candidate_relative_scores`	Auxiliary candidate evaluation time when enabled.

Capture with this codebase

The following is a configuration overlay, not a standalone configuration. Merge it into examples/capture.json, supply a real model and prompt, and compute any region spans from that prompt.

{
  "max_tokens": 64,
  "context_size": 4096,
  "candidates": [],
  "attention_rows": false,
  "hidden_vectors": false,
  "projection_dims": 0
}

Reading the evidence

Normalize comparisons by the appropriate work unit, and retain absolute token counts. First-token and total-trial timing include work such as auxiliary scoring and telemetry on their measured paths. Sampling timing also includes per-stage telemetry work. None of these are uncontaminated production benchmarks.

Track missing or incomplete logs separately from successful completion. A nonzero process exit is relevant evidence when a run cannot emit its own final record.

Output example

Illustrative event excerpt: omitted envelope and data fields are intentional. These values are not experiment results or detection thresholds. Complete event envelopes are documented in the implementation guide.

{
  "kind": "timing",
  "data": {
    "phase_id": 2,
    "elapsed_ms": 20.0,
    "tokens_processed": 1.0,
    "tokens_per_second": 50.0
  }
}

Implementation source

Expand a file-and-line reference to read the actual source. These excerpts are embedded snapshots from this checkout. They show the measurement implementation; the experiments and output examples above remain illustrative.

Read configuration and validate positive limits

src/cst_capture.cpp:83–92View code

83int main(int argc,char ** argv){84    if(argc!=2){std::cerr<<"Usage: cst-capture config.json\n";return 2;}85    try {86        json j=json::parse(read_file(argv[1]));87        std::string model_path=j.at("model"),prompt=read_file(j.at("prompt_file"));88        std::string log=j.value("log_path",std::string("trial.cstbin")),jsonl=j.value("jsonl_path",std::string("trial.telemetry.jsonl"));89        std::string run=j.value("run_id",std::string("local"));90        int max_tokens=j.value("max_tokens",32),nctx=j.value("context_size",4096),stride=j.value("score_stride",1);91        if(max_tokens<=0 || nctx<=0 || stride<=0)throw std::runtime_error("invalid token/context/score limits");92        auto hash=hash_file(model_path);

Rendered input, capture settings and fresh context

src/cst_capture.cpp:107–137View code

107        auto input=tokenize(vocab,prompt,true);108        if(input.empty() || input.size()+(size_t)max_tokens>(size_t)nctx)throw std::runtime_error("prompt plus generation exceeds context_size");109        std::vector<int> layers=j.value("hidden_layers",std::vector<int>{0,llama_model_n_layer(model.get())-1});110        for(int l:layers)if(l<0 || l>=llama_model_n_layer(model.get()))throw std::runtime_error("invalid hidden layer");111        layers.push_back(-1);112        cst_config_t cfg{};cfg.log_path=log.c_str();cfg.jsonl_path=jsonl.empty()?nullptr:jsonl.c_str();cfg.run_id=run.c_str();113        cfg.hidden_state_layers=layers.data();cfg.logit_distribution_top_k=j.value("top_k_capture",20);cfg.emit_token_entropy=1;114        cfg.emit_model_timing=1;cfg.emit_kv_cache_state=j.value("capture_kv_projections",false);cfg.kv_cache_stride=j.value("kv_stride",8);115        cfg.capture_full_logits=j.value("full_logits",false);cfg.hidden_projection_dims=j.value("projection_dims",0);116        cfg.capture_hidden_vectors=j.value("hidden_vectors",false);cfg.capture_attention_rows=j.value("attention_rows",false);117        cfg.capture_prompt_text=j.value("capture_prompt_text",false);cfg.max_tensor_bytes=j.value("max_tensor_bytes",64u*1024u*1024u);118        if(cst_init(&cfg))throw std::runtime_error("telemetry initialization failed");119        cst_trial_boundary(j.value("trial_id",0u),-1,-1);120        event("run_metadata",{{"model_path",model_path},{"model_sha256",hash},{"tokenizer_identity","embedded_in_model_sha256"},121             {"architecture",arch},{"llama_commit",CST_LLAMA_COMMIT},{"configuration",j},122             {"prompt_contract","already_rendered_exact_text; no chat template applied"},{"system_info",llama_print_system_info()}});123        auto cp=llama_context_default_params();cp.n_ctx=nctx;cp.n_batch=1;cp.n_ubatch=1;124        cp.n_threads=j.value("threads",4);cp.n_threads_batch=cp.n_threads;125        cp.flash_attn_type=LLAMA_FLASH_ATTN_TYPE_DISABLED;cp.cb_eval=cst_eval_callback;cp.no_perf=false;126        Context ctx(llama_init_from_model(model.get(),cp),llama_free);127        if(!ctx)throw std::runtime_error("context init failed");128        event("capture_capabilities",{{"mode","diagnostic_single_token_decode"},{"flash_attention",false},129             {"context_size_actual",llama_n_ctx(ctx.get())},{"cache_key_mapping","linear_single_sequence_no_shift"},130             {"stored_kv_rows","unavailable"},{"speculative_decoding","disabled"},{"recurrent_state","unsupported"},131             {"tensor_support","observed tensors reported individually; absence is not zero"}});132        for(const auto & r:j.value("regions",json::array())) {133            auto meta=r.value("metadata",json::object()).dump();std::string src=r.value("source_id",std::string(""));134            int start=r.at("start_token"),end=r.at("end_token");135            if(start<0 || end<=start || end>(int)input.size())throw std::runtime_error("region outside tokenized prompt");136            cst_add_context_region(start,end,r.at("region_type"),r.at("trust_level"),src.c_str(),meta.c_str());137        }

Generated bytes, stopping, decode timing and logical cache appends

src/cst_capture.cpp:219–240View code

219            for(auto & s:samplers)llama_sampler_accept(s.get(),id);220            std::string text=piece(vocab,id);cst_log_token_bytes(text.data(),text.size());221            event("special_token",{{"token_id",id},{"is_eog",llama_vocab_is_eog(vocab,id)},222                  {"eos_raw_logprob",llama_vocab_eos(vocab)>=0?json(selected_logprob(raw,nv,llama_vocab_eos(vocab))):json(nullptr)}});223            if(t==0){double ttft=ms(trial_start);cst_log_timing(CST_TIMING_FIRST_TOKEN,ttft,1,ttft>0?1000.0/ttft:0);}224            cst_log_generated_token(id,raw,nv);++generated;prefix.push_back(id);225            if(llama_vocab_is_eog(vocab,id)){stop="eog";break;}226            std::cout<<text<<std::flush;227            if(t+1==max_tokens)break;228            cst_set_position(0,0,(int)prefix.size()-1,t+1,"decode");229            keys.push_back((int)prefix.size()-1);cst_set_attention_key_positions(keys.data(),(int)keys.size());230            start=Clock::now();decode_one(ctx.get(),id);elapsed=ms(start);231            cst_log_timing(CST_TIMING_DECODE,elapsed,1,elapsed>0?1000.0/elapsed:0);232            auto memory=llama_get_memory(ctx.get());233            event("cache_lifecycle",{{"operation","append"},{"min_position",llama_memory_seq_pos_min(memory,0)},234                  {"max_position",llama_memory_seq_pos_max(memory,0)},{"logical_tokens",keys.size()},235                  {"allocated_context_capacity",llama_n_ctx(ctx.get())}});236        }237        elapsed=ms(trial_start);cst_log_timing(CST_TIMING_TOTAL_TRIAL,elapsed,generated,elapsed>0?generated*1000.0/elapsed:0);238        event("generation_end",{{"stop_reason",stop},{"generated_tokens",generated},{"prompt_tokens",input.size()}});239        cst_close();std::cout<<"\n";return cst_last_error()?1:0;240    }catch(const std::exception & e){event("generation_end",{{"stop_reason","error"},{"message",e.what()}});cst_close();std::cerr<<e.what()<<"\n";return 1;}

Callback coverage and unavailable tensors

src/cst_hook.cpp:29–76View code

29extern "C" bool cst_eval_callback(ggml_tensor * t,bool ask,void *) {30    if(!t)return !ask;31    std::string name;int layer=-1;32    if(!parse_name(t->name,name,layer))return !ask;33    bool hidden=name=="l_out" && cst_hidden_layer_enabled(layer);34    bool attention=name=="kq_soft_max";35    bool output=name=="attn_out";36    bool router=name=="ffn_moe_logits";37    bool selection=name=="ffn_moe_topk";38    bool kv=(name=="Kcur" || name=="Vcur") && cst_active_config().emit_kv_cache_state;39    if(ask)return hidden || attention || output || router || selection || kv;40    if(!(hidden || attention || output || router || selection || kv))return true;41    const size_t bytes=ggml_nbytes(t);42    if(bytes>cst_active_config().max_tensor_bytes || !ggml_is_contiguous(t) ||43       (selection?t->type!=GGML_TYPE_I32:t->type!=GGML_TYPE_F32)) {44        event("signal_unavailable",{{"tensor",t->name},{"reason","requires contiguous F32 (I32 expert IDs) within byte cap"},45                                  {"tensor_type",ggml_type_name(t->type)},{"bytes",bytes}});return true;46    }47    // This adapter intentionally accepts single-query graphs only. Never silently label batch rows as one token.48    bool single=attention?(t->ne[1]==1 && t->ne[3]==1):49                kv?(t->ne[2]==1 && t->ne[3]==1):(t->ne[1]==1 && t->ne[2]==1 && t->ne[3]==1);50    if(!single){event("signal_unavailable",{{"tensor",t->name},{"reason","adapter requires single-token single-sequence decode"}});return true;}51    if(selection) {52        std::vector<int32_t> data(bytes/sizeof(int32_t));53        ggml_backend_tensor_get(t,data.data(),0,bytes);54        cst_log_expert_selection((uint16_t)layer,data.data(),(int)data.size());return true;55    }56    std::vector<float> data(bytes/sizeof(float));57    ggml_backend_tensor_get(t,data.data(),0,bytes);58    if(hidden)cst_log_hidden((uint16_t)layer,data.data(),(int)data.size());59    if(router)cst_log_router((uint16_t)layer,data.data(),(int)data.size());60    if(kv)cst_log_kv_projection((uint16_t)layer,name.c_str(),data.data(),(int)data.size());61    if(output) {62        double sq=0;for(float v:data)sq+=(double)v*v;63        event("attention_output",{{"layer",layer},{"l2",std::sqrt(sq)},{"dimensions",data.size()}});64    }65    if(attention) {66        int n=(int)t->ne[0];67        // Padding keys are unmapped. Mapping must be supplied by the integrating caller after any cache mutation.68        std::vector<int32_t> keys(n,-1);69        const int32_t * mapping=nullptr;70        if(!key_positions.empty() && key_positions.size()<=(size_t)n) {71            std::copy(key_positions.begin(),key_positions.end(),keys.begin());mapping=keys.data();72        }73        for(int h=0;h<t->ne[2];++h)cst_log_attention((uint16_t)layer,h,data.data()+(size_t)h*n,n,mapping);74    }75    event("tensor_observed",{{"tensor",t->name},{"layer",layer}});76    return true;

Implementation status and limits

The driver uses single-token prefill and disables Flash Attention for visibility. Timings are diagnostic. It does not collect process RSS, GPU utilization, energy use or per-allocation traces. No configurable wall-time cancellation is implemented. Context capacity is not a measurement of actual memory bytes.

All tensor-based interpretations depend on actual tensor_observed coverage and the tested model/backend. See validation results. There is no universal security score or calibrated threshold in this implementation.

Offline HTML with embedded source excerpts. Rebuild from the repository root with python3 scripts/build_security_docs.py after changing the Markdown or source files. Install scripts/requirements-docs.txt first; pass --llama-cpp-dir /path/to/llama.cpp for a different checkout location. The builder requires the pinned source revision. Review excerpt ranges if source lines move. No inference runs in this page.