Attaching as a Callee: Defensive Probes

Host: Welcome back to Runtime AI Security. This is episode four, and the title is one that made me pause when I first saw it: Attaching as a Callee, Defensive Probes. Over the last three episodes we've been living in the attacker's head. We looked at why a trusted author is not the same thing as safe code, then at how you can jump a sandbox without ever touching the sandbox itself, and last time at impersonation, time bombs, and legal loopholes. Today we flip it. What does the defense actually look like?

Expert: Today we're in chapter three of the book, and the framing there is quite specific. To validate and harden the sandbox boundary, the security tester attaches directly to four things: the sandbox, the trust verification system, the application, and the policy engine. They attach as defensive probes. The important word in that list, for me, is the policy engine.

Host: Why the policy engine specifically? I'd have assumed the sandbox is the thing you test.

Expert: Because that assumption is exactly the gap the book is pointing at. The manuscript says it plainly: the goal is not merely to test whether the sandbox works, but to test whether the host's policy engine can be manipulated into disabling the sandbox. Those are two completely different questions. You can have a technically excellent sandbox with a policy layer that will cheerfully stand down when it sees a verified publisher certificate. If you only test the first thing, you'll pass every audit and still be wide open.

Host: So the security system itself becomes the target of the test.

Expert: That's the phrase the book uses. Treating the security system itself as the target of testing. And that lets an organization proactively identify vulnerabilities in how trust signals influence technical isolation. Not whether they influence outcomes at the user interface level, but whether they influence isolation.

Host: Let's walk through the probes. Where does the suite start?

Expert: With the foundation, which the book calls the zero-trust enforcement probe. This probe attaches to the policy enforcer and verifies that it ignores all author trust signals when it determines sandbox permissions. Concretely, it confirms that identical isolation parameters are applied to every skill. The book names three: filesystem access, network egress, and system call filtering. Same parameters, every skill, regardless of author reputation. And it confirms that no skill can bypass sandbox restrictions on the basis of signatures or endorsements.

Host: Identical is a strong word. There's no tier where a well-known publisher gets a slightly wider lane?

Expert: No, and that's deliberate. If there is a wider lane, the entire adversary strategy from episode two becomes: get into the wider lane. You've handed them a target that's far easier than the sandbox itself. Uniformity isn't a nicety here, it's what makes the reputation attacks pointless.

Host: But the book doesn't say throw reputation away entirely. There's still a trust score in the picture.

Expert: Correct, and that's the second probe, trust score isolation. The trust scoring engine still does its job. It computes a score from author reputation, account age, contribution history, and community endorsements. The probe verifies that this score is used only for risk assessment and user education. It validates that the trust score is never used to grant additional permissions or to relax sandbox controls.

Host: So where does the score actually show up?

Expert: In the permission prompt. The book is explicit: the trust score should be displayed to the user, to help them make an informed decision. But it must not alter the actual sandbox restrictions. That's the clean separation. The score talks to the human. It does not talk to the enforcer.

Host: That's a nice line. The score is a conversation with the user, not an instruction to the machine.

Expert: That's essentially the book's thesis compressed into one probe.

Host: Okay. Next group in the chapter is detecting compromise and malicious behavior. This is where you catch the attacks we described last episode?

Expert: Yes. The book's argument is that supply chain attacks are dynamic, so you need behavioral and anomaly detection probes rather than a one-time verdict. The first one is malicious update detection. It attaches a behavioral change monitor that compares a new skill's behavior against the previous version's behavior. Three dimensions again: syscall patterns, network destinations, and filesystem access.

Host: And what triggers an alert?

Expert: Significant behavioral change. The book gives a lovely, very concrete example: a markdown formatter that suddenly starts making network calls. Nothing about that update is illegal, nothing fails a signature check, but a tool that turns text into other text has no business opening sockets. The probe also flags any update that adds new dependencies, new network endpoints, or new system calls.

Host: That's a diff, but of behavior instead of source code.

Expert: Exactly right, and it's a diff the attacker can't easily hide behind a valid signature. Recall from episode two that the SolarWinds artifact was legitimately signed. Signature verification had nothing to say about it. Behavioral change monitoring does.

Host: What about catching the account takeover before the bad update ships?

Expert: That's the account compromise detection probe. It monitors the author account for suspicious activity. The book lists a sudden change in geographic location, the addition of new SSH keys, and a recent password change. When it sees those signs, it doesn't block outright, it triggers enhanced sandbox restrictions for skills published from that account.

Host: Enhanced restrictions rather than a hard block. That feels like a deliberate design choice.

Expert: It is, and it fits the philosophy. Trust signals shouldn't unlock things, but suspicion signals can tighten things. Loosening is the dangerous direction. Tightening is safe.

Host: And the sock puppet farming we talked about?

Expert: Reputation farming detection. That probe examines the social graph of endorsements and collaborations between developer accounts. It's looking for coordinated sock puppet rings, and the book gives two identifying patterns: multiple accounts sharing the same IP address, and a rapid creation of new accounts that all endorse the same skill. It's a graph problem, not a per-account problem, which is why individual account review misses it.

Host: Then there were the impersonation attacks. The Cyrillic character trick.

Expert: The impersonation and homoglyph detection probe compares skill metadata against a registry of known trusted brands, and detects homoglyph attacks and typosquatting. Machines are actually better than humans at this one. A person cannot see the difference between those two characters. A comparison against a brand registry can.

Host: And the deepfake endorsements?

Expert: There's a deepfake and synthetic content detection probe that analyzes video and audio endorsements for signs of AI-generated content. But the crucial part is the default it sets. It treats unverifiable endorsements as untrusted rather than as positive signals. That's the correct failure mode. If you can't verify it, it doesn't count in your favor.

Host: And that covers the legal loophole we discussed last time too?

Expert: There's a dedicated Terms of Service scanner. It parses the skill's legal documents to detect clauses that permit malicious actions, and it flags them for the user in plain language. Remember the vector: an author writes terms that make the user accept all liability, and a naive automated policy engine reads that as a legitimate disclaimer and lowers the risk score. The scanner's job is to surface those clauses to a human in words they can understand, rather than let an automated scorer quietly reward them.

Host: Let's move to supply chain. The book has probes for the build pipeline itself.

Expert: The build pipeline integrity check verifies that the entire publication chain, from source code to published artifact, is free from tampering. It blocks skills published through compromised pipelines. And then there's the requirement that I think is the single most operationally interesting line in the chapter: it requires the author to sign the final artifact with a hardware security key directly, not through CI/CD automation.

Host: Why does that distinction matter so much?

Expert: Because of the SolarWinds-shaped hole. If your signing key lives inside the build system, then compromising the build system gets you signing. The book's phrasing is that this requirement ensures that even a fully compromised build system cannot produce a validly attested artifact without physical access to the author's hardware key. You've moved the last step of trust out of software and into a physical object that a remote attacker cannot reach.

Host: So automation is the convenience that quietly becomes the vulnerability.

Expert: That pattern recurs throughout the whole book. Convenience exceptions are attack surface.

Host: What about the malicious dependency problem? The trusted author whose library isn't trustworthy.

Expert: Dependency trust isolation. And the design here is counterintuitive but correct. It applies stricter sandboxing to dependencies than to the main skill. That's defense in depth. All dependencies run in a separate, highly restricted namespace with no network access and no filesystem writes, and they're monitored for suspicious behavior independently of the main skill.

Host: Independently is doing work there.

Expert: It is. The failure mode from episode two was that the host sees the primary author's reputation and relaxes sandboxing for the entire application, which inadvertently grants the malicious dependency elevated access. Monitoring the dependency on its own terms breaks that inheritance of reputation.

Host: And time bombs? The attack where the payload waits for the trust to go stale.

Expert: The time-aware trust validator. It requires trust to be re-verified periodically, and the book's example figure is every thirty days, for high-risk permissions. It also flags skills that have been dormant for long periods and are suddenly updated, and triggers enhanced verification for delayed payloads. The book says explicitly that this probe is designed to counter time-based trust activation attacks.

Host: So trust has an expiry date rather than being a permanent property.

Expert: Right. A trust signal that never expires is a trust signal that only ever gets more stale.

Host: The last group in the chapter is fuzzing, transparency, and incident response. Tell me about the fuzzing part, because I gather this is where you attack your own trust system.

Expert: This is my favorite section. First there's decoy trust manipulation. You deliberately inject a fake trusted author skill into your own system and monitor whether it gets granted elevated privileges based on that fake trust signal. It's a canary for policy drift. If your decoy ever gets more than baseline permissions, you've learned that somewhere in your stack, reputation is still wired to the enforcer.

Host: And the fuzzer?

Expert: The trust boundary fuzzer attempts to impersonate a trusted author at every stage. The book lists metadata, signature, commits, and endorsements. Then it validates that the sandbox does not relax isolation even when all trust signals appear valid. That last clause is the whole point. Not when the signals look suspicious. When they all look perfect. And any bypass discovered is reported immediately to the security team.

Host: You're testing the assumption that a flawless-looking author changes nothing.

Expert: Yes. And then the policy guard closes the human loophole. It must prevent any manual override of sandbox restrictions based on author trust. The book is blunt about the specific control: there must be no allow this publisher button that grants persistent elevated permissions. All permission grants are per-session and per-skill.

Host: That's the sort of button every product manager wants to ship, though.

Expert: It is, and it's precisely why the book calls it out. It's the most requested feature and the cleanest bypass. If a listener wants one concrete thing to act on this week, that's it. Go look at your own system, or the tooling you use, and find every place where a user can say always allow this publisher. Every one of those is a persistent trust grant that survives account takeover, survives a compromised build pipeline, and survives a time bomb. Convert them to per-session, per-skill grants. That single change removes an entire category of the attacks we covered in episodes two and three.

Host: Good. And on transparency?

Expert: The trust score transparency module gives the user a clear, human-readable explanation of the trust score and displays the evidence behind it. The book's illustration is something like: this author has made fifty previous contributions and has two hundred stars. The user can see exactly why a skill received a particular score. It respects the user enough to show its work rather than handing down a number.

Host: There's also a community verification mechanism mentioned, isn't there?

Expert: A decentralized community verification mechanism, where multiple independent verifiers attest to the author's identity and the skill's safety, requiring a quorum before a skill is considered community-vetted. The book's example quorum is five out of seven. But it immediately adds the caveat that runs through everything: community vetting must never be used as a basis for relaxing sandbox restrictions. It's a supplementary signal only.

Host: Which brings us to behavior at runtime.

Expert: Continuous behavioral trust scoring. It evaluates the skill's behavior during execution, not just at installation. A skill that behaves benignly receives higher real-time trust. A skill that starts making unexpected network calls receives lower trust and faces dynamically increased sandbox restrictions. Notice the asymmetry again. Good behavior earns a better score. Bad behavior earns tighter walls, immediately.

Host: And when something does go wrong?

Expert: The incident response hook. If any one skill from an author is found malicious, it immediately downgrades trust for all skills from that author, alerts every user who has used that author's skills, and revokes all elevated permissions across the entire system. That's blast radius containment. One compromised skill from an author is evidence about the author, not just about that skill.

Host: So if I zoom out, this suite of probes is really one argument made about fifteen different ways.

Expert: It is. Every probe is asking the same question from a different angle: can a trust signal reach the enforcer? Zero-trust enforcement asks it at the policy layer. Trust score isolation asks it at the scoring layer. The decoy and the fuzzer ask it adversarially. The policy guard asks it at the user interface. And behavioral monitoring asks it continuously, at runtime, after every static check has already passed.

Host: That's a good place to hand off. Next episode we take the last step and put all of this into a single structure.

Expert: Chapter four, the secure boundary model, and its thirteen principles. Everything we've discussed today becomes a control mapped to a principle, from the zero-trust baseline through to red-team resilience. It's the design document version of this conversation.

Host: Then we'll see you for episode five, The Secure Boundary Model and Its Thirteen Principles. Thanks for listening to Runtime AI Security.