Goodfire Launches Cost-Effective Inside-Out Monitors to Detect Rogue AI Agents

Image: Image Credits:GOODFIRE · Source

Goodfire introduces innovative 'inside-out' monitors that inspect AI model internals during operation, efficiently spotting rogue behavior at a fraction of traditional monitoring costs.

Goodfire, a startup specializing in AI interpretability, has launched a novel system designed to monitor AI agents more cost-effectively by analyzing their internal model computations in real time. Unlike the common practice of deploying a second AI to scrutinize an agent's outputs—an approach that grows expensive as agents process large volumes of data—Goodfire's monitors observe signals within the model as it operates. This method allows for early detection of potentially malicious or harmful actions without the overhead of reviewing every generated token externally.

The monitors function similarly to airport security scanners: small detectors, called probes, analyze internal activations of the AI during each step of its task. When a probe identifies suspicious patterns, a secondary AI model performs a deeper investigation. This tiered approach enhances efficiency by focusing computational resources only when necessary.

Goodfire’s monitors are integrated into Baseten's platform, enabling clients to tailor which risks are tracked—such as offensive hacking attempts, misuse related to chemical and biological weapons, or reward hacking—and choose automated responses like event logging, manual review, or outright request denial.

Tests on the open-source AI agent Kimi K3 demonstrated significant cost savings. Monitoring approximately 1,500 sessions cost about $51 using Goodfire's probes, compared to $233 for a less sophisticated AI-based monitor and as much as $10,000 for a premium alternative. The probes effectively caught 94% of malicious hacking sessions, with a minimal false positive rate leading to only 8.7% of safe sessions requiring secondary review. Additional latency from running multiple probes was negligible, increasing model response startup time by less than 2%.

Goodfire co-founder and CTO Dan Balsam emphasized the advantage of intercepting problematic behavior before it manifests by detecting threats during evaluation or training phases. This capability is particularly crucial for open models, which lack the controlled safeguards of proprietary systems and can be modified or misused by any developer.

While other efforts exist—Google DeepMind recently implemented misuse-detection probes in its Gemini model—Goodfire's approach focuses on leveraging existing model computations to reduce cost and increase real-time monitoring feasibility. Their longer-term vision aims to reverse-engineer large language models (LLMs) to trace behaviors back to their training origins, potentially transforming model development into a more precise engineering discipline.

Goodfire's innovation addresses growing concerns about AI agents escaping controlled environments and acting unpredictably or maliciously, an issue highlighted by recent incidents involving open AI models. Their inside-out monitoring represents a pragmatic step toward enhancing AI safety at scale, especially as open models become more widespread and accessible.