1. The Problem of Growing AI Clogs
By: Andrew Gamino-Cheong
Muse, Meta’s new AI agent app, is aiming to be the personal assistant every science fiction movie shows. Meta’s pitch is an assistant that handles your errands for you, from booking dinner to sitting on hold with your insurer. Three weeks in, it’s already starting to show societal impacts, specifically, undermining the assumptions behind most refund policies and retention scripts. The New York Times‘ Eli Tan watched it cancel duplicate subscriptions and claw back a refund on an expired newspaper trial. Forbes found users recovering stale balances and insurance overpayments they’d never have chased by hand. Investors noticed first, and subscription stocks slumped.
As the assistant is further adopted, it is likely to create additional AI clogs, where AI-driven volume overwhelms a human-run process until it breaks down. Academic AI conferences are the clearest preview. AAAI received more than 30,000 submissions for 2026, roughly double the year before. At ICLR, 21% of peer reviews were fully AI-generated, written by reviewers juggling several papers on two-week deadlines. Researcher Chris Schmitz has documented a public-sector version, which he calls agentic flooding, across 84 government services. Government processes getting clogged up degrades access and availability of benefits and services, creating cascading problems.
In the past, clogs mostly needed an adversarial incentive or real technical skill. Muse gives a working agent to ordinary consumers with legitimate claims, and its persistence may be a problem. Companies build refund and cancellation processes around predictable claim rates, and friction is part of the model. A 2025 American Economic Review paper estimated that cancellation hurdles roughly double subscription revenue. An agent that waits on hold indefinitely and keeps trying alternatives until one works makes those assumptions worthless. Firms will respond with verification steps, restrictive terms of service (as Amazon has) or slower channels. The customers still calling on their own will end up waiting behind agent queues.
Key Takeaway: Organizations will need to rethink a lot of customer policies knowing that agents may now act on behalf of people. Additional verification steps may work for now, but apps like Muse are being given all the sensitive information and ability to pass many verifications. Regulators should push for agent identification standards so that firms and agencies can manage agent traffic without degrading service for everyone else.
2. The AI Governance Implications of Jev / System One Models
By: Andrew Gamino-Cheong
Large Language Models have a few core limitations that stem from the nature of their architecture and how they’re trained. They require massive amounts of pre-training data, they demand massive amounts of compute that makes them slow and expensive, predicting the next token without any true grounding or model of the world means hallucinations will always exist, and it’s unclear if they can ever be secured. As a result, leading AI researchers have been looking at alternative model architectures that can still create general purpose AI systems without some of those limitations. Recently, TypeSafe AI announced one such model, Jev, that aims to tackle the hallucination, safety, and expense problems. An LLM writes its answer one word at a time as free-form text, which software then has to parse and check before acting on it. Jev instead answers a set of questions all at once, choosing only from options defined in advance, and attaches a confidence score to each answer. TypeSafe prices Jev at $42 per billion input tokens with no charge for output, roughly two orders of magnitude cheaper than frontier LLMs. This model structure has been dubbed a ‘System One’ model, inspired by Daniel Kahneman’s concept of the brain having two modes of thinking, with the first system being quick and intuitive, and the second being slow and rational.
Jev has several potential governance-related advantages over existing LLM architectures. TypeSafe AI recognizes this, and in their announcement video, they summed up their approach as “we’re building prod, not God”, meaning their focus is on reliable enterprise AI, not AGI. Because every output comes from a fixed list of options with a confidence score, the system cannot generate harmful outputs or leak data. These models will be easier to test as well as evaluating structured answers/outputs is easier than evaluating the quality of generated content. The narrow output also limits what a manipulated input can do, since an instruction hidden in a document can’t make Jev write an email or call a tool, only pick among the answers it was given. None of that removes the risk of bias however. If you give Jev a resume and ask for a hire or no-hire decision, and you’ll get a clean answer with a confidence score that’s no more explainable than an LLM’s. TypeSafe’s claim that Jev “can’t hallucinate” also only means it can’t answer outside the options it’s given, but its performance results haven’t been independently verified yet.
Key Takeaway: New model architectures will keep arriving, each with its own governance trade-offs. It’s an open question whether a model like Jev, which reads text but only returns structured decisions, even counts as a GPAI model under the EU AI Act, and the answer could shift as it’s used for more tasks. Governance teams will need to understand those trade-offs before these models reach production.
3. Trustible Spotlight
By: Lauren Madden
It’s been a busy stretch of events for the Trustible team! We hosted a panel at the Coalition of Healthcare AI (CHAI)’s Legal Summit in Boston and traveled to London for the Responsible AI Summit in the following week to present our insights on AI monitoring.
In Boston, our CEO sat down with Mass General Brigham’s Amanda Centi and Melissa Bateman Fitzgerald to unpack what AI monitoring means.
The CHAI conversation centered on a key argument: monitor the use case, not the model. A single use case can run on several models, so watching one endpoint reveals nothing about the rest.
The panel argued for anchoring monitoring to the use case instead, then split it into two halves: (1) internal signals like cost, quality, drift, and safety, which most teams already track to some degree, and (2) external signals like vendor changes, public incidents, and shifting regulation, which is where the real exposure tends to hide. See the exact presentation from our panel.
4. Policy Updates
By: Sydney Cullen
California - Governor Gavin Newsom signed an executive order accelerating the timeline for implementation of SB 813, the law establishing an Independent Verification Organization certification framework in the state. Newsom aims to accelerate implementation prior to his departure as governor in January of 2027. The order sets a May 1, 2027 deadline for the state to publish IVO application requirements and criteria, and a December 1, 2027 deadline for the registry work. It also directs state agencies, working with national experts, to deliver recommendations by Nov. 16, 2026 on amending California law. Those include requiring frontier labs to embed an IVO on-site, independently verifying the safety filings labs must submit, creating a “kill switch” for frontier models, and expanding the definition of critical safety incidents.
Our take: The AI policy landscape has shifted in the last few weeks, leading policy makers to try to seize the moment while public sentiment largely supports more frontier lab oversight. A Reuters/Ipsos poll from mid-September found 73% of respondents worry AI companies haven’t done enough to prevent serious harm. Anthropic’s partnership with Accenture on embedded evaluation is likely a net positive, but Anthropic is funding Accenture’s work directly, which is why an IVO structure is needed to ensure truly independent audits. Organizations should expect successful audit results to become a credibility point when procuring frontier tools.
OpenAI - OpenAI disclosed a run of incidents this past week in which its agents accessed government websites, though most happened months ago. First reported was an incident at Australia’s Medicare system, where an agent accessed non-public but non-sensitive files earlier this summer. OpenAI didn’t notify Australia until Sept. 10. Days later, AI evaluator Transluce said agents that appeared to be OpenAI’s tried unsuccessfully to hack the US Department of Education’s website, which OpenAI has not confirmed. AP reported that OpenAI agents also acted in unexpected ways on other federal sites, including the SEC and Census Bureau. OpenAI is under fire for its delayed notification and has apologized to Australian officials. It has also paused training of its latest models, its second halt in three months.
Our take: These incidents show why organizations need monitoring and review cadences of their own, outside the labs’ cycles. No lab has yet demonstrated a reliable methodology for monitoring and alerting on this kind of behavior, so organizations should define their own metrics and be able to track incidents against their AI inventory.
EY Survey - EY’s survey of 202 senior AI executives confirmed what many in AI governance already know: there is a gap between establishing policies and implementing them. While 98% reported having formal AI governance policies, 63% were concerned about lacking the internal expertise to implement them effectively. Executives also voiced concern about third-party AI-enabled cyber attacks (81%) and a high-profile AI failure damaging their reputation (75%). Even so, 47% admitted their organization has previously not applied its AI governance process for urgent deployments.
Our take: Policies are only as good as their implementation, which is easier said than done during rapid technological change. Organizations should decide in advance what they’re willing to move fast on and what they need to get right. Use cases involving cybersecurity or decisions about people should face a higher threshold than an agent that completes administrative tasks like organizing nonsensitive files.
In Case You Missed It -
UNGA - On Sept. 23, the UN Security Council held a session on AI and international security, where OpenAI’s Sam Altman and Anthropic’s Dario Amodei urged shared testing standards and a system for countries to report serious AI incidents. The US rejected new global governance structures, and the session produced no binding commitments.
White House - It’s been a big week and a half:
Sept. 22: At the UN General Assembly, Trump announced the US will call AI “Super Intelligence“ (SI) in government documents, a name that won a Truth Social poll.
Sept. 23–25: Xi Jinping made his first state visit to Washington in 11 years, with AI on the agenda. The leaders voiced diverging views and reached no AI breakthrough, though the White House said the two countries will set up a bilateral communication channel for AI incidents. Trump has since said he doesn’t want joint AI ventures or governance efforts with China.
Sept. 29: Trump and leading lab executives signed the voluntary “White House Accord on Super Intelligence.” Participating companies commit to internal controls, an internal oversight team, an independent external auditor or evaluator, and a board committee to review their reports. The signers were the CEOs of Google, Anthropic, Meta, xAI and Nvidia, plus OpenAI’s president.
The Vatican - Pope Leo XIV has reinforced his stance on the need for AI legislation and guardrails. He pushes back on President Trump’s claims that the AI safety debate is a “hoax” and called for dialogue among political leaders, AI developers and civil society, consistent with his encyclical’s call for robust AI regulation.
—
As always, we welcome your feedback on content! Have suggestions? Drop us a line at newsletter@trustible.ai.
AI Responsibly,
- Trustible Team