An Organizational Second Brain: Building an AI That Learns From Experts

This Meta Engineering blog describes an internal AI agent for a specific compliance domain that is meant to capture specialist knowledge usually stuck in people's heads. The novelty, as Meta tells it, is two layers: a structured, auditable knowledge architecture that separates what the agent knows from how it reasons, and a self-improvement loop that compiles expert feedback into verified, regression-tested file updates without model retraining. More than 200 files sit in a taxonomy: position files, vocabulary, routing indexes, gateway files, with YAML depends_on / referenced_by forming a dependency graph. High-density knowledge lives in the wiki; sparse material stays in RAG. Procedures are 'recipes' that reference knowledge but contain no domain facts. Restructuring from a flat instruction file cut tokens per turn by around 80%.

Human experts remain in the loop via checkpoints and escalations. The improvement loop diagnoses corrections (knowledge gap vs recipe vs genuine ambiguity), compiles minimal edits, runs an adversarial reviewer in a fresh context, lints structure, replays the original scenario with a blind judge, and regression-tests. Failures retry. Landed fixes become new regression cases. Meta reports SMEs now rate outputs useful almost all the time, experts say the agent handles the vast majority of analytical work, automated edits at a pace that used to take engineering sprints, and zero regressions across improvement cycles. The authors say the pattern fits other text-governed domains: finance, security, engineering standards.

Vendor case study. The domain is unnamed beyond 'compliance,' the usefulness numbers are unquantified ('almost all the time,' 'vast majority'), and zero regressions is a claim from the team that built the gates. Discount accordingly. The architecture is still one of the clearer enterprise-agent writeups out: put the institution in files, not weights, and treat expert chat as a compiler target.

Karpathy's LLM Wiki and Google's Open Knowledge Format are cited as cousins. Meta's addition is the validation rigor — blind replay, adversarial diff review, deterministic lint — on a knowledge base that lawyers can read. That is the point in compliance. If a position changes, you want a diff, not a finetune.

Signal Desk's read: this is how you build an agent you can audit. It is also a lot of machinery for a problem RAG demos pretend is easy. The 80% token cut from recipes is the practical win; progressive disclosure beats a stuffed context. 'Without retraining' is the slogan that will travel. It is true only if experts keep correcting, and if the compiler's edits stay smaller than the graph. Ambiguity flagged for human discussion is the adult failure mode. Force a resolution and you get confident wrongness.

Other companies should steal the split (knowledge vs recipes) and be skeptical of 'zero regressions' until they see the benchmark design.

Enterprises have been bolting chat onto document piles and calling it institutional knowledge. The documents are not the knowledge; the reasoning is. Meta is describing an internal attempt to pre-extract that reasoning for a high-stakes review workflow where inconsistency is a risk.