Knowledge is the layer that makes Agentic AI dependable
Knowledge Sidekick is a practical knowledge layer for organizations building or using Agentic AI, LLMs, and personal assistants. This org site is the free-to-read side of that work: not-for-profit, no fee, no ads. It covers research, standards and adoption practice — open and proprietary alike — and the courses.
Start here, depending on where you are
Four situations bring people to this site. Pick the one that is yours and follow two or three pages; each path is a short route, not a curriculum.
Nothing is broken yet. You want the knowledge layer designed before it becomes the thing that fails in production.
-
1
Why knowledge fails first in agentic systems
The core argument, with the published evidence for it. Twenty minutes, and it sets the vocabulary for everything else.
-
2
DIKW: the shape of the layer
What the knowledge layer actually contains, from data to wisdom, and where agentic systems usually stop too early.
-
3
Context engineering
Context as a knowledge-delivery problem rather than a bucket. The design constraint most teams discover too late.
You are mid-project and need conventions that hold: formats, standards, and an interface agents can rely on.
-
1
Knowledge formats
How to shape knowledge so agents can use it: entities, facts, procedures, policies. Consistent enough to trust.
-
2
Standards index
RDF, SKOS, JSON-LD, PROV-O and the rest, each with its real status stated plainly. llms.txt is labelled a proposal, not a standard.
-
3
MCP knowledge resources
Exposing the knowledge layer to agents through the Model Context Protocol: servers, tool-backed knowledge, and what dependable means.
The system is live and the hard part is not the model. These three pages map directly onto the three ways knowledge fails: stale, ungrounded, too broad.
-
1
Why knowledge fails first
Deprecation and staleness: the system quotes what used to be true. The failure mode that looks like a model problem.
-
2
Grounding is the flip side of token savings
The claim and the receipt: tying what the agent says to knowledge it can point to, and what that costs or saves.
-
3
Memory research
Scoped versus broad retrieval, with the measurement: 0.713 against 0.816, and broad retrieval falling below no memory at all.
You publish knowledge so agents can use it, not only so people can read it. This site calls the work AEO (agent engine optimization); the term is this site’s own shorthand, and llms.txt is a proposal, not a standard.
- 1llms.txt
The file that hands an agent engine a map of your knowledge layer: what it does, what it does not do, and its status as a proposal.
- 2Agent interface conventions
The documented surface an agent relies on: metadata, structure, and the disciplines that make a page machine-usable without flattening it.
- 3Free resources
Templates and checklists from this site’s own setup: the agents.md and llms.txt this page is served with, as working examples.
What Knowledge Sidekick is
Most accounts of failure in agentic systems blame the model. The published record points somewhere more specific: a large share of those failures are knowledge failures. The content is stale, the representation is too coarse to carry the distinction that matters, the extraction misses the places the knowledge actually lives, the updates lag behind the change, and nothing deprecates what is no longer true.
That is a claim about where failures concentrate, not a claim that the model is good enough. The evidence for it is set out below, and its limits are stated alongside it.
- Liu et al., Lost in the Middle: How Language Models Use Long Contexts (TACL 2023) — accuracy falls when the needed passage sits in the middle of a long context, “even for explicitly long-context models”. A larger window does not solve delivery.
- Xu et al., Knowledge Conflicts for LLMs: A Survey — models carry stale and mutually conflicting knowledge, and resolve the conflict inconsistently when context contradicts it.
- CRISS (25 Sep 2026) — a retrieval-augmented assistant for cancer registrars reached mean grounding scores of 0.62 / 0.56 / 0.59 across easy, medium and hard questions, against 0.29 / 0.26 / 0.29 for the same models without retrieval.
- IndicBankBench (25 Sep 2026) — across eleven models on 799 Indian retail-banking cases, strict reliability was 43.7–58.2%; among the named failure modes is reliance on stale context.
Knowledge Sidekick helps you treat knowledge as a first-class layer: how you create it, how you represent it, how you extract it from the places it actually lives, how you keep it current, how you deprecate what is no longer true, and how you evaluate whether your knowledge maturity is good enough for the systems you are building.
This site is where those ideas are developed openly: research positions, standards and conventions, adoption playbooks, internships, training, and free resources.
The measured numbers behind the argument
Four results from the published record, cited above and at the foot of this page. They are benchmark measurements, not promises: large enough to show the effect is real, not evidence of its size in your organisation. The limits of this evidence are stated in full under "What this page does not claim", further down this page.
Cancer-registrar assistant with retrieval versus the same models without it, on easy questions (0.56/0.26 medium, 0.59/0.29 hard). Grounding roughly doubled when the knowledge could be pointed to. CRISS
Strict reliability across eleven models on 799 Indian retail-banking cases. The named failure modes include reliance on stale context. The best model still got most hard cases wrong. IndicBankBench
Of model spend recovered by routing work against a taxonomy of agent requests, in an emulated 10,000-seat organisation: $3.3M to $5.0M a year at list prices of 21 September 2026. Control the Harness
Broadly retrieved agent memory scored below the agent with no memory at all; scoped retrieval reached 0.816. Bad knowledge can cost more than no knowledge. Scope Before You Persist
The shape of this site
Every page here is a file, and every line between two pages is a link a reader can follow. This figure is drawn from those links, so it shows the site as it is rather than as it is described.
Every page on this site, and every internal link between pages, is on one page: the site map.
- SStandards9 links
- RResearch8 links
- AAgents7 links
- DAdoption6 links
- AA2A5 links
- AMCP5 links
- OWhy Knowledge4 links
- OContext Engineering3 links
- KTaxonomies3 links
- KOntologies2 links
The same ten hubs the figure shows, ranked by links received inside it. The figure draws this data; the list states it. Letters: K Knowledge · R Research · S Standards · A Agents · D Adoption · O Other.
The open work, three pillars
The three pillars are also three different kinds of evidence: research is what is possible, standards are what an industry has agreed, and adoption is what is actually happening. Each can be checked, and each is checked here.
Where the ideas are developed
Short, self-contained position pieces on why knowledge matters for Agentic AI, how it behaves in enterprise and personal contexts, and what it means to engineer context as a knowledge-delivery problem. This is the research arm, written to be read by a technical leader.
- Lewis et al., Retrieval-Augmented Generation — the pattern this site argues past, and the baseline it starts from.
- Asai et al., Self-RAG — retrieval with self-critique; grounding as architecture rather than a prompt instruction.
- Ji et al., Survey of Hallucination in Natural Language Generation — the grounding problem, surveyed.
Conventions anyone can adopt
Open conventions for knowledge representation, formats, lifecycle, grounding, and the agent interface — including things like llms.txt and a clean, documented surface that agent engines can rely on. The goal is a small, usable set of standards, not a catalogue.
- Specifications with a defined status and an open process: RDF 1.1, RDFS, OWL 2, SKOS, PROV-O, JSON-LD 1.1, SPARQL 1.1, Turtle (all W3C Recommendations), and RFC 7089 (Memento).
- ISO 25964 — thesauri and interoperability with other vocabularies.
- Stated plainly: llms.txt is a proposal, not a standard. It has no standards body behind it, and this site does not present it as one.
Practice, not theory
How teams actually put knowledge-first practice into place: maturity evaluation, lifecycle adoption, knowledge audits and benchmarks, advisory, and training. Adoption is where the ideas meet the messy middle.
- Control the Harness, Control the Cost (25 Sep 2026) — enterprises are buying agent harnesses at ten-thousand-seat scale; in an emulated 10,000-seat organisation the paper’s router recovers 14–21% of model spend, $3.3M–$5.0M a year at list prices of 21 September 2026.
- Scope Before You Persist (25 Sep 2026) — memory that is retrieved too broadly scored 0.713 against 0.816 for scoped retrieval, and fell below the no-memory agent at 0.775. Bad knowledge can cost more than no knowledge.
- Companies ahead — what named organisations have actually put in place.
What this page does not claim
Three limits, stated because the rest of this page asks you to trust its sources:
- The benchmark results above are benchmarks. They are measured on constructed tasks and emulations, not on field studies of production systems. They establish that the effect exists and is large enough to measure; they do not establish its size in your organisation.
- Adoption evidence is largely self-reported. Where a vendor or an employer publishes a number, that number is evidence of what they chose to publish. It is not independent measurement.
- A retrieval advantage is not a business case. That grounding improves on a benchmark does not show that a knowledge programme will pay for itself at a given company. The cost side has to be argued separately, and is.
Where the evidence is thin, this site says so rather than hedging the sentence into something unfalsifiable. When a claim here cannot be traced to a retrievable source, it does not belong on the page.
Who this is for
This site is written for two moments in an organization's knowledge journey:
- Earlier, proactive. You are planning an agentic system, an LLM-based product, or a personal assistant, and you want the knowledge layer designed before it becomes the thing that fails in production.
- Later, reactive. The system is live, and you are now discovering that the hard part is not the model but the knowledge — stale content, weak formats, missing updates, no deprecation, poor grounding.
Readers tend to be AI-native business CIOs and founders who are funding or directing this work, often with an engineering leader or architect executing it. The site is written so both can read it: business-case enough for the person signing off, technical enough for the person building.
Where Knowledge Sidekick shows up
Sidekick to your primary business
Helping companies evaluate knowledge maturity and improve the full lifecycle — creation, representation, extraction, updates, deprecation — so agentic systems and LLM products are built on dependable knowledge rather than whatever is easy to scrape.
Sidekick for LLMs
Two related but separate goals: token savings — spending less on moving and re-reading the wrong knowledge — and grounding — tying what the system says or does to knowledge it can point to, so it is more dependable and more auditable. The cost claim is measurable rather than rhetorical: one 2026 study recovers 14–21% of model spend by routing work against a taxonomy of the requests an agent receives.
Sidekick for personal assistants
The knowledge layer for assistants that see email, calendar, messages, and habits: memory across short, medium, and long term, plus privacy, control, and compliance under regimes such as the DPDP Act in India (MeitY) and the GDPR in the EU (Regulation 2016/679).
A fourth sidekick — AEO / Agent Engine Optimization — sits across these: making the knowledge layer discoverable and usable by agent engines through llms.txt (a proposal, not a standard) and a clean agent interface.
About this site
This site is intentionally small and flat. The core argument — that knowledge is a cross-cutting layer in agentic systems, not a single module — does not need a large site to make its case. The pages here are written as short, self-contained position pieces. Each one opens with the direct answer to why the topic matters, then goes into enough substance to be useful on its own.
The topics are meant to be read by a technical leader who is funding or building agentic work and wants a clearer vocabulary for the knowledge layer.
Everything here is free to read: not-for-profit, no-fee and no-ads, and that is intended to hold permanently. The pages are written to be used directly — read them, quote them, adopt the conventions, and run the practice in your own team.
Where to go next from here
Three routes out of this page, depending on what brought you.
-
Why knowledge is the layer
Read this if you want the case made end to end, with every source attached: the evidence, what it establishes, and what it does not.
-
Context engineering
Read this if you are deciding what goes into a context window and in what order. It treats the window as a delivery problem, not storage.
-
What companies have put in place
Read this if you need to show a colleague that this is practice, not theory: named organisations, what they did, and where each claim comes from.
Sources cited on this page
- Liu, N. F., et al. Lost in the Middle: How Language Models Use Long Contexts. TACL, 2023. arXiv:2307.03172
- Lewis, P., et al. Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks. 2020. arXiv:2005.11401
- Meng, K., et al. Locating and Editing Factual Associations in GPT. 2022. arXiv:2202.05262
- Ji, Z., et al. Survey of Hallucination in Natural Language Generation. ACM Computing Surveys. arXiv:2202.03629
- Asai, A., et al. Self-RAG: Learning to Retrieve, Generate and Critique through Self-Reflection. 2023. arXiv:2310.11511
- Xu, R., et al. Knowledge Conflicts for LLMs: A Survey. 2024. arXiv:2403.08319
- CRISS: A Retrieval-Augmented AI Chatbot for Assisting Cancer Registrars. 25 Sep 2026. arXiv:2609.29075
- Scope Before You Persist: Preventing Cross-Family Interference in Agent Memory. 25 Sep 2026. arXiv:2609.29144
- IndicBankBench: Evaluating Safety and Reliability of Language Model Assistants in Indian Retail Banking. 25 Sep 2026. arXiv:2609.29167
- Control the Harness, Control the Cost: Routing and Governing AI Coding Agents in the Enterprise. 25 Sep 2026. arXiv:2609.28919
- Specifications: RDF 1.1, RDFS, OWL 2, SKOS, PROV-O, JSON-LD 1.1, SPARQL 1.1, Turtle (W3C Recommendations); RFC 7089 (Memento); ISO 25964. Linked individually from Standards.
- llms.txt — a proposal, linked from Standards → llms.txt.
Page updated .