Taxonomies and controlled vocabulary

Why free-text retrieval fails

Free text fails retrieval for two reasons that have nothing to do with the quality of the search engine. The same thing is written many ways — abbreviations, plurals, product names, internal jargon, misspellings — and the same string means different things in different places. A system that matches strings has to bridge both gaps by guesswork, which is what an agent acting on the result cannot afford.

A controlled vocabulary fixes the vocabulary end of that problem rather than the search end. Each concept gets one preferred label and one identifier; everything else becomes an alternate label pointing at the same concept. The identifier is the part that lasts: labels get corrected, translated and retired while it stays put, so links, queries and annotations keep pointing at the same concept as the words change above it. That separation is also the economic argument for doing this first — a taxonomy needs no logic, no reasoner and no query engine, only agreement on names and a place to keep them.

Trees, polyhierarchy, and the relations a thesaurus records

A strict tree gives every concept exactly one broader concept. It is easy to reason about, but it forces false choices onto real content: a medical device that is both a clinical item and a regulated product has to be filed under one of them, and either choice is wrong for half the queries. Vocabularies that survive contact with use are usually polyhierarchies, in which a concept may sit under more than one broader concept.

Thesaurus practice names the relations instead of leaving them implied. Broader and narrower carry the hierarchy in both directions; preferred and alternate labels separate the term to use from the equivalents that should lead to it; related links concepts that belong near each other without either containing the other. The kind of hierarchy is recorded too — generic, partitive or instance — because the three behave differently when queried. Treating them as one kind of edge is a common way to build a vocabulary that looks tidy and retrieves badly.

The hard part is not the shape of the tree. It is holding identifiers steady while labels move, and keeping the alias set complete enough that a query written by someone who does not know the vocabulary still lands on the right concept.

SKOS on the web, ISO 25964 in library practice

SKOS, the Simple Knowledge Organization System, is the W3C vocabulary for expressing these schemes on the web. It gives named terms for the pieces a taxonomy already has — a concept, a concept scheme, preferred and alternate labels, a hidden label for system-only terms, broader, narrower and related relations — alongside documentation properties such as definitions and scope notes, and mapping properties for linking vocabularies. Each concept is identified by a resolvable IRI, which is what lets a published vocabulary be linked from outside and reused by systems that did not build it. See SKOS for the standard itself.

ISO 25964 is the library and information science standard for thesauri and their interoperability. ISO 25964-1:2011 covers thesauri for information retrieval; ISO 25964-2:2013 covers interoperability with other vocabularies. Both are published by ISO and are not freely available online, so this page cites them by number and year rather than linking them. The two bodies of work are complementary: ISO 25964 carries the construction and interoperability guidance, SKOS the web expression that most modern tools read.

How a taxonomy differs from an ontology

A taxonomy organises labels into a hierarchy and carries no formal meaning. Nothing in it tells a machine that two concepts are disjoint, or that a property holds a single value, or that one fact follows from another. An ontology adds that layer, expressing classes, properties and individuals in a logic a reasoner can compute over.

The choice is not a matter of maturity. A taxonomy is enough when the job is finding content; an ontology earns its cost when the job includes checking, deducing or merging across sources. In practice the two stack, and once the vocabulary is populated with instances and queried by traversal rather than lookup, what you have is a knowledge graph.