Auditable knowledge graphs from documents

Pointing a large language model at a PDF and asking for triples takes an afternoon, and the first result is usually impressive. The trouble starts with the second document, and the tenth, and the moment someone asks where a particular statement in the graph came from. This paper is about what has to be true of a document-to-graph pipeline before its output can be reviewed rather than believed, and how OntoCast is built to meet it.

Easy to demo, hard to trust

The obvious design is the one most prototypes use: hand the model a passage and an ontology, ask for a graph back, and store whatever returns. Each step of it quietly discards something a reviewer needs.

• The vocabulary drifts. The model is shown your classes and is free to improve on them. It invents a near-synonym of a property you already have, mints an instance inside your ontology's namespace, or writes a fresh owl:Ontology header with an IRI of its choosing. None of this is an error a parser will catch.
• Change is invisible. If each call regenerates the graph, the only record of what a run did is the difference between two large files, and that difference mixes what the model meant to change with what it simply forgot to repeat.
• Facts lose their origin. Once the triples from many passages are pooled, a statement no longer says which sentence supports it, so it can be neither checked nor contested.
• Validation turns into filtering. When a check fails, the tempting fix is to drop whatever failed. The graph that remains looks clean and says nothing about what was lost.

A graph with these properties still answers SPARQL queries. That is precisely the danger: nothing about it looks broken, and nothing about it can be audited. The requirement is not a better model. It is a pipeline in which every step leaves something a person can inspect.

Your vocabulary, not the model’s

The first line OntoCast draws is between schema and data. Your ontology is the schema, and facts extracted from a document use its classes, its properties and the individuals it already declares — the entries of a controlled vocabulary, say — without adding to them. Each part of a document is shown the slice of your catalogue it needs: chosen from your ontologies, retrieved by search over their terms, or one ontology you fix for the run. A facts-only run changes no ontology at all.

An ontology is identified by its IRI, the subject of its owl:Ontology statement. The short handle and the Turtle prefix its author chose are aliases of that IRI, and the two may differ; an ontology you upload keeps the IRI you gave it. Identity is never left to the model. When a run may create vocabulary, the model proposes a name and a description for where the document's new terms belong, and code decides: a name already in the catalogue extends that ontology, any other becomes a new IRI minted under your domain. A header the model writes anyway is dropped and counted, not trusted.

No ontology yet? It drafts one with you. Run a document with nothing in the catalogue and OntoCast proposes the vocabulary as it reads — classes and properties with labels, under an IRI it mints for you — and then extracts the facts in those terms. The draft is an ordinary OWL ontology: you read it, compare its versions, download it, and, once it says what you mean, fix later runs to it so they extract against it rather than extend it.

Changes as patches, not regenerated graphs

The model never rewrites an ontology. It returns a patch: the triples to insert and the triples to delete, which OntoCast checks and then applies, deletions first. Fixes from the review pass that follows each extraction are applied the same way, as a patch to that part's graph.

Two consequences matter for an audit. A removal is always an explicit delete, so the model cannot drop a triple by forgetting to repeat it; and the patch is the record of what the model meant to change, small enough to read. Patches from the parts of one document are merged into one change per ontology, where an insertion from one part wins over another's deletion of the same triple, so parallel work cannot quietly erase itself.

A run that changes nothing writes nothing: the same content is never stored as a new version, so a new version is itself the signal that something changed, and comparing it with its prior shows what.

Every fact traceable to its source

Facts and schema live in different namespaces, and the rule between them is what keeps your ontology clean. Every entity found in the text is named in a separate facts namespace, even when its class comes from your ontology: two mentions of a supplier are new facts-namespace entities typed with your Supplier class, not the class itself. An IRI of your ontology is reused only for an individual your ontology declares, and a term in its namespace that it does not declare is flagged as unknown at review rather than accepted as vocabulary.

Each asserted triple then carries its origin in standard terms. OntoCast uses RDF 1.2 reification: a reifier points from the statement to the part of the document that stated it, or to several parts when more than one did. The parts themselves are described once — their position in the document, a hash of their text, when they were extracted, and the section they belong to.

@prefix ex:   <https://example.org/supply#> .
@prefix cd:   <https://growgraph.dev/facts/> .
@prefix rdf:  <http://www.w3.org/1999/02/22-rdf-syntax-ns#> .
@prefix rdfs: <http://www.w3.org/2000/01/rdf-schema#> .
@prefix prov: <http://www.w3.org/ns/prov#> .

cd:acme_gmbh a ex:Supplier ;
    rdfs:label "ACME GmbH" ;
    ex:supplies cd:valve_housing .

_:r rdf:reifies <<( cd:acme_gmbh ex:supplies cd:valve_housing )>> ;
    prov:wasDerivedFrom <unit_iri> .

Provenance is kept apart from the clean graph. Where ontology changes came from is recorded beside the ontology rather than inside it, so the ontology you load elsewhere carries none of it, and facts can be taken with their provenance or without it. A graph delivered without provenance has no link back to the text, which is why the service keeps it unless asked not to. In the console the same link is one click away: the graph explorer on an extraction's page shows, for any entity, its values, its links and the sections of the document it came from.

The step most likely to corrupt a graph silently is deciding that two mentions are the same entity. A shared name proves nothing here: guards check what two candidates actually state — conflicting values, a single-valued property pointing at two things — and any one of them can refuse the merge. Refusals are counted and can be named with their reasons, and a merged graph that shows the signature of a wrong merge is split and merged again. The entity disambiguation guide has the detail.

Checked against your shapes

Shapes are where a team writes down what a correct graph looks like, and OntoCast uses yours twice. While extracting, the model is shown the constraints as a conformance chapter it can read, each line taken from the constraint's sh:message or generated from the constraint when it has none. Once the parts of a document are merged, the facts are validated against the shapes together with the ontology they were extracted against, so that a sh:class constraint pointing at one of your terms is judged with the vocabulary that defines it.

What follows the check is where auditability is won or lost. A small set of repairs needs no model and is done in code: a literal is retyped to the declared datatype when it parses as that type; a string is replaced by a catalogue IRI when exactly one term carries that label or code; a node that states nothing beyond its type and label is removed. A repair pass is kept only if it lowers the number of violations, and every change is listed with its reason. A node that carries a real value but lacks a required property is reported, never filled in and never deleted.

Validation never withholds output. A run through the library returns its facts with a conformance report: whether they conform, the violations still open, grouped by constraint and by shape, and the repairs applied. When no shape was evaluated, or none matched anything, the report says nothing was checked rather than reporting a pass. That is the difference between “checked against your shapes” and a graph that merely looks valid: the residual violations stay where you can see and triage them. In the console, shapes are uploaded beside your ontologies and kept per project. The validation guide describes each check.

Ontologies with a history

An ontology is never edited in place. Each version is an immutable named graph — for a version a run writes, its IRI ends in a hash of its content — and inside it the ontology carries a header in the vocabulary an RDF practitioner already reads:

<https://example.org/supply> a owl:Ontology ;
    owl:versionIRI   <https://example.org/supply#9f2c41> ;
    owl:priorVersion <https://example.org/supply#a71e08> ;
    owl:versionInfo  "1.1.0" ;
    prov:wasDerivedFrom <document_iri> .

A first version has no prior; every later one names the version it was made from, and one written by a run names the document it was derived from. The label moves by the size of the change, deletions weighing more than additions. Removing a version keeps its header and marks it as removed, so a lineage that passes through it never points at nothing.

The version that no other version descends from is the head, and a run reads one head per ontology while every older version stays in the store. When two runs extend the same version at once, the history has two heads and is listed as branched. A branched ontology stays readable, but nothing is extracted against it or extends it until it has a single head again, so no run silently picks a side.

In the console this history is something you look at rather than query for. Each ontology shows its newest version and how many there are, with a mark when its history has branched; its page draws the version history, shows any version as a graph or as Turtle, and compares any two. An extraction's page links the versions that run created, extended or only read. The ontologies and facts page of the documentation states the contract in full.

Standard RDF out

Your ontology in, standard RDF out. Nothing OntoCast produces needs OntoCast to remain readable. Ontologies are OWL, facts are RDF in your terms, provenance is RDF 1.2 reification over PROV, shapes are plain SHACL, and the library returns its graphs as Turtle and writes them to a triple store — in memory by default, or Apache Jena Fuseki, partitioned by tenant and project.

The console downloads each run as Turtle, N-Triples, N-Quads, TriG or JSON-LD, with provenance either as RDF 1.2 or as classic RDF 1.1 reification that any tool reads. A whole project exports as one TriG file or as a zip bundle, and SPARQL in the console queries a project read-only. Taking the result into your own store is a download, not a migration.

The engine is open source. The OntoCast library is Apache-2.0 on GitHub and PyPI, and runs as an HTTP service, as a batch command or inside your own agent framework, with your own model provider. The console is the hosted way to run the same engine without operating a language-model pipeline and a triple store yourself; it is commercial, usage-based, and open in early access.

What to ask of any document-to-graph tool

Questions worth putting to any candidate, including ours:

• Are facts extracted into my ontology, or into a vocabulary the model improvises? Can I run it so that my ontology is not changed at all?
• Who mints ontology and instance identifiers — code under rules I can read, or the model?
• Are new instances kept out of my ontology's namespace?
• When a run changes the ontology, do I get an explicit patch and a new version beside the old one, or a regenerated file to diff?
• Can every fact be traced to the passage that stated it, in standard RDF, and can I keep or strip that provenance?
• When two mentions were merged into one entity, can I find out why — and why two others were not?
• Are my SHACL shapes checked, and does the result list every repair and every remaining violation? When nothing was checked, does it say so, or report a pass?
• Is failing output dropped quietly, or returned with the findings attached?
• Can I take the whole result into my own triple store in a standard serialisation, and read the engine that produced it?

The engine described here is OntoCast, open source under Apache-2.0 and documented in full in the OntoCast documentation. The hosted console that runs it is described on the platform page, and the terms used above — ontology, SHACL, provenance — are defined in the glossary.