Essential Graphonotolgy screenshot

Hallucination and why AI needs better Enterprise Context

Hallucination – AI isn’t the problem, it Needs Better Enterprise Context.

We see more and more tool vendors claim to be ‘AI-ready’, ‘AI-everywhere’, ‘AI-aware’, leveraging the latest models to interrogate their data, but with warnings, such as:

  • “Please check the answers as they may be inaccurate” – So does that mean that you have do the the work anyway to double check the response?
  • “We cannot guarantee the validity of any answers” – not so useful when making a key decision
  • “Results are generated by AI and may not reflect real-world constraints”

It is the fear of hallucination that drives the vendors to caveat the answers their tools give when attached to AI, a bit like having a speedometer in your car that may or may not be correct.   The problem isn’t that AI lacks intelligence, it is that it often lacks the enterprise context that it can use to work on the information it has.

If you want to ask:

  • Which business-critical processes depend on end-of-life technology?
  • What applications support this strategic capability?
  • Where are the single points of failure in my IT landscape?
  • Which projects are addressing our highest risks?

The completeness and correctness of the answer you get back depends entirely on what the AI knows about your enterprise, from two perspectives. Not only the data itself, but also on the semantic structure that underpins the data and how it is related – anyone who understands ontologies and the importance of semantics knows that without a coherent structure, AI understanding will fail.  If you connect the knee bone to the elbow rather than the thigh bone in your model (‘It’s quicker’, ‘We don’t need that data’, ‘We don’t care about that relationship’) then of course AI is going to hallucinate, because it doesn’t understand what it is being given – none of the patterns that it was trained in had that relationship as a valid mapping.  It will give you answers that may look convincing, but underneath it may lie assumptions, inferred dependencies or invented connections that don’t actually exist.

Different Hallucinations

There are actually different types of hallucination:

Structural Hallucinations

When an AI looks at the data and invents its own structures, such as a dependency that exists, skipping relationships or traversing invalid paths in the model.  The problem is that they can appear to be factual but are actually based on evidence that doesn’t exist.

Interpretive Hallucinations

Here, even if the underlying data is complete and correct, if the reasoning applied by the AI during its analysis is incorrect, then it will reach the wrong conclusions.

Minimising Hallucination

It’s interesting seeing some vendors suggest that they are moving to solve the issue through various forms of graph DB technology.

Let’s be clear, we have a graph underpinning the Essential ontology, we always did, and it is great for maintaining information and relationships between things.  But the technology alone does not solve the hallucination problem (it’s a technology).  It doesn’t know which relationships are valid, it doesn’t know architectural relationships, it doesn’t prevent you connecting the knee bone to the elbow nor can it stop AI drawing conclusions based on assumptions.

Importantly, hallucinations are rarely caused by a lack of technology, they’re usually caused by a lack of governed enterprise meaning.

It’s the Model that Matters

It is the quality of the enterprise model itself that reduces hallucination.  A governed ontology of the organisation clearly defines:

  • What concepts exist.
  • How they relate to each other.
  • Which relationships are authoritative.
  • What evidence is required before conclusions can be drawn.

Without this layer of governance, AI is free to make assumptions when it doesn’t know something.  The graph is providing the connections between things, the ontology model provides meaning and the governance framework around that ontology provides trust.  It is this combination that gives you more reliable AI assisted outputs.

The Essential Approach

Our Essential MCP has been designed around a simple principle that the AI should be prevented from making things up.  We have focused on removing structural hallucination and minimising interpretive hallucinations (it’s almost impossible to remove these latter hallucinations completely).   We govern the interface to the data and apply codified rules for the AI to follow. In other words, rather than giving AI unrestricted access to data and hoping it understands the architecture correctly, Essential MCP provides a governed layer that sits over the enterprise semantic model.  The AI has access to the self-describing Essential meta model, which holds the information that the AI needs to traverse the ontology, remembering that it reflects a real-world view of how an organisation is structured (not an EA specific slice), and we provide resources to help.  The MCP also applies rigour so that relationships are validated, preferred paths are governed and evidence is required before answers are given, along with some EAS magic sauce. The result is that the AI cannot simply make up how the enterprise is connected, if it fails any governance steps then your AI will either tell you it cannot return anything, or will tell you what the gap is.

Trustworthy but not Infallible

Essential’s MCP does not make AI infallible, but importantly, it makes the underlying enterprise context trustworthy.  This means the data being presented to the AI is correct (pretty much zero hallucination here), so we aren’t questioning if it’s brought back valid data, we’re focused on whether we agree with the interpretation of the evidence that the AI found. In other words, do we agree with the conclusions reached by the AI.

This is key as traditional approaches leave risk in both the data layer and the reasoning layer.  We can see other tools struggling with this – structural hallucinations leading to invented evidence which needs checking and then interpretive hallucination that is based on imaginary data.  The trick is to remove structural hallucination risk and minimise interpretive hallucination risk.

Evolving AI In Enterprise Architecture

Conversations are focusing on the capabilities of the AI models evolving (“they will be able to handle it in the end”) and the underlying infrastructure; the graph DB discussion being an example.  Both need to be considered but the reality is that unless the model of the enterprise is semantically accurate and the structural hallucinations removed, the capabilities of the tooling and its infrastructure won’t make much difference.

We want proper decisions based on trustworthy data, and it is a governed enterprise model that gives AI the context it needs to reason about the enterprise with confidence.  And for those that get it, they know that the knee bone really does need to be connected to the thigh bone.

 

Contact Us