EAS’s Approach to AI
Like everyone else, at EAS we have been considering how we can apply AI in our Essential EA Management tool. To date we have taken a cautious approach, choosing to wait and see how things progress before we move too far. The title of the recent HBR article, AI Can (Mostly) Outperform Human CEOs really caught our attention, therefore. Have we been too cautious we wondered?
It’s an interesting read, but one which we think validates the considered approach that we have adopted. Basically, it is great at speeding up coding or automating tasks, but for strategic decision-making processes, it’s not quite there yet.
Our approach has been to introduce AI into the Essential tool to support data capture and maintenance, such as creating descriptions and business processes to support business capabilities, where an organisation does not have these defined, effectively helping the user by speeding up data capture, data mapping and filling gaps. Even in these fairly simple areas, our approach is to allow the user to check and amend before the data is added, rather than taking the AI response verbatim. We have also implemented AI providing some advice in our EA Assistant, based on the answers to a set of questions, but this is used as input to decision making, it is not directly actionable.
The HBR Experiment
The HBR report, describes an experiment where a group of CEOs and students were pitted against AI, in the form of GPT-4o, in a “gamified simulation designed to replicate the kinds of decision-making challenges CEOs face, with various metrics tracking the quality of their choices.”. The authors were surprised at how well GPT-4o performed, “outperforming the top human participants on nearly every metric”. However, as the experiment progressed, GPT-4o was fired by the board quicker than the humans, exposing the issues that still exist with implementing AI as an autonomous service.
It became apparent that the AI was unable to respond to unexpected and unpredictable events. It adopted a short-term view centred on maximising growth rather than a long-term strategic approach that provided the flexibility to cope with the unexpected.
The Result
The results outline how useful AI can be as a strategic partner, providing unique insight and helping to avoid costly mistakes, however, there are a number of considerations outlined that need to be taken into account. Key among these is the scope and quality of the data that the AI is exposed to. Many companies do not generate sufficient, high-quality data that is required to enable AI to support the boardroom. In these situations, the output can be erroneous and misleading – “hallucinations”. Within Essential, you could end up with the problem where a lack of data completeness in some areas could lead to advice which is sub optimal, so we need to consider carefully how we approach this.
Our Approach in More Detail
So what is our approach? Our frames-based knowledge graph is very powerful, our engineers have backgrounds in graph theory and intelligent systems and we designed the model from the start to be ready for advances in AI. The frames-based model allows you to represent the semantics of the real-world (there are no artificial concepts in our meta model) This approach allows AI systems to understand and reason about complex data more effectively and at a granular level, primarily as they have been trained based on the real world. And by using frames, each entity in the knowledge graph is not just a simple node but a rich, descriptive object that contains detailed information, making AI’s understanding more nuanced and accurate.
This opens up a wide variety of opportunities to use the data in Essential to get more informed feedback and potentially automate enterprise architecture decisions. But, as mentioned before, it needs to balance the amount and quality of the data to do that effectively. This is where we are just holding back for now. While the Essential engine has the necessary capabilities, we are intentionally pacing the introduction of new features as we wait for AI to mature further.
Summary
In summary, we think expanding the use to make data management easier is obvious, but for automated decision-making, we think our current, cautious approach is the right one.