AI Capability Case Study — Decision Support AI: When a Conventional Rules-Based Solution Proved Better Than AI
- Apr 24
- 6 min read
Updated: Jul 26
1. Organisational Problem
A large property and facilities organisation was experiencing growing challenges in prioritising maintenance requests across a geographically dispersed asset base.
Managers were required to balance multiple competing factors, including:
Safety risk
Service impact
Regulatory obligations
Asset condition
Cost
Resource availability
As demand increased, leaders questioned whether Decision Support AI could help improve prioritisation decisions, increase consistency and ensure that the most important work was addressed first.
Decision Support AI helps people make more informed decisions by analysing information, identifying patterns and presenting structured recommendations.
In the Orr Consulting AI Universe, Decision Support AI helps address the question:
How can we improve the quality of decisions?
2. Situation
The organisation already operated a structured maintenance process based around a legacy maintenance and repairs system supported by experienced operational managers.
The existing system included limited decision support capability through manually configured prioritisation rules.
As part of a wider technology refresh programme, the organisation was developing a business case to procure a replacement maintenance platform that included AI-enabled decision support functionality.
The proposed investment was expected to involve significant software procurement costs, implementation effort, organisational change activity and user training.
Leadership increasingly believed that AI-enabled decision support and platform replacement would be required to achieve the desired improvements.
However, the scale of the proposed investment created an important question.
Was the organisation facing a genuinely complex decision-making problem that required AI, or was a simpler and more proportionate solution available?
Proceeding without answering that question risked committing substantial resources to a solution that might not be necessary.
Before progressing, the organisation engaged Orr Consulting to review the business case and assess whether AI-enabled decision support represented the most appropriate solution to the underlying business problem.
The objective was not to validate the technology.
The objective was to determine whether AI was genuinely required and, if not, what the most appropriate solution would be.
The review was intended to establish an evidence-based view of the problem, the available solution options and the likelihood that the proposed investment would deliver sufficient value to justify its cost and complexity.
The engagement was delivered through an AI Business Case Review.
3. Background
Although Orr Consulting was engaged through an AI Business Case Review, the assessment considered the wider transformation context that had led to the proposed investment.
The initiative was assessed against the standards of the Orr Consulting AI Transformation Process, a structured strategic framework for selecting, designing and delivering AI opportunities through a Discover, Design and Deliver approach.
Rather than starting with assumptions about the technology, the assessment focused on understanding the organisation's current position, identifying gaps in readiness and determining what actions were required before implementation could proceed with confidence.
The assessment identified a potential mismatch between the proposed AI-enabled solution and the underlying business problem.
The findings are summarised below.
3.1 Discover
In the AI Transformation Process, the purpose of the Discover stage is to build understanding, assess readiness and identify realistic AI opportunities before committing to strategy or investment.
The Discover stage assessment identified an important challenge.
The organisation had largely focused on potential technology solutions before fully testing the nature of the underlying decision problem.
Several key questions had not been explored in detail, including:
Whether the decision complexity genuinely required AI
Whether simpler alternatives existed
Whether expected benefits justified investment
Whether organisational readiness supported adoption
These questions ultimately became central to the business case review.
This created a risk that the organisation would commit to an AI-enabled solution before establishing whether AI was genuinely required.
3.2 Design
In the AI Transformation Process, the purpose of the Design stage is to define direction, establish governance and control and justify investment before delivery begins.
The Design stage assessment identified a potential mismatch between the perceived complexity of the proposed solution and the actual complexity of the business problem.
As part of the AI Business Case Review, Orr Consulting assessed:
The nature of the prioritisation decisions being made
Existing system capabilities
Data availability and quality
Expected benefits
Delivery complexity
Governance implications
Alternative solution options
The review also considered how expected benefits would be defined, measured and tracked through a structured AI Benefits Realisation approach.
While the prioritisation process involved multiple factors, analysis showed that the underlying decision logic was already relatively well understood.
Many of the prioritisation decisions could be expressed through clear business rules based on established organisational policies and operational priorities.
The review therefore raised an important question:
Was AI genuinely required, or could the problem be solved more simply and cost-effectively?
This created a risk that investment decisions would be driven by perceived technology needs rather than the actual requirements of the business problem.
3.3 Deliver
In the AI Transformation Process, the purpose of the Deliver stage is to deliver AI initiatives in a controlled way and embed them into business-as-usual operations.
The Deliver stage assessment identified an opportunity to challenge assumptions before significant investment was committed.
Implementation had not yet commenced.
Procurement decisions had not been finalised and major organisational change activity had not yet begun.
This created a clear decision point: whether the initiative should proceed, be redesigned or stop altogether based on evidence rather than assumptions.
4. Action Taken
Orr Consulting reviewed the proposed business case and assessed the maintenance prioritisation process in detail.
The review applied principles from the Orr Consulting AI Business Case Development approach, focusing on whether AI was genuinely required to achieve the intended business outcomes.
The review found that many of the perceived shortcomings were not caused by an absence of AI capability.
Instead, they reflected opportunities to improve configuration, standardisation and adoption of existing functionality within the current system.
Analysis showed that most prioritisation decisions could be supported through transparent rules-based logic aligned to existing organisational policies and operational priorities.
As a result, the recommended approach was not to procure a new AI-enabled platform.
Instead, the organisation enhanced and expanded the existing rules-based prioritisation framework within its current system.
This included:
Refining prioritisation criteria
Improving scoring and weighting logic
Standardising configuration across teams
Improving visibility of decision rationale
Maintaining manager oversight and accountability
The solution delivered the required operational improvements while avoiding the cost, complexity and organisational disruption associated with a major system replacement.
5. Outcomes
The review created several notable outcomes, observations and lessons learned.
5.1 Improved Consistency
Prioritisation decisions became more consistent across teams because common criteria and scoring logic were applied, reducing variation in how comparable maintenance requests were assessed.
5.2 Greater Transparency
Managers could clearly understand how priorities were determined, explain decisions when challenged and retain confidence in the rationale behind each prioritisation outcome.
5.3 Reduced Delivery Complexity
The organisation avoided the additional complexity associated with AI model development, training, monitoring and assurance.
This demonstrated that solution complexity should remain proportionate to the complexity of the underlying decision problem.
5.4 Significant Cost Avoidance
The organisation avoided substantial software procurement, implementation and ongoing support costs associated with replacing the existing platform.
5.5 Reduced Change Effort
The recommended approach avoided the operational disruption, extensive retraining and adoption risks typically associated with major system replacement programmes.
5.6 Retained Human Judgement
Managers continued to exercise judgement where exceptions, local conditions or operational context required flexibility, while using transparent rules to support routine prioritisation decisions.
5.7 Evidence-Based Decisions
The review demonstrated that the organisational problem could be solved effectively through a simpler rules-based approach.
The successful outcome came not from implementing AI, but from correctly determining that AI was not required and directing investment towards the most proportionate solution.
6. Final Thoughts
For Decision Support AI, the best answer is not always an AI answer.
Decision Support AI can be highly effective where decision-making relies on complex patterns, uncertainty or large volumes of information. However, where decision logic is already clear, stable and explainable, a simpler rules-based approach may provide a more proportionate solution.
In this case, the organisation's maintenance prioritisation challenge was real. The review demonstrated that the required improvements could be achieved through clearer rules, improved configuration and better use of existing capability without the need for AI-enabled decision support.
More broadly, successful AI transformation depends on selecting the right solution to the problem rather than seeking opportunities to deploy AI.
The most successful outcome was not implementing AI. It was establishing, through structured assessment and business case review, that AI was not required to achieve the intended business outcomes.
Progression, redesign, pause or selecting a non-AI solution can all represent successful outcomes when decisions are based on evidence and aligned with organisational needs.
This AI Capability Case Study is part of the Orr Consulting AI Insights Library — structured thinking for AI transformation leaders and decision makers.
If your organisation is considering AI-enabled decision support or evaluating a proposed AI investment, we would be pleased to discuss your next AI steps.
Subscribe to Orr Consulting to receive occasional emails with practical AI Insights and updates.
