AI Services Case Study — Configure-Mode Project Management for an AI Digital Assistant
- Jan 20
- 7 min read
1. Organisational Problem
Orr Consulting identified an internal strategic opportunity to improve how website visitors accessed its AI transformation services, case studies and AI Insights Library.
Applying its AI Use Case Discovery approach, the organisation initiated a Configure-mode project to select, configure and deploy an AI Digital Assistant.
The project did not involve building a new AI model or developing bespoke software. It involved selecting and configuring an existing Conversational AI product around the organisation’s purpose, content and website environment.
Several potential products were identified. An application integrated with the existing website platform was selected as the initial option because it offered the lowest-friction route to implementation.
However, ease of installation did not establish that the product would be sufficiently capable.
The assistant still needed to demonstrate that it could:
use the organisation’s knowledge content effectively
answer and route questions appropriately
remain within defined operating boundaries
support a useful visitor experience
operate credibly in a live environment
The final behaviour of the assistant could not be fully specified in advance. It would emerge through configuration, testing, observed performance and controlled refinement.
This created a central project-management challenge:
How could the assistant be selected, configured and tested progressively without allowing uncertain product capability or evolving requirements to weaken project control?
In the Orr Consulting AI Transformation Process, this case study demonstrates the Deliver-stage role of AI Project Management in controlling a Configure-mode AI project while progressively reducing uncertainty about product suitability, performance, governance and readiness for live use.
2. Situation
The organisation had developed a substantial website and AI Insights Library containing highly structured information on:
AI capabilities
AI business problems
the AI Transformation Process
AI services
capability and service case studies
practical AI Insights
Conversational AI created an opportunity to turn these knowledge assets into a more interactive and accessible experience.
This was a Configure-mode AI project.
The underlying technology already existed, but the selected product had to be configured around the organisation’s:
knowledge content
operating rules
user journeys
service boundaries
website environment
The principal uncertainties related to whether the configured assistant could:
access and use the available knowledge reliably
provide relevant responses
route visitors towards suitable content
remain within its intended role
handle unclear or out-of-scope requests appropriately
create sufficient value to justify live deployment
Product selection was therefore treated as a testable project decision rather than an irreversible commitment.
If the initial product could not meet the required performance, governance or usability standards, alternative options could be considered rather than continuing to invest in an unsuitable solution.
The project required disciplined AI Project Management that allowed configuration to evolve while maintaining clear boundaries, governance and deployment control.
3. Background
Traditional configuration projects can often follow a relatively predictable sequence:
define requirements
configure the product
test the configuration
correct defects
deploy the solution
The AI Digital Assistant differed because its behaviour was not entirely deterministic.
Operating instructions, knowledge sources and expected behaviours could be defined, but it was not possible to predict every question a visitor might ask or every response the assistant might produce.
Several important questions could only be answered through testing:
Was the product sufficiently capable?
Could it use the knowledge content reliably?
Would it guide visitors towards relevant information?
Could it remain within its intended role?
How would it respond to unclear or out-of-scope requests?
Were the website content and metadata sufficient?
Was the configured capability fit for live use?
Treating configuration as a one-off technical activity would therefore have created false confidence.
The project required a disciplined but progressive approach in which configuration generated evidence, testing reduced uncertainty and deployment remained a controlled decision rather than an assumed outcome.
4. Action Taken
A Progressive Project Management approach was applied, combining clear project boundaries, governance and accountability with short, evidence-led delivery cycles designed to reduce uncertainty progressively.
The work followed Orr Consulting’s structured AI Project Management approach.
4.1 Project Boundaries
The assistant’s role was defined as helping website visitors:
understand the organisation’s AI transformation proposition
access relevant services, case studies and Insights
navigate the website and AI Insights Library
identify appropriate next steps
It was not intended to provide confidential advice, make binding recommendations, commit pricing or operate as an unrestricted general-purpose chatbot.
The project also established the available knowledge sources, expected users, operating environment and areas requiring human accountability.
This separated uncertainty about the assistant’s eventual performance from uncertainty about the purpose, scope and governance of the project itself.
4.2 Managing Uncertainty
Material project uncertainties were made explicit rather than being hidden within configuration activity.
These included uncertainty about:
product suitability and capability
knowledge quality and coverage
response accuracy and relevance
content routing
operating-boundary adherence
privacy and transparency
user value
readiness for live deployment
Depending on their nature, these uncertainties were linked to configuration deliverables, dependencies, assumptions, risks or issues.
A single uncertainty could require several complementary responses.
For example, uncertainty about answer quality could require revised operating instructions, stronger website content, additional test scenarios and a decision about whether any remaining limitations were acceptable.
The objective was not to create a separate control system for AI. It was to use established project-management controls to reduce uncertainty progressively and support informed delivery decisions.
4.3 Progressive Configuration
Configuration was undertaken through short cycles:
configure the assistant
test its behaviour
review the evidence
identify weaknesses
refine the configuration or supporting content
retest the solution
Early cycles focused on the assistant’s role, knowledge access and operating boundaries.
Later cycles focused on improving:
response relevance
content routing
handling of unclear requests
responses outside the intended scope
readiness for live use
Testing concentrated on two principal questions:
Did the assistant remain within its intended role?
Could it explain and route users towards relevant content reliably?
A small number of areas requiring clearer boundaries or improved routing were identified. These were addressed through targeted refinement and retesting.
The cycles also allowed the initial product-selection decision to be revisited. If the product had failed to meet the required standards, an alternative option could have been assessed.
4.4 Managing Content Dependencies
Testing demonstrated that the assistant’s performance depended heavily on the quality and structure of the information available to it.
The AI Insights Library provided a strong knowledge foundation, but testing also identified opportunities to improve:
website wording
page relationships
metadata
content structure
expression of service boundaries
These were managed as project dependencies rather than attributed solely to the AI product.
This helped distinguish between limitations caused by the technology and those caused by incomplete content, unclear source material or insufficient configuration.
4.5 Governance and Deployment
Governance was proportionate to the scale of the project but remained central to delivery.
The project considered:
acceptable use
privacy and transparency
operating boundaries
responsibility for final service advice
testing evidence
ongoing monitoring
Live deployment was not assumed simply because the product had been installed.
Progression depended on whether testing demonstrated that the assistant was sufficiently useful, remained within its intended role and could be monitored and refined after release.
Deployment therefore represented an evidence-based governance decision rather than technical completion alone.
4.6 Tracking Benefits
The project assessed whether the assistant created meaningful value rather than merely adding another website feature.
Following deployment, several practical benefits were observed:
easier access to website information
improved navigation of the AI Insights Library
stronger utilisation of existing content assets
a more natural route into AI services
practical learning about Conversational AI implementation
These outcomes provided evidence that the configured capability delivered value and created a basis for monitoring continued usage, performance and benefits through the structured AI Benefits Realisation approach.
5. Outcomes
The Configure-mode project created several notable outcomes, observations and lessons learned.
5.1 Product Suitability Confirmed
The integrated application was initially selected because it offered the lowest-friction route to implementation.
Configuration and testing demonstrated that the product was sufficiently capable for the intended purpose, avoiding the need to move to a more complex alternative platform.
5.2 Controlled Configuration
The assistant was configured and refined through short, evidence-led cycles without allowing changing requirements or uncertain behaviour to weaken delivery control.
Clear boundaries, testing priorities and deployment criteria enabled the project to adapt while remaining governed and purposeful.
5.3 Live Capability Implemented
A live AI Digital Assistant was implemented rapidly and placed into operational use on the website.
This demonstrated that a practical Conversational AI capability could be delivered through controlled configuration rather than a bespoke AI build.
5.4 Information Access Improved
The assistant created a conversational route into the AI Insights Library, services, case studies and contact options.
Visitors could ask questions in natural language and be guided towards relevant content or next steps, reducing the effort required to navigate a substantial knowledge resource.
5.5 Knowledge Assets Enhanced
The implementation made the existing website and AI Insights Library more interactive and accessible.
Testing also exposed opportunities to strengthen content, metadata and page relationships, improving the wider knowledge environment supporting the assistant.
5.6 Configure-Mode Learning
The project reinforced an important lesson: selecting an existing AI product does not remove project uncertainty.
The uncertainty shifts towards product suitability, configuration quality, knowledge readiness, operating behaviour, governance and fitness for operational use.
Progressive Project Management provided a structured way to reduce those uncertainties through product testing, controlled configuration, evidence-led refinement and a clear deployment decision.
6. Final Thoughts
Configure-mode AI projects can appear simpler than they are.
Because the underlying product already exists, organisations may assume that delivery involves little more than selecting options, adding content and enabling the technology.
The real project-management challenge is establishing whether the configured capability is fit for its intended organisational purpose.
That requires clear boundaries, reliable knowledge content, proportionate governance and short cycles of configuration, testing and refinement.
In this case, Progressive Project Management allowed the AI Digital Assistant to evolve through evidence without losing delivery control. The initial product was treated as a testable choice, material uncertainties were translated into controlled project activity and deployment remained a governed decision.
The assistant was successfully implemented not because uncertainty had been removed before configuration began, but because the project created a disciplined process for reducing it progressively.
This AI Services Case Study is part of the Orr Consulting AI Insights Library — structured thinking for AI transformation leaders and decision makers.
If your organisation is configuring an existing AI product and would benefit from stronger delivery control, structured testing and clearer deployment decisions, we would be pleased to discuss your next AI steps.
Subscribe to Orr Consulting to receive occasional emails with practical AI Insights and updates.
