AI Services Case Study — Use-Mode Project Management for a Generative AI Pilot
- Jan 21
- 7 min read
Updated: 3 days ago
1. Organisational Problem
A large public sector organisation initiated a Generative AI pilot to explore the potential of AI tools to improve productivity, knowledge access and service delivery.
The pilot was supported at senior levels and aligned to broader digital transformation ambitions. However, unlike more traditional technology initiatives, the project began without:
Fully defined requirements
Stable or predictable delivery scope
Clear, pre-validated assumptions about outcomes
Established delivery patterns or precedents within the organisation
This created a fundamental challenge for project management:
How best to maintain delivery control when the solution, outcomes and risks are not fully knowable at the outset?
The organisation was not lacking project management capability. It had established delivery disciplines, governance processes and reporting expectations.
The issue was that AI delivery did not behave like traditional project delivery.
In the Orr Consulting AI Transformation Process, this case study demonstrates the Deliver-stage role of AI Project Management in recognising, documenting and progressively reducing uncertainty while maintaining clear governance, delivery control and accountability.
2. Situation
The Generative AI pilot was positioned as an early exploration of how AI tools could support:
Internal productivity
Knowledge management
Drafting and content generation
User-facing service improvements
This was a Use-mode AI project. The organisation was applying existing Generative AI products rather than building a new AI model or configuring a bespoke solution.
The principal uncertainties therefore related less to whether the underlying technology could be developed and more to where the tools would create value, how users would adopt them, how information and outputs should be governed and whether the evidence would justify wider deployment.
The project had:
Defined timelines
Allocated resources
Senior stakeholder interest
Expectations of demonstrable outcomes
However, several delivery tensions quickly emerged:
Use cases were evolving as understanding of the technology developed
Initial assumptions about value and applicability were being tested in real time
Different teams experienced different levels of benefit and adoption
Risks, particularly around data usage and governance, required ongoing clarification
Stakeholder expectations were high, but outcomes were inherently uncertain
This created a delivery environment where:
Scope was fluid
Learning was continuous
Certainty could not be assumed upfront
The project needed to maintain control and credibility while operating in this context.
The organisation therefore engaged Orr Consulting to provide structured AI Project Management support, maintaining clear governance and delivery discipline while allowing the pilot to learn and adapt.
3. Background
The organisation had a strong foundation in project delivery, with established approaches to:
Planning and scheduling
Governance and reporting
Risk and issue management
Stakeholder engagement
However, these approaches were typically applied to initiatives where:
Requirements were more stable
Solutions were better understood
Outcomes could be more confidently defined in advance
The generative AI pilot differed in several important ways:
The technology was new to the organisation
Use cases were exploratory rather than fully specified
Benefits needed to be observed and validated, not assumed
User adoption and behaviour were central to success
Applying a fixed and fully predictive delivery model would therefore risk constraining learning, creating unrealistic stakeholder expectations and interpreting necessary adaptation as delivery failure.
The project therefore required a disciplined but progressive project management approach.
4. Action Taken
Orr Consulting introduced a Progressive Project Management approach that combined clear project boundaries, governance and accountability with short, evidence-led delivery cycles designed to reduce uncertainty progressively.
The work followed Orr Consulting’s structured AI Project Management approach.
4.1 Project Boundaries
At the outset, the project defined:
The purpose of the pilot
The scope of tools and user groups involved
The duration and structure of the pilot
Success criteria at a high level
This ensured that, while detailed outcomes were uncertain, the project itself remained bounded and controlled.
This separated uncertainty within the pilot from uncertainty about how the pilot itself would be governed and managed.
4.2 Managing Uncertainty
Material project uncertainties were made explicit rather than being hidden within plans or presented as settled facts.
These included uncertainty relating to use-case value, user adoption, information governance, the quality and reliability of outputs, expected benefits and the case for future scale.
The uncertainties were then managed through established project controls. Depending on their nature, they were linked to evidence-generating deliverables, dependencies, assumptions, risks or issues.
A single uncertainty could result in several complementary project responses. The objective was not to create a separate control system, but to use established project-management disciplines to eliminate uncertainty where possible and reduce it to an acceptable level where it could not be removed completely.
4.3 Progressive Delivery Cycles
Rather than attempting to define all requirements upfront, delivery was structured into short cycles:
Early cycles focused on exploration and testing
Later cycles focused on refinement and targeted application
Learning from each cycle informed the next
Each cycle applied a short Discover–Design–Deliver sequence and was designed not only to produce project outputs, but also to reduce defined uncertainty relating to use cases, adoption, governance, risks and expected benefits.
This allowed the team to:
test assumptions early
adapt direction based on evidence
improve confidence progressively
maintain forward momentum without losing control
Regular checkpoints considered what evidence had been produced, which uncertainties had been reduced and whether the project should continue as planned, reshape its focus, pause particular activity or stop it.
4.4 Managing Evolving Scope
Scope was actively managed as an evolving construct:
New use cases were identified and assessed during the pilot
Some initial ideas were deprioritised or dropped
Focus was adjusted toward areas showing the most value
This avoided forcing the project into a fixed scope that did not reflect emerging understanding.
Scope change was therefore treated as evidence-led refinement rather than uncontrolled expansion.
4.5 Strengthening Governance
Given the nature of AI, particular attention was paid to governance:
Data usage and handling were monitored and controlled
Acceptable use of AI tools was defined and communicated
Risks were reviewed regularly as understanding developed
Stakeholder oversight was maintained through structured reporting
This ensured that control was maintained even as delivery evolved.
Governance decisions were revisited as evidence developed, recognising that the project’s risk profile could change alongside its use cases, users and delivery approach.
4.6 Tracking Benefits
Benefits were not assumed — they were actively tracked:
Expected benefits were defined at the outset
Real-world usage and outcomes were monitored during the pilot
Both anticipated and unanticipated benefits were captured
Results were reported through formal governance channels
This created a clear link between delivery activity and measurable value and provided the basis for defining, measuring and tracking benefits through a structured AI Benefits Realisation approach throughout the pilot.
4.7 Supporting Adoption
The project recognised that outcomes depended heavily on users:
Users were actively engaged in testing and feedback
Training and support were provided
Use cases were shaped based on real user experience
Confidence in using AI tools was built over time
This helped translate technical capability into practical value.
Adoption evidence also helped distinguish between limitations in the technology, weaknesses in particular use cases and issues that could be addressed through training, guidance or support.
5. Outcomes
The project management engagement created several notable outcomes, observations and lessons learned.
5.1 Controlled Pilot Delivery
The pilot was completed within its defined parameters despite evolving use cases, assumptions and delivery learning. Clear project boundaries, progressive delivery cycles and structured governance enabled the project to adapt without losing control or credibility.
5.2 Benefits Evidenced
More than 80% of the anticipated pilot benefits were realised during the pilot period, with additional previously unrecognised benefits also identified.
This demonstrated that benefits could be observed and tested during delivery rather than assumed in advance or assessed only after completion.
5.3 Understanding Value
The pilot improved organisational understanding of where Generative AI created the most value, where outcomes varied between user groups and which use cases warranted further development.
This enabled future decisions to be based on observed operational evidence rather than general enthusiasm for the technology.
5.4 Stakeholder Confidence
Regular risk review, controlled data use, acceptable-use guidance and structured reporting increased confidence among executive, risk and delivery stakeholders.
The project demonstrated that exploratory AI activity could proceed within credible governance arrangements.
5.5 Investment Evidence
The pilot created a more robust evidence base for subsequent business-case and scaling decisions. Leadership was able to consider future investment with clearer information about adoption, benefits, governance requirements, practical use cases and delivery capability.
This created a controlled path from exploratory pilot activity towards more structured and scalable AI adoption.
5.6 Reducing Uncertainty
The engagement reinforced an important lesson: effective AI Project Management does not depend on removing all uncertainty before delivery begins.
It depends on recognising and documenting uncertainty, linking it to appropriate deliverables and project controls, and using short delivery cycles and regular checkpoints to reduce it progressively.
This allowed the project to remain controlled without relying on false certainty or preventing evidence-led adaptation.
6. Final Thoughts
AI Project Management requires a shift in emphasis. It is not about abandoning control. It is about applying control differently.
AI projects are often characterised by higher uncertainty. Progressive Project Management provides the structured response by making that uncertainty visible and using evidence-generating deliverables, established project controls and short Discover–Design–Deliver cycles to reduce it progressively.
In a Use-mode Generative AI pilot, uncertainty may relate less to whether a new model can be developed and more to where existing tools create value, how users adopt them, how information and outputs should be governed and whether the benefits justify wider investment.
In this case, the pilot remained controlled and credible while use cases, benefits and risks were tested through delivery. The organisation completed the pilot, realised more than 80% of its anticipated benefits and created a stronger evidence base for subsequent investment decisions.
The most important outcome was not simply successful pilot completion. It was demonstrating that uncertainty could be managed through disciplined and progressive project leadership rather than disguised through unrealistic upfront assumptions.
This AI Services Case Study is part of the Orr Consulting AI Insights Library — structured thinking for AI transformation leaders and decision makers.
If your organisation would benefit from stronger delivery control across an uncertain or evolving AI project, we would be pleased to discuss your next AI steps.
Subscribe to Orr Consulting to receive occasional emails with practical AI Insights and updates.
