Case Study — Maintaining Project Management Control in a High Uncertainty AI Pilot Project

1. Organisational Problem — AI Project Management in an AI Pilot Where Requirements, Outcomes and Risks Are Not Fully Defined

A large public sector organisation initiated a Generative AI pilot to explore the potential of AI tools to improve productivity, knowledge access and service delivery.

The pilot was supported at senior levels and aligned to broader digital transformation ambitions. However, unlike more traditional technology initiatives, the project began without:

• fully defined requirements

• stable or predictable delivery scope

• clear, pre-validated assumptions about outcomes

• established delivery patterns or precedents within the organisation

This created a fundamental challenge for project management:

How best to maintain delivery control when the solution, outcomes and risks are not fully knowable at the outset.

The organisation was not lacking project management capability. It had established delivery disciplines, governance processes and reporting expectations.

The issue was that AI delivery did not behave like traditional project delivery.

In the Orr Consulting AI Transformation Process, this reflects delivery activity where uncertainty remains high, and where Discover, Design and Deliver must operate in close, iterative cycles rather than as distinct sequential phases.

2. Situation

The generative AI pilot was positioned as an early exploration of how AI tools could support:

• internal productivity

• knowledge management

• drafting and content generation

• user-facing service improvements

The project had:

• defined timelines

• allocated resources

• senior stakeholder interest

• expectations of demonstrable outcomes

However, several delivery tensions quickly emerged:

• use cases were evolving as understanding of the technology developed

• initial assumptions about value and applicability were being tested in real time

• different teams experienced different levels of benefit and adoption

• risks, particularly around data usage and governance, required ongoing clarification

• stakeholder expectations were high, but outcomes were inherently uncertain

This created a delivery environment where:

• scope was fluid

• learning was continuous

• and certainty could not be assumed upfront

The project needed to maintain control and credibility while operating in this context.

3. Background

The organisation had a strong foundation in project delivery, with established approaches to:

• planning and scheduling

• governance and reporting

• risk and issue management

• stakeholder engagement

However, these approaches were typically applied to initiatives where:

• requirements were more stable

• solutions were better understood

• and outcomes could be more confidently defined in advance

The generative AI pilot differed in several important ways:

• the technology was new to the organisation

• use cases were exploratory rather than fully specified

• benefits needed to be observed and validated, not assumed

• user adoption and behaviour were central to success

This meant that applying a fully traditional delivery model without adaptation would risk:

• loss of control

• misaligned expectations

• or perceived delivery failure

The project therefore required a disciplined but adaptive project management approach.

4. Action Taken

The project adopted a structured approach to maintaining delivery control while recognising the inherent uncertainty of AI.

This combined clear project governance with iterative delivery cycles and active learning.

4.1 Establishing Clear Objectives and Boundaries

At the outset, the project defined:

• the purpose of the pilot

• the scope of tools and user groups involved

• the duration and structure of the pilot

• success criteria at a high level

This ensured that, while detailed outcomes were uncertain, the project itself remained bounded and controlled.

4.2 Structuring Delivery into Short Iterative Cycles

Rather than attempting to define all requirements upfront, delivery was structured into short cycles:

• early cycles focused on exploration and testing

• later cycles focused on refinement and targeted application

• learning from each cycle informed the next

This reflected a Discover → Design → Deliver micro-cycle within the project, allowing the team to:

• test assumptions early

• adapt direction based on evidence

• maintain forward momentum without losing control

4.3 Managing Scope as Evolving Rather Than Fixed

Scope was actively managed as an evolving construct:

• new use cases were identified and assessed during the pilot

• some initial ideas were deprioritised or dropped

• focus was adjusted toward areas showing the most value

This avoided forcing the project into a fixed scope that did not reflect emerging understanding.

4.4 Strengthening Governance and Risk Management

Given the nature of AI, particular attention was paid to governance:

• data usage and handling were monitored and controlled

• acceptable use of AI tools was defined and communicated

• risks were reviewed regularly as understanding developed

• stakeholder oversight was maintained through structured reporting

This ensured that control was maintained even as delivery evolved.

4.5 Actively Tracking and Evidencing Benefits

Benefits were not assumed — they were actively tracked:

• expected benefits were defined at the outset

• real-world usage and outcomes were monitored during the pilot

• both anticipated and unanticipated benefits were captured

• results were reported through formal governance channels

This created a clear link between delivery activity and measurable value.

4.6 Supporting Adoption and User Engagement

The project recognised that outcomes depended heavily on users:

• users were actively engaged in testing and feedback

• training and support were provided

• use cases were shaped based on real user experience

• confidence in using AI tools was built over time

This helped translate technical capability into practical value.

5. Outcomes

The project was able to maintain delivery control while operating in a high-uncertainty environment.

This resulted in:

• successful completion of the pilot within defined parameters

• more than 80% of anticipated benefits being realised during the pilot period

• identification of additional, previously unrecognised benefits

• improved understanding of where AI added most value within the organisation

• increased confidence among stakeholders in both the technology and its governance

• a stronger evidence base for future investment decisions

• a formal project review was completed, capturing lessons learned and informing future AI delivery approaches

Importantly, the project demonstrated that:

• AI delivery can be controlled

• value can be evidenced

• and uncertainty can be managed without requiring full upfront definition of scope and outcomes

  
  

6. Recommended Next Steps

Following the pilot, the organisation identified several next steps.

First, it would use the evidence generated to support business case development for scaling AI capability, including broader deployment of generative AI tools.

Second, it would continue structured use case identification and prioritisation, focusing on areas with demonstrated value.

Third, it would strengthen governance and assurance arrangements to support wider adoption.

Fourth, it would build organisational capability through ongoing training and support.

These steps would enable the organisation to move from pilot activity toward more structured and scalable AI delivery.

7. Final Thoughts

AI project management requires a shift in emphasis.

It is not about abandoning control. It is about applying control differently.

Where traditional projects rely on upfront definition and predictability, AI projects require:

• iterative learning

• active assumption testing

• adaptive scope management

• and continuous alignment between delivery and value

This case study shows that, with the right approach, it is possible to maintain delivery discipline while working in an environment where certainty is limited.

This Case Study is part of the Orr Consulting AI Insights Library — structured thinking for AI transformation leaders and decision makers.

8. Call to Action

If your organisation is delivering AI pilots and facing uncertainty around scope, outcomes or control, a structured project management approach can help maintain discipline while enabling learning.

If this reflects your current situation, Orr Consulting would be pleased to discuss your next AI steps.

Discuss your next AI steps

Subscribe to Orr Consulting to receive occasional emails with practical AI Insights and updates.

Subscribe to Orr Consulting