Implementing an Assurance Strategy
Although there are many ways of implementing an Assurance Strategy, perhaps the most structured and widely used is the Three Lines of Defence (3 LoD) Model, which will be used as an assurance strategy exemplar for this article.
Following the financial crisis of 2008/9, there was a realisation that many organisations did not have a structured approach to managing risk and did not provide assurance to Board level that risks were being appropriately identified and controlled.
Thus, in January 2013, the Institute of Internal Auditors published the 3 LoD Model. Its aim was to provide a comprehensive framework to consider the overall arrangements for managing risk and exercising control within an organisation.
Setting up a 3 LoD organisation is not a small undertaking, nor is it guaranteed to avoid unwelcome surprises.
Crossrail, for example, had a fully-fledged 3 LoD system in place and yet, on 31 August 2018 when the project had been inaccurately reported to be 97% complete, there was an announcement of a 9-month delay, which eventually became 4 years with the inevitable resulting cost overrun.
The question, quite rightly, is why, with all that assurance and governance in place, did Crossrail not know their project schedule was out of control.
This article is not a Lessons Learned for Crossrail, who are by no means alone in their experience.
Rather, it looks at the 3 LoD Model to see how it should be applied to major projects and the essentiality of using Data Analytics and Artificial Intelligence (AI) to enable meaningful insights and assurance to a project.
In so doing, it shows how Knowledge Concierge (KC) from Foresight Works can be used as part of a governance framework in order to assure against such surprises.
The Three Lines of Defence Model
Let’s start by looking at a typical 3 LoD Model definition:
- First line: Management (process owners) has the primary responsibility to own and manage risks associated with day-to-day operational activities. Other accountabilities assumed by the first line include the design, operation, and implementation of controls.
- Second line: The second-line function enables the identification of emerging risks in the daily operation of the business. It does this by providing compliance and oversight in the form of frameworks, policies, tools, and techniques to support risk and compliance management.
- Third line: The third-line function provides objective and independent assurance. The third line’s key responsibility is to assess whether the first- and second-line functions are operating effectively.
The financial sector roots are evident in this definition with emphasis on compliance, day-to-day operational activities, and risk management.
Day-to-day operational activities are controlled by processes and regulations which can be audited against by the 3rd LoD with the 2nd LoD providing the risk controls framework.
Indeed, this applies in a major project setting with designs to follow, regulations to comply with, as well as managing external risks such as supply chain stability, labour shortages, cost increases etc.
However, in addition to these risks, which should be part of any risk controls framework, major projects are held at gunpoint by time.
Getting the Right Controls
Much effort is put into controlling cost, often with complex governance regimes, but the cost is an outcome, mostly of time slippage.
Quality issues, for example, will cost in terms of rework but this can be overshadowed by the impact of the delay on other parts of the projects.
To put this into perspective, a major international organisation undertook a study across thousands of its projects and found that time contingency is worth twice that of budget.
In other words, if we were to put as much effort into schedule control and forward-looking predictive risk assessment as we do cost control, we could see twice the effect.
The reality is that we rarely do, and it is particularly difficult for the 2nd or 3rd LoDs to provide genuine insightful assurance of schedules that are owned and controlled by the 1st LoD.
The result is, according to research from Oxford University, that fewer than one in ten major projects deliver on time.
Schedule assurance is often a monthly meeting with a thick project report and a deck of slides with the project lead, or his/her scheduler, assuring everyone that everything is on track.
Earned Value Management (EVM) is a useful checking tool, but it deals with what ‘should’ have been completed and what ‘should’ have been spent i.e. it is a lag indicator.
EVM is beneficial as a project monitoring tool, but it is not a risk management tool as required by an assurance framework such as the 3 LoD model.
Furthermore, in major projects tasks, that can be seemingly innocuous and cost very little in terms of budget, can suddenly have a major impact on the overall schedule because of their interdependencies and criticality, leading to an axiomatic impact on time and therefore cost.
So ‘value’ in a major project is not just a measure of budget, but a measure of project schedule criticality and therefore the potential overall impact on the project should the task not be completed on time.
In major projects, it is essential that value is assessed in terms of project criticality, which is a core function of KC, as well as cost/budget.
The use of KC and EVM together in a project controls framework will be the subject of a follow-up article.
Managing Complexity
Major projects are very complex with countless interdependencies hidden in the depths of the schedule, having knock-on effects with each other.
Indeed, these interdependencies mean the idea of a ‘single critical’ path in the constantly changing environment of a major project, is a misnomer.
Expecting humans to be able to manage all that complexity and identify the level of criticality of each task in their heads is unrealistic and can only be achieved through Data Analytics and AI.
In terms of assurance, periodic auditing alone is unlikely to be an effective control, as a major project can slip months overnight due to the constantly changing nature of projects.
It is also a difficult and time-consuming exercise to effectively manually audit the contents of a complex schedule.
A Quantitative Schedule Risk Analysis (QSRA) exercise for example will take weeks to complete on a major project and it is based on judgement and therefore subject to optimism bias or, at worst, strategic misrepresentation (cover-up).
The previously referred to surprises happened when the problems had been buried deep in the data for months if not years, unidentified by traditional governance regimes and project management tools.
KC controls risk by assessing on a nearly continuous basis the criticality of every task in the schedule, recommends where to apply strict controls, and identifies what impacts seemingly minor slippages can have.
Additionally, it measures task intensity in any given period enabling resource smoothing.
As KC is accounting for every single task, it also gives an extremely accurate percentage of completion.
By analysing the data in this way, surprise slippages can be avoided and accurate forecasting achieved.
This is a fundamental part of the risk management process and should be the basis of a major project risk framework.
Therefore, the risk controls framework implemented by a 2nd LoD, or any assurance strategy, needs to be based on a deep assessment of the data and close tracking of all critical tasks as lead indicators using a tool such as KC.
The Value of Knowledge Concierge
In sum, time is the biggest enemy for major projects and the threats are buried in the data.
KC is a defensive weapon that has been developed specifically to look deep into schedules to identify risks that would otherwise not be visible and then produce prioritised task lists to keep the project on track.
In so doing, KC engenders a culture of curiosity by prompting and enabling leaders to ask insightful questions.
KC holds the project line and provides assurance on a nearly continuous basis, in a way that simply cannot be done with traditional tools and techniques.
Total visibility of the data and the ability to analyse and test it using algorithmic techniques provide transparency which is a great enabler for the frontline, but it is also critical to any governance framework in order to provide meaningful insights and assurance of the schedule.
Furthermore, using data that is readily available in existing tools, such as Primavera of Microsoft Project, provides assurance with a much-reduced need for audit and thereby reducing the impact on productivity.
Technical reviews and compliance checks will still be needed, but the crucial schedule assurance can be done remotely through the data using KC.
It is time to move our thinking into the data world to simplify the complexity of major projects and start to deliver on time and budget in a consistent manner.
For this to happen, we need to embrace Data Analytics and AI at every opportunity we can and place them at the heart of our project assurance strategy.
Author: Dr Peter Ewen, written for Foresight Works
