Project Estimation: Wideband Delphi (WBD)
Project estimates give an indicator of how long and what resources are required to complete a project. There are many methods available such as formalised procedures based upon database of historical metrics taken from completed projects and applied using mathematical calculation through to pure guesstimates using ‘feel-good’ figures for the various stages of a project.
This article presents an overview of the Wideband Delphi estimation technique, a relatively simple iterative method using experts (the project team) to provide input and actively take part in the estimation process. The article will describe the technique and how to apply it on software projects, and also consider some of the benefits this technique can provide.
Table of contents
; WBD Overview
; The Process
; Estimation Session Ground Rules
; WBD Strengths
; WBD Weaknesses
Project estimates often have a habit of becoming predictions for the project schedule and resource requirements. However, no matter what estimation technique is used the result is still just an estimate. There is always a degree of uncertainty in the figures – hopefully this diminishes
throughout the project lifecycle.
Software development is inherently unpredictable in nature. Therefore, any estimation figures produced need to be believable, first and foremost by the project team responsible for implementing the system. This is essential in order that the team collectively works together to realise the projects in-line with the estimation. There is also an additional effect that team confidence in the estimation figures will give your project stakeholders confidence. The importance of estimation is to try using best efforts understand the size of the project. One of the best ways of achieving this and at the same time getting the project team agreeing with the estimates is to actively include the team during the estimation process.
One such estimation technique that follows this ideal is Wideband Delphi (WBD). With this technique the project team members actually perform the role of estimators.
Unlike many formalised parametric estimation processes (COCOMO, Function Point Analysis...), WBD does not rely upon historical sets of metrics derived from previous projects. There are no magic formulae that are used. Instead the process uses the personal experience of the domain and technical expertise to derive the project estimation figures. These experts are the actual project team members.
WBD uses an iterative process, in which each team member is an estimator and is responsible for producing an individual estimate for the project. Through a process of review and re-estimation convergence on an acceptable set estimation values is achieved.
The following shows the activities undertaken when performing WBD for a project.
Before commencing the first estimation session, all participants must understand the objectives of the technique and also the all important ‘ground rules’ to be observed throughout the entire estimation activity.
During estimation each individual will use a standard form to record their estimation figures and any notes deemed relevant. The estimation session will continue until each individual has produced estimation figures, at which point the facilitator retrieves these figures.
The facilitator assesses the estimates and prepares a presentation including at least the following;
; All estimation figures
; Task list based upon system requirements
; Any identified factors affecting task estimation
A simple graph or table should be used to capture these figures. However it is also important to include the set of tasks used and present this alongside the estimation figures. The information can be presented individually to each member or to all as a group. In either case estimator anonymity must be maintained.
The primary objective of the estimation and assessment session is to gain convergence to within a predefined limit across all estimates.
Assuming convergence has not been achieved, the facilitator will present the estimates and task list to the project team. The aim is for all estimators to recognise valid tasks that have been missed and any factors that need to be taken into account in the next estimation iteration. The facilitator must continue the estimation process until convergence is achieved, usually within 3 iterations. Assessment
It is typical for an initial estimate to be over optimistic and to miss many tasks, including but not limited to the following;
; Sufficient testing (all types – unit, integration, system, performance, load…)
; Environment set up
; Preparation for deployment (training, user documentation…)
; Meetings and reviews
; Sufficient analysis and design
It is the job of the facilitator to recognise activities missing in the different project disciplines, but not to go into great detail of what must be included.
The second set of estimates may well err on the cautious side and be too high. This is the reason for performing at least 3 estimation sessions before considering that estimation is complete. The estimates plotted may look something like the following;
It is unlikely that final set of estimation figures will all coincide and the spread should give a guide to the uncertainty when present final estimation figures.
The estimation sessions are ideally fixed duration and this will vary depending upon the size of the project. For large projects, system partitioning may be required and estimation on subsystems performed.
Estimation Session Ground Rules
; All estimation sessions must be performed individually with no pair or group interaction.
o This is important to ensure that individual bias does not skew results
o Also individual estimation is likely to reveal a wider number of activities that need
to be taken into consideration
; Estimators must ignore external influencing factors – i.e. perceived expectation of a
‘suitable’ project estimate
o The aim is to achieve consensus amongst the team on a realistic figure rather
than provide one that the project sponsor wants to hear.
; Estimators must indicate factors affecting their figures
o This can help build a clearer picture of the task list
; Estimators will ignore holidays and other similar factors
o Ignore unnecessary complexity during the estimation process. Apply these
factors when defining the actual project timescale estimation.
; Anonymity must be observed during the whole estimation process.
o It is essential that the facilitator ensures bias does not creep into the process.
o Estimators must not reveal which figures they derived
; Estimation sessions ideally have a predefined time limit.
o Don’t worry about getting it wrong (its an estimate anyway) and figures can be
refined during successive iterations.
o Helps to maintain focus, continuity and progress
; It is a simple technique not requiring estimation experts
; Applicable to original projects where no previous metrics exist
; The process is an inclusive approach using all the project team to perform an active role
; Estimation figures are produced by team consensus through estimation iteration
sessions. More likely to mitigate impact of large individual errors
; An expert judgement driven technique using developers to estimate. They are most
likely to understand technical complexity and challenges when considering the
requirements in context
; Must have strong facilitator for estimation sessions to remain unbiased
; Estimates are no better than the abilities of the participants
WBD is a very simple technique that is as equally applicable to software as to any other discipline. Therefore there is virtually no scope for misinterpretation or ambiguity in applying the technique, and also no requirement for estimation experts.
Estimation is not a science and WBD technique recognises this by treating each project separately and using the project team (domain and technical experts) as the estimators. This is one of the key factors to achieving any degree of success with WBD, the other lies in applying an iterative approach (typically 3+ estimation sessions) in order to achieve convergence on estimates. WBD removes bias by ensuring estimators work alone and are not constrained by any perceived customer timescale constraints. This results in more believable estimates being produced and potentially highlighting to project sponsors, shortfalls in funding, inadequate resourcing or widely optimistic delivery dates.
At the end of the day, WBD results are still an estimate, just as they are for any other estimation technique.
One last point is that it is important not to fall into trap of turning estimates into predictions. This is very difficult to achieve (customers love to use estimates for absolute release dates). Therefore it is advisable that any estimates are qualified with a degree of variance (e.g. +/-30-40% at beginning of the project, and decreasing during the project lifecycle) whenever they are presented to project stakeholders.