Benchmarking and Best Practices
Confidentiality Statement
This <document title> along with all attachments hereto shall be considered <company>’s
Proprietary/Confidential Information
gantthead.com Benchmarking and Best Practices
The objective of this piece is to identify the best practices in the field of multi-project management and summarize the findings. After an extensive literature survey and discussions with the academic community, it was found that a benchmarked process that completely defines the needs of multi-project management is yet to be established. However, the idea of multi-project management been has introduced, and benchmarks for several components of that process have been used in practice. The following categorizes these findings.
Project Selection and Prioritization Methodology
The critical function in a multi-project organization is to have a well-balanced portfolio of projects supporting the initiatives outlined in the IT strategy and the business objectives. Prioritization is required at the beginning of each project to position it relative to other projects already underway and to new projects coming into the queue. In addition, factors such as financial means, people and competition change with time and will, in turn, cause changes to the prioritization of projects in the portfolio. To create a truly effective mix of projects, the organization must consider a wide range of factors that include mission, vision objectives, short-, mid-, long-term goals of the organization in addition to the size of the proposed project portfolio and funding guideline. The conventional prioritization process includes the following sub-processes:
; Setting investment strategy and objectives
; Classification of project investments
; Use of pre-launch charters
; Constructing project portfolio
Setting Investment Strategy and Objectives
The critical aspect that needs to be covered before investing capital in any
project is to derive a method to measure the project proposals against business
objectives and as well as other project proposals. An unbiased project monitoring
method must be established to measure individual project return, weigh multiple
projects within the portfolio and ensure continued alignment with overall
corporate objectives. The measuring techniques that are used to rank the
projects in the portfolio must be established. Some of more widely practiced
methods include:
; Financial methods (NPV, ROI or pay back period)
; Expected commercial value method
; Productivity index
; Balance score card method
; Portfolio mapping (bubble diagrams)
Expected Commercial Value Method (ECV)
The Expected Commercial Value method seeks to maximize the "value" or
commercial worth of the portfolio subject to certain budget constraints. It is one of
the better thought out financial methods featuring new twists that makes it
particularly appropriate to portfolio management. ECV determines the
commercial worth of each of the projects for the organization. It is based on a
decision tree analysis and considers the future stream of earnings from the
2 ?2007 gantthead.com 2
gantthead.com Benchmarking and Best Practices
project, probability of success and development costs. In addition, it incorporates the strategic importance of the project. The method also incorporates the strategic importance of the project. Because the process is based on a decision tree approach, the Go/Kill decision process is an incremental one. The method also recognizes the issue of constrained resources. It attempts to maximize the expected value in light of this constraint. The major weakness of this method is its reliance on financial and other quantitative data. Accurate estimates of all projects future stream of earnings, development costs and the probability of success is often unreliable or simply not available early in the life of a project. In addition, this method does not look at how risk is distributed throughout the portfolio.
Productivity Index Method (PI)
The productivity index method is similar to the ECV method and shares many of its strengths and weakness. The PI method also tries to maximize the financial value of the portfolio for a given resource.
Balance Scorecard Method (BSC)
Scoring models have long been used to make Go/Kill decisions at individual project reviews but are also applicable to project prioritization and portfolio management. BSC defines a variety of metrics that measure how IT departments deliver services and support to line of business operations. The BSC method can compare and rank multiple, disparate projects of similar merit, map them against scarce resources, and provide a metric system to measure a particular project progress and/or success. The best approach to building a BSC involves establishing cross a functional management team to drive the effort with subgroups established as needed. These subgroups would be responsible for individual processes or vertical functions. The steps involved in building a BSC include:
1. Define the organization's long term strategic goals
2. Interpret these goals at the process level from the perspective of each BSC
component (customer, internal business process, financial, learning/growth) 3. Translate these goals into measurable metrics
4. Determine how and when data for the metrics are gathered and how they
interact
5. Develop metric report cards that track performance versus goals and provide
a detailed gap analysis
6. Map metrics into compensation plans and tangible rewards for the
achievement of BSC goals
7. Develop feedback loops when goals are not achieved.
Meta Group Inc advocates all organizations to utilize the BSC method as a component in the IT project fulfillment process.
3 ?2007 gantthead.com 3
gantthead.com Benchmarking and Best Practices
Portfolio Mapping (Bubble Diagrams)
This method utilizes the bubble diagrams with projects falling into different quadrants like pearls, oysters, white elephants and bread-butter projects. Nearly 8.8% of business surveyed in a recent study use this as the dominant method in the prioritization of projects in their portfolio. The best feature of portfolio mapping is that it forces management to deal with the issue of resources. The map forces the management to consider the resource implication of adding one or more projects to the portfolio. The disadvantage of this method is that it deals with many data factors that are uncertain in nature. An example would be Net Present Value (NPV). NPV requires optimistic and pessimistic estimates for many variables. Estimates that are skewed significantly can lead an organization to place one project over another.
Classification of Project Investments
Projects are generally classified under two broad categories, survival and growth. If the proposed project does not fall under either of these categories, the validity of the investment on these projects must be further examined. This plays a critical role in determining where the discretionary and non-discretionary funding choices are to be made.
Survival Projects
Survival projects are "must-do" projects. Failure to include these projects in the portfolio could cause repercussions that could seriously hamper the financial growth of the business. These projects would be considered a non-discretionary investment and would by default be included in the project portfolio. Growth Projects
Growth projects would be considered a discretionary investment. The business would have the option to go forward, discard the project all together, or reevaluate the project's value and validity. In essence, these projects will allow the organization to grow and flourish.
Since the accomplishment of company strategy depends on the utilization of internal resources and the development of company know how by means of project portfolio, classification of projects is also done with reference to critical resource consumption and innovative content of the project. This method proposes construction of a two-dimension project classification matrix that includes critical resource consumption (X) and innovative content (Y). The components of the matrix included the following groups of projects. Know-How Oriented Projects
These are innovative projects where the resources are managed effectively and high levels of efficiency are achieved. Development of know-how within a company is crucial to increasing projects' efficiency and effectiveness.
4 ?2007 gantthead.com 4
gantthead.com Benchmarking and Best Practices
High Critical Projects
These projects consume a large number of resources with the project and contain a high level of innovative content. These projects will increase the development of know-how development.
High Performer Projects
A project in this category has a low criticality; however there are many resources with experience in the subject area of the project. Therefore, wide margins can be achieved by product quality and the efficiency of resources at the project and functional level.
Resource-Oriented Projects
These projects are characterized by significant output - due to the fact that they are repeated often. However, because the project consumes so many critical resources, resource management may become convoluted; therefore, reducing efficiency.
Based on these classifications, a methodology for setting priority of projects in light of resource allocation in order to achieve company objectives has been used in some cases. It is known as the Analytic Hierarchy Process (AHP) which is a decision methodology used in prioritizing its elements in each level by pair wise comparison importance with respect to a common attribute.
Use of Pre-launch Charter
IT organizations are required to justify/qualify projects thoroughly before committing IT resources to their delivery. The use of pre-launch charters increase the effectiveness of IT project efforts by enabling the business to be focussed on those projects with the greatest business impact. The pre-launch consists of research, discovery and high-level design activities. It is also a consensus building exercise, concluding with an agreed upon project charter that serves as a project-based, service-level agreement for its implementation. Meta Group recommends the use of project pre-launch charters because of the fact that it will lead to the early disqualification of non-justifiable projects resulting in significant savings in otherwise wasted IT resources. Both the business technology strategists and the project managers should develop charters. Typical elements of a project charter include background, project objectives, scope, approach, deliverables, schedule and staffing and issues and assumptions.
Portfolio Construction
Construction of the project portfolio begins once the projects are classified as either survival or growth. The main components that play a critical role in construction are project allocation and project selection. There are two approaches popularly used in the project allocation and selection step of the analysis. The top down approach takes an overview of economic themes, trends and how they affect the overall organizational growth. It does not validate and approve individual projects. The bottom up approach is based on individual project benefits and projects Roes to the business unit.
5 ?2007 gantthead.com 5
gantthead.com Benchmarking and Best Practices
Linking Strategy to Portfolio
The study of portfolio management practices revealed three common goals. They
are as follows:
1. Maximizing the value of the portfolio
2. Achieving the right balance or mix of projects in the portfolio
3. Linking the portfolio to the strategy of the business.
Methods discussed for project prioritization and selection can be used to
accomplish the first two goals. To achieve strategic alignment in the portfolio, two
general approaches were observed.
; Building strategic criteria into project selection tools
; Applying top down strategy models
Building strategic criteria into project selection tools
Scoring models not only maximize the value of the portfolio but also can be used
to ensure strategic fit. This can be achieved by building a scoring model with
number of strategic questions.
Top down strategy models.
These strategy models ensure that the eventual portfolio coincides with the
business strategy. There are two variations of this model, the strategic bucket
model and the stratplan model.
Strategic Bucket Model
This top down method operates from the principle that implementing strategy equates to spending money on specific projects. Thus setting the portfolio really means setting spending targets. This method begins with the business strategy and requires the upper management of the business to make decisions on how they wish to allocate their financial resources. This includes defining a vision and strategy for the business and a general plan of attack to achieve those goals. The criteria for fund allocation includes strategic goals, type of the project, familiarity to the business, geography etc. These enable the creation of "envelopes of money" also called buckets. Existing projects are categorized into each bucket. Then, a dollar amount can be determined for each bucket. Once the spending amount is determined, the project can be prioritized, using either a scoring model or predetermine financial criteria. This brings about the creation of the ultimate portfolio. Upon examining the projects contained within the portfolio, one can see they coincide with strategy and desired spending levels desired by management. The strategic bucket model has several strengths. First, it creates a strong link between the business strategy and spending. As a firm continues to use this approach, spending levels will continue to meet management's expectations. In addition, this model recognizes that different types of projects (example: development vs. cost reduction) may compete for the same resources. Considering these projects within a portfolio framework allows management receive a more realistic and informed view on resource allocation. Lastly, different criteria can be used for different types of projects. Therefore, a criteria list that applies to every type of project that might take place within an organization need not be created. The major weakness of this model is that is takes a
6 ?2007 gantthead.com 6
gantthead.com Benchmarking and Best Practices
substantial amount of time and effort on the part of senior management. In addition, by forcing management to make decisions on resources may provoke unrealistic decisions without consideration placed on specific projects.
StratPlan Model
This method is similar to the strategic bucket model in principle except that it reverses the order of steps. Instead of prioritizing projects that bring about the creation of a portfolio, it ranks the portfolio of projects using a scoring model. Then, it looks to see if the projects derived from the prioritization parallels with the business strategy. It determines desired spending levels, identifies gaps between desired and actual spending levels, and portfolio of projects are then complied accordingly, which is similar to the Strategic Buck model. In contrast to the Strategic Bucket Model, the StratPlan model is not a difficult to implement and it does not require as much time and effort of senior management.
Portfolio Validation
To help IS organizations prioritize and schedule, account for costs and efficiently
apply resources across the project portfolios, Gartner Group recommends not
only to analyze the project portfolio mix but also to validate it on an ongoing basis.
Because phases interact on a continuous basis, the initial order or priority of the
portfolio of projects may change. Changes in market and business conditions,
and technology may change the prioritization as well as introduction of new
projects, and the availability of critical resources.
Once the portfolio is completed, the business unit approves it and measurement
can begin. The following factors are generally considered in portfolio selection:
; Fit with business or corporate strategy
; Strategic merit to business
; Competitive advantage
; Forecast on return for each project
; Estimated risk for each project
; Prioritization technique for ordering projects
; Time and cost for completion of each project
Re-balancing the portfolio is carried out when new projects enter the queue. The
same weighing criteria listed above are used to change the portfolio or replace
any projects within the portfolio.
Based on the study conducted, the following section summarizes the suggested
best practices that can be incorporated to achieve business objectives. Many of
the benchmarked processes discussed are practiced widely. Sufficient
information has been documented to corroborate the application of those
processes to manage IT projects and needs.
Best Practice: Organizations incorporate mechanisms to manage both the
approval/prioritization of projects and the coordination of project delivery. These
organizations adopt a formal process for tracking project progress from inception
through completion or cancellation.
7 ?2007 gantthead.com 7
gantthead.com Benchmarking and Best Practices
Mechanisms that completely define the needs of multi-project organizations are yet to be established. However, commonly practiced methods for several components of the overall process have been identified. These include: 1. Multi-project organizational structures
2. Projects in the portfolio
3. Project portfolio construction integrated with business strategy 4. Validation and re-prioritization of the project portfolio
5. Controlling and Tracking project execution
Best Practice: IT organizations adopt project pre-launch charters that
justify/qualify projects thoroughly before committing IT resources to their delivery. Pre-launch charters are used to justify/qualify projects thoroughly before committing resources to their delivery. Disqualifying non-justifiable projects result in significant cost savings. The use of charters will result in increased effectiveness of IT project efforts by enabling resources to be focussed on those projects with the greatest business impact. The pre-launch represents the call for IT departments to step beyond the role of blind technology implementers to a proactive leader in the effective application of technology for the advancement of business.
Best Practice: Management should apply financial methods to justify and to carry out financial analysis on IT investment proposals.
CIO's and senior IT managers increasingly face pressure to cost-justify IT investments on a financial basis. To ensure that IT investments are appropriately positioned, insight into how investments are viewed from a financial perspective is essential. Net present value (NPV), Internal rate of return (IRR) and payback periods are some of the commonly practiced methods among many
organizations. Since each method of analysis has its own drawbacks, Gartner Research group advocates using a combination of methods. Because of emerging investment practices, CIO's and senior IT managers should become well versed in various financial investment justification methods. Best Practice: Management should conduct stringent risk assessment of any significant project before deciding whether or not to charter it. Prior to chartering a project, IS/AD management should make a disciplined risk assessment of any project likely to take more than one month of man effort. This will help organizations to avoid expending substantial resources on a potentially risky project that may be cancelled. Organizations that lack stringent risk assessment procedures will continue to cancel more than 20 percent of projects in the execution phase through 2002.
Best Practice: Organizations should implement the balanced score card (BSC) as a tool to help set strategic direction, assist with portfolio construction and link tactical actions with strategic vision.
Well-established project prioritization methods for multi-project organizations have been commonly practiced, and they can be adopted to suit the client’s
needs. Scoring models are commonly used to make go/kill decisions for
8 ?2007 gantthead.com 8
gantthead.com Benchmarking and Best Practices
individual projects. They can also be applied when applied to project prioritization and portfolio management. An appropriate scoring model based on several criteria including business objectives and goals is the simplest technique that can be applied for project prioritization and selection. BSC defines a variety of metrics that measure how IT departments deliver services and support to line of business operations. BSC can compare and rank multiple, disparate projects of similar merit; map them against scarce resources, and provide a metric system against which to measure project success/progress. Discussions with Meta Group revealed that the BSC method has been recommended as a component in the project fulfillment process and is proven to be effective.
Best Practice: Periodic validation and re-prioritization of projects must be performed in order to efficiently apply resources across the portfolio. Our research revealed that organizations should not only analyze the project portfolio, but continuously validate it. A four phased process that explains portfolio validation has been defined. The suggested phases include: 1. Chartering
2. Prioritization
3. Execution
4. Closure
Prioritization and Execution phases are interlinked to one another suggesting the re-prioritization and re-scheduling of projects based on portfolio validation. In order to prioritize, schedule, account for costs and effectively apply resources across project portfolios, organization must continually validate the project portfolio mix and re-prioritize when needed. The proposed four-phase framework allows the organization to understand their project portfolios and avoid cancelled projects and insufficient resource utilization. Research studies have revealed that a typical AD organization with 100 developers can expect to spend more than $10 million on cancelled software projects through 2002, without changes to their project management processes. Therefore, continuous validation of the project portfolio will help multi-project organizations to maximize return on their investments and reduce potential cost overruns, late delivery and scope creep. Best Practice: Management should establish a simple progress reporting policy that keeps sponsors involved in the delivery of IT projects.
Organizations commonly use formal and proven techniques to track project progress. Effective project managers assign milestones to break projects into smaller and manageable chunks. This study revealed that the most common and effective progress reporting technique is the Red Light/Green Light approach. Gartner Research Group advocates the use of this technique noting its simplicity and effectiveness in tracking project progress. This approach has been successfully implemented at USAA for progress reporting at specified time intervals. In addition, the Earned Value Analysis (EVA) method is highly recommended to predict the project outcomes and track project progress.
9 ?2007 gantthead.com 9