DOC

IT Showcase LOB Application Compatibility Technical White Paper

By Doris Greene,2014-11-13 15:10
16 views 0
IT Showcase LOB Application Compatibility Technical White Paper

    LOB Application Compatibility Using Microsoft IT‘s LOB Application Compatibility Testing Processes

    Technical White Paper

    Published: March 2008

CONTENTS

    Executive Summary ....................................................................................................... 3 Introduction .................................................................................................................... 4 Portfolio Management .................................................................................................... 6 Application Targeting ..................................................................................................... 8 Planning ......................................................................................................................... 9 Communication .............................................................................................................. 10 Testing ............................................................................................................................ 11 Reporting ........................................................................................................................ 13 Pilot Program: Dogfooding ............................................................................................ 14 Conclusion ..................................................................................................................... 15 For More Information...................................................................................................... 16

Situation EXECUTIVE SUMMARY

    Microsoft IT maintains a software Application compatibility is a critical blocking issue during the deployment of new portfolio of approximately 1,500

    applications. These applications must technologies or applications in an enterprise environment. Microsoft Information Technology be tested prior to software (Microsoft IT) faces the same challenge, with the added complexity of multiple beta releases deployments. Testers belong to of new applications and technologies, while maintaining an application portfolio that consists different business units within the of more than 1,500 applications. Microsoft IT tests approximately 46 percent of the company and are distributed around the world. application portfolio prior to each internal product release. This group of key applications

    consists of the most critical line-of-business (LOB) applications used within Microsoft. Solution

    Microsoft IT developed an application-Microsoft IT‘s application compatibility testing program achieves the following goals: tracking method that simplified the

    process of selecting applications for ; Minimize investment in application testing by targeting only a subset of the most sample-based testing. By identifying important and representative applications. groups of applications that have

    similar data processing, controls, ; Critical applications do not break during software deployment. underlying technology, and methods, ; The cost associated with application compatibility testing is minimized by using virtual Microsoft IT is able to test machine (VM) images instead of hardware with multiple configurations. approximately 4 to 6 percent of the total applications and gain a ; Minimize employee dissatisfaction resulting from application compatibility issues. reasonable assurance of compatibility

    for all. Testers from various business units within the Microsoft IT, located around the world perform

    the testing on selected critical LOB applications. They perform the testing using VM images Benefits

    hosted on a centralized server computer running Microsoft? Virtual Server 2005 Release 2 ; Improved employee satisfaction (R2), the second release of Virtual Server 2005. By using VM images, the testers can avoid with the solution

    ; Streamlined and automated testing hardware or software configuration issues that can delay the testing process. processes and workflow All of Microsoft IT‘s testing data and application data is stored within a database-driven, ; Ability to gain reliable results while

    testing fewer applications in each internal, Web-based software portfolio tool. Microsoft IT can query the data to generate pass detailed reports of application compatibility status throughout the testing process, and it can ; Reduced effort to administer and use the data to identify applications for the key test group. Combining individual data results support the solution with historical trends enables Microsoft IT to predict the compatibility results of the entire Products & Technologies portfolio based on the 46 percent of applications tested.

    ; Microsoft Office SharePoint The process that Microsoft IT uses is separated into phases: Server 2007

    ; Microsoft Virtual Server 2005 ; Portfolio management, Track all applications present in the enterprise environment, ; Internal software portfolio tool with appropriate contact information for personnel responsible for testing and support.

    ; Application targeting. Identify applications that are critical to the business flow of the

    organization based on portfolio data and historical trends. Create a list of core, critical

    applications to be tested for each software deployment.

    ; Planning. Create application testing schedules that permit enough time to fully test the

    core group of applications before each deployment milestone.

    ; Communication. Frequently communicate schedule and testing status with the test

    team and stakeholders. Clearly communicate testing configurations and goals with

    testers.

    ; Testing. The testing phase includes the tests that the team runs as well as limited pilot

    deployments.

    ; Reporting. Test results are recorded in the application portfolio tool to provide accurate

    historical data.

    LOB Application Compatibility Page 3

INTRODUCTION

    Application compatibility is a concern any time an organization deploys new software to an existing computer environment. Whether the organization is deploying a new operating system or a new Web browser, it must ensure that the applications that users need to perform their daily tasks continue to function properly. Microsoft IT shares this challenge by being responsible for deploying and maintaining more than 1,500 internal LOB applications in an environment of mixed and changing operating systems, productivity applications, Web browsers, and development tools.

    With such a large portfolio of applications and a rapidly changing environment, Microsoft IT must identify commonalities between the applications in the portfolio as a means of grouping applications for testing. Microsoft IT has found that applications can be grouped by the common components or underlying technologies, such as Microsoft Office Excel? spreadsheets, Microsoft Office Word documents, or the Microsoft .NET Framework. Or, the applications can be tracked over time for a historical sense of similarity to other applications. Based on this technology-dependency mapping, Microsoft IT can provide advance warning to application owners when changes may affect their application. This methodology, combined with substantial historical testing data, has enabled Microsoft IT to require testing only on approximately 60 key applications among the more than 1,500 present in the organization. For purposes of this paper, Microsoft IT defines an application as a piece of software that

    more than one person uses. This broad definition encompasses Web pages, databases, and client server tools. An LOB application is defined here as an application used to perform a

    key business function, such as purchasing, staffing, or expense reporting, and is most likely an application developed within the company. The LOB applications referenced in this paper are nearly all Web-based applications used for internal business purposes. Because of the broad variety of applications that internal employees use, the task of tracking them is monumental; however, having to test every one of those applications individually would require a vast number of testers. Microsoft IT has instead placed the emphasis on testing key ―indicator‖ applications. Just as indicator species in wildlife can point out a looming danger in nature, a problem with a key indicator application can signal likely issues with other, similar applications. The key indicator applications that Microsoft IT tests tend to be large, complex business solutions. By conducting a successful test pass on one of these applications, Microsoft IT gains confidence that the other applications with similar design, dependencies, and underlying technology will also pass if tested.

    Microsoft IT tests applications for several key reasons. The primary reason is to ensure that the internal LOB applications are compatible with new software before it is deployed to the corporate desktop environment. This means more than just performing tests in preparation for deploy a new operating system. It also means performing standardized testing when a new user application, such as a new version of Microsoft Office, is deployed. The testing process that Microsoft IT uses identifies issues with key functionality in the LOB applications and focuses on enterprise user scenarios.

    Jim LaBreck, a senior program manager in Microsoft IT, describes his worldwide virtual team of testers as the enterprise quality bar of Microsoft. ―The issues that we find and get the

    product teams to fix prior to release are issues that our external customers won‘t have to face,‖ LaBreck says.

    LOB Application Compatibility Page 4

    A goal of the testing process is to identify issues early in the product cycles for new applications or new version of applications. By identifying issues early in the process and providing clear feedback to the product teams responsible for the applications, Microsoft IT can minimize the support impact on users and help the product teams create high-quality applications.

    LOB Application Compatibility Page 5

PORTFOLIO MANAGEMENT

    The first step in testing for application compatibility is knowing the applications present in an environment. Microsoft IT gains this knowledge through an internal software portfolio-management tool that maintains a database of all applications deployed in the organization for general use. Microsoft IT has found it important to track not only the basic information of application name, function, and location, but also details that identify:

    ; The number of people who use the application.

    ; The people responsible for supporting the application.

    ; The dependencies that the application may have on other software or specific

    technologies.

    Microsoft IT requires that any time a new application is created for internal use, it must be entered into the portfolio tool. Microsoft IT further requires that, at a minimum, the person responsible for the application must enter the basic identifying information for the application, such as name and version, and the contact information for the people responsible for the application‘s compatibility impact. Typically, that contact person will be the person who is dedicated to testing the LOB applications for that business unit.

    Microsoft IT tracks the following information for each LOB application:

    ; Application name. The name by which the application is known in the network

    environment. The application name is useful mostly in terms of describing an issue when

    Microsoft IT contacts the product group.

    ; Application version. Microsoft IT tracks each version of an application as if it were a

    completely separate program. The reason is that not only can underlying technologies

    change from one version to the next, but the contact information and responsible parties

    can also change.

    ; Version identification. Because Microsoft IT tracks individual versions as separate

    entries, proper version identification is vital. Each version of an application may have

    different dependencies or contact information.

    ; Estimated number of users. This item is an estimate on the part of application owners.

    The main importance of this information is to help assess the impact on the company if

    the application fails. This is also an important criterion for selecting applications to be

    part of the key test group.

    ; Whether the application is a non-Microsoft application. Because Microsoft IT has no

    control over the development of non-Microsoft applications and because Microsoft

    customers likely use non-Microsoft applications, these applications receive a high priority

    in testing.

    ; Business priority (for example, mission, business, external). The highest-priority

    applications affect Microsoft as a wholefor example, an expense tool or an ordering

    tool that employees across the entire company usemay be considered either mission

    or business critical. Other factors include the nature of the application‘s function or even

    the visibility the application has because a senior executive uses it.

    ; Technology dependencies. The technology dependencies field is one of the most

    important, because it enables Microsoft IT to group the application with other

    applications based on the underlying technology. Knowing that a change is coming in a

    technology dependency, such as a new version of Microsoft .NET Framework, enables LOB Application Compatibility Page 6

    Microsoft IT to query the portfolio tool to find applications that depend on that technology

    and build a test group. Based on this information, Microsoft IT can notify users of the

    applications that such changes will potentially affect, and can provide these users with

    details of the change and the expected impact.

    ; Test results. Microsoft IT uses the test results to analyze compatibility trends over time

    and to generate reports. Tracking the results over time enables more-accurate

    predictions of compatibility issues and more-focused testing. Applications that do not

    present any failing marks during testing might even be bypassed in future testing. ; Contact information. This is a critical data item for testing program success. Having

    accurate contact information enables Microsoft IT to pass along vital information

    efficiently. Without this information, Microsoft IT would lose valuable time trying to find

    the responsible party.

    Tracking this information reduces the time and effort required in subsequent testing and identifies risks. For example, if one Web-based application that depends on a common underlying technology fails, the portfolio tool enables Microsoft IT to determine other applications that depend on the same underlying technology so that the team members can quickly evaluate whether those applications have the same issue.

    A software portfolio tool can take many formsfrom a simple spreadsheet to a Web-based

    interface to a centralized database. The important factors to consider in choosing a portfolio solution include ease of long-term use, the ability to find information quickly as the number of applications increases, and the completeness of data stored. A database is highly useful because of the ability to group records by various relationships in the data, such as the technology dependencies.

    LOB Application Compatibility Page 7

APPLICATION TARGETING

    Perhaps the most remarkable aspect of the LOB testing that Microsoft IT performs is the success of the testing program that results from application targeting. Because the group already has a complete list of the applications present in the organization, Microsoft IT can focus on testing a small group of high-priority applications that will serve as a litmus test for the remaining applications.

    Microsoft IT maintains a list of approximately 60 applications each for the Windows? operating system and Microsoft Office in its key priority test group. These applications are crucial to business functions across the company. Microsoft IT cross-references the applications by technology dependency within this group. Microsoft IT asks individual business groups within the company to volunteer information about the importance of the LOB applications that they use. The information enables Microsoft IT to make better decisions on the relative priority of the applications. Microsoft IT also uses historical data from past testing cycles to identify applications that had issues in the past. For instance, an application that always breaks when Excel is upgraded would be identified as an application to include in the key test group.

    According to Jim LaBreck, ―Based on historical data, we know which applications are more likely to have trouble and which applications basic business (payroll, expense reporting, procurement, etc.) simply cannot continue without.‖

    Inclusion in the key test group has benefits to the business groups responsible for the applications in the group. The product groups give extra effort to resolving any bugs filed against applications in the key test group, which directly benefits Microsoft IT and other business groups. If Microsoft IT finds an issue in one of the key test applications, it may stop the deployment of the related product or technology until the issue is resolved. If Microsoft IT determines that an issue is related to the underlying technology, such as a new release of Microsoft .NET Framework, it works with the product group to resolve the issue, directly benefitting all customers who will use the new version of the technology in the future. Microsoft IT expects applications in the key test group to go through testing for every scheduled test pass. The business units with applications in the key test group make a commitment to test on each pass but are permitted to skip a test pass if business activities require it. This system helps to maintain the working relationship between Microsoft IT and the product groups, and it enables Microsoft IT to respond with valuable feedback during the application development phase.

    LOB Application Compatibility Page 8

PLANNING

    Planning for application compatibility testing is crucial to the success of a deployment. For the testing to be successful, enough time must exist to schedule testing, perform the tests, gather results, and forward all data to the product groups so that they can resolve the bugs before deployment. Microsoft IT schedules test passes before key product milestones and internal interim production builds (often referred to as dog food releases at Microsoft). The

    intent of this scheduling is to be certain that potential compatibility issues are properly identified while there is time to resolve the issues before the customer is affected. ―Dogfooding‖ is a way of life at Microsoft. The term originally came from an internal communication that expressed the idea that ―we need to use our own products to prove that

    they're good enough for our customers.‖ In real terms, the dog food concept means that long

    before a software product enters a public beta test phase, Microsoft employees have been using it in their daily work. Using beta applications in the daily workflow is a clearly stated internal expectation at Microsoft, and every employee participates.

    For major test plans, Microsoft IT conducts one-week test passes approximately one month before scheduled major release milestones. At the end of that time, Microsoft IT gathers the results and communicates the feedback to the stakeholders, including the Microsoft IT desktop deployment team, the Helpdesk, and the product groups.

    When Microsoft IT finishes defining the schedule, it communicates the schedule to the project stakeholders. In some cases, the stakeholders have additional concerns that may cause a reevaluation of the schedule. The LOB application testing is an iterative process that may require numerous passes before Microsoft IT deems a product ready for general consumption. Scheduling the test passes within Microsoft IT typically means an initial discussion with the product team or Technology Adoption Partner (TAP) team and coordinating with other, concurrent testing schedules. Microsoft IT publishes the proposed schedule to a Windows SharePoint? Services team site, which serves as the central repository for team communication, scheduling, and reporting. Microsoft IT also communicates schedules through e-mail to distribution lists that contain testers and stakeholders.

    LOB Application Compatibility Page 9

COMMUNICATION

    Preparing for a deployment can be a very busy time. By providing multiple channels of communication between itself and the various project stakeholders, Microsoft IT ensures prompt delivery of feedback, sharing of suggestions for features and functionality with the product teams, and sharing of the latest support information with users. Microsoft IT uses a combination of the following communication channels:

    ; LOB testing aliases. E-mail is one of the primary modes of communication in the

    organization because of the wide geographic distribution of the testers. By creating e-

    mail group aliases, Microsoft IT can share status messages and other communication

    directly with the people involved in the project. E-mail communications tend to be timely

    and immediate; they communicate the current status and any information the

    stakeholders need to know at that time.

    ; LOB testing Web site. Maintaining a Web site on the organization‘s intranet is another

    convenient mode of communication. The LOB Windows SharePoint Services site for

    Microsoft IT shares information about the current testing status (duplicating the schedule

    information sent in e-mail), the current application list for testing, known issues with the

    applications being tested, and any supporting documentation for the applications and

    testing process that may be pertinent. The Web site is an effective archive of past

    communications and current information.

    ; Schedule. The testing schedule itself is an effective form of communication among all

    members of the projectfrom testers to stakeholders. Keeping the testing schedule

    updated and communicating any changes to the entire team sets expectations for all

    parties and lets them coordinate with their own internal test schedules. ; Testing instructions. If there are specific scenarios to be tested in a particular test pass,

    Microsoft IT will communicate these instructions both during the scheduling and in

    specific e-mail communications at the beginning of the test pass.

    ; Daily and final status. Sharing the status of the LOB testing on a daily basis helps to

    identify areas of concern in the process. If the daily status indicates that an application is

    failing the test pass, the test pass can be canceled for that application until the product

    team can improve the build stability. The final status is the core piece of the project

    postmortem; it is a chance to list all results, successes, and elements to be improved for

    the next round of testing.

    LOB Application Compatibility Page 10

Report this document

For any questions or suggestions please email
cust-service@docsford.com