DOC

FEASIBILITY STUDY OF SOFTWARE ACTIVITIES AT VILSPA

By Katherine Payne,2014-05-17 12:05
12 views 0
No ESA Study Contract Report will be accepted unless this sheet is inserted at theThe study has analysed the feasibility of an Independent Software

FEASIBILITY STUDY OF SOFTWARE

    ACTIVITIES AT VILSPA

    Executive Summary

    ESA Contract Number Nº 12692/97/NL/PA(SC)

    Ingeniería y Servicios Aeroespaciales, S.A. (INSA).- Spain

The work described in this report was done under ESA contract. Responsibility for the contents

    resides in the author or organisation that prepared it.

    ESA STUDY CONTRACT REPORT

    No ESA Study Contract Report will be accepted unless this sheet is inserted at the beginning of each volume of the Report

    erESA CONTRACT N: SUBJECT: Feasibility Study of NAME OF CONTRACTOR: INSA, 12692/97/NL/PA(SC) Software Activities at VILSPA Ingenieria y Servicios Aerospaciales, S.A. ESA CR () Nº: STAR CODE: Nº of volumes: 1 CONTRACTORS’s REFERENCE: This volume is Nº:

    ABSTRACT:

    The study has analysed the feasibility of an Independent Software Verification and Validation Activity Center

    located at the VILSPA in Spain.

    The results are documented in four Technical Notes and a Business Plan that cover the Technical Concept, the

    Marketing aspects, and the Operations and Financial aspects.

    An ISVV activity of this nature is oriented at providing mission assurance at a reasonable cost through combined

    Product and Process Assessment techniques.

    The design of the ISVVA process at VILSPA is based on the idea that by consolidating Verification and

    Validation activities across several similar programs the experience and lessons learned in detection and

    elimination of software errors and their sources is optimised.

    The services are performed on behalf and under contract with the acquirer of a system, primarily agencies like

    for example ESA, or a service provider company such as Intelsat, EUMETSAT, or whatever other agency

    agreed. Initially we refer to services for the space-related domain of technologies, but also to non-Space related system software, such as civil aviation, Air Traffic Management, Medical Services, Institutional Software, etc.

    Services may comprise:

     Program Oversight Independent SW Verification and Validation

     Supplier Independent Verification and Validation

     Product Independent Certification with emphasis on Integrity

     Production Unit Process Assessment and Verification

     Consultancy and Miscellaneous Services

    The Center will be capable of financial self sustaining provided there is a sufficient initial flow of projects.

    Although there is no guarantee of such flow, all the indications are that if some coordination with European

    Administrations and Agencies is achieved, it should be possible to have a secured forecast at the end of the

    second year of operation. For the first two years the set up and Launch phase of the Center, including the

    investment in material fixed assets and non-material assets, primarily R&D activities, have an expected

    requirement for support funds that has been identified.

    The Center is viable in the VILSPA facility since the infrastructure is available to locate it, and the surrounding

    conditions are generally adequate. The technology is well defined and available to ESA. The commercial market is a mildly attractive one, so strong commercial competition is not to be expected at least

    in the high integrity segment, for several years.

    The strategy of the Center is based on the unique opportunity of being assigned the responsibility of conducting at the Center Oversight ISVVA for ESA Programs, and the capability to attract the support of the European

    Commission in R&D Programs. Adequate coordination with other “customers”, like European Administrations

    and participation as Supplier ISVV agent in Private Partnership Programs will allow a sustained activity during

    the development and running phases.

    The Center is set to be the reference for Verification/Validation/Certification of high integrity

    software in Europe.

    The work described in this report was done under ESA contract. Responsibility for the contents resides in the author or organisation that prepare it.

    NAMES OF AUTHORS: J.A. Ruiz, A. Sanchez. M. Aznar, M.A. Garcia Moreno, M. Bermudez, C. Martin

    NAME OF ESA STUDY MANAGER: ESA BUDGET HEADING: P. RODRIGUEZ, ESTEC

     Page 2

    TABLE OF CONTENTS

    EXECUTIVE SUMMARY ........................................................................................................................ 4 1 SCOPE ............................................................................................................................................... 4 2 BACKGROUND ............................................................................................................................... 4 2.1 VILSPA HISTORICAL BACKGROUND .............................................................................................. 4

    2.2 TECHNICAL BACKGROUND .............................................................................................................. 5

    3 SUMMARY OF THE STUDY FINDINGS .................................................................................... 6

    3.1 CONCLUSIONS REGARDING THE SERVICES ...................................................................................... 6

    3.2 CONCLUSIONS ON THE CENTER ACTIVITY ...................................................................................... 7

    3.3 ORGANISATION APPROACH ........................................................................................................... 11

    3.4 CONCLUSIONS REGARDING THE MARKET ANALYSIS ...................................................................... 11

    3.5 CONCLUSIONS ON MARKETING STRATEGY .................................................................................... 12

    3.6 CONCLUSIONS ON FINANCIAL PLAN .............................................................................................. 12

    4 STUDY CONCLUSION ................................................................................................................. 13

     Page 3

    Executive Summary

    1 Scope

    This is the Executive Summary in the Contract nº 12692/97/NL/PA(SC),

    Feasibility Study of Software Activities at VILSPA. The purpose of this document is to summarise the study on the Independent Software Verification and Validation Activity, ISVVA, at VILSPA.

    The term ISVVA refers to an Activity, an Administrative Unit with a specific objective, a set of resources, and an institutionalised interface with other elements within the ESA organisation. The ISVVA can be identified with a physical Centre, although it may be at a later stage a network over several different locations. On the other hand Independent Software Verification and Validation, ISVV, refers to a technical action or process that takes place as one element of the Activity.

    2 Background

    2.1 VILSPA Historical Background

    VILSPA, the Villafranca del Castillo Station in Spain, was initiated in 1975, after an international agreement between the European Space Agency, ESA, and the Spanish government, VILSPA is part of the worldwide European Space

    Operations Centre (ESOC) Station Network (ESTRACK).

    In the last twenty years VILSPA has supported operationally many ESA and International Programs.

    However the level of activity that was attained in the 95-98 timeframe for Mission Control purposes would not be attained again with the scheduled missions and the level of autonomy envisaged in the future. Therefore there is a margin for availability of installations and site capability that could be put to service in other fields.

    One such field of activity is the one discussed in this report, namely the Independent Software Verification and Validation Activity. This is a possible future Operating Unit of the European Space Agency, mostly contractor operated, and specialised in the performance of activities related to software assurance, primarily but not exclusively, of mission critical software for client programs within and outside ESA.

     Page 4

    The location of that activity on the VILSPA site can be the IDT Building, subject to the evaluation of ESA incumbent authorities.

    It can host comfortably 20 personnel, and up to 30, and is ideally suited since it is independent, with all services available and can be controlled efficiently for security purposes.

    The tasks of the ISVVA Center are or have been performed in the framework of the European Space Agency usual engineering activities. The identification of a separate ISVVA Operating Unit with the potential for external business has not been documented up to now. This type of formal organisations on the side of the acquiring Agencies are relatively new but experiences are available with the military aviation technical services and with NASA.

    2.2 Technical background

    The contribution of software to the functions in an Aerospace system is increasing rapidly. Just as an example the software load on the B-777 is estimated at 2 million ADA lines of code, that is around 20 times the volume of a B-747-300. Furthermore critical functions have been allocated to software subsystems making the consequences of possible failures much more severe. The complexity of the on board software has also increased substantially.

    On the other hand and despite tremendous advances, software engineering is still a relatively immature discipline, as shown by the striking record of failures in software projects and in software intensive systems.

    As a result a significant focus has been given in the last few years to improve the standardisation environment that regulates the software production process, that has resulted in more stringent requirements regarding the software quality characteristics and their evaluation and certification. Best Practice Models have evolved, that establish Key Process Areas or required activities for the production units, to which the degree of adherence can be measured and assessed The various contract elements that govern today the acquirer-supplier relationship establish the characteristics of the software product to be supplied and also the requirements on the SW generation process. That includes the set of goals, policies, abilities, activities, measurements and verifications that the production unit has to deploy in order to perform the contract. Both requirements have to be verified by qualified agents.

    The verification and validation itself has become an identified and separated activity that has to be performed formally by a designated group, with responsibilities and techniques that depend fundamentally on the criticality of the software produced.

     Page 5

    3 Summary of the study findings

    3.1 Conclusions regarding the services

    An ISVV activity of this nature is oriented at providing mission assurance at a

    reasonable cost.

    The design of the ISVVA process at VILSPA is based on the idea that by

    consolidating Verification and Validation activities across several similar

    programs the experience and lessons learned in detection and elimination of

    software errors and their sources is optimised. Several factors point along this

    direction:

    ? The ISVV activity is largely composed of system and software engineering,

    with specific emphasis on software error and failure detection and avoidance.

    Experience is essential: errors that are incurred, detected and identified as

    associated with a faulty process will be avoided, by appropriate process

    improvements, or immediately detected in a second occurrence. Useful error

    avoidance directions should eventually reach industry, after appropriate

    filtering, sometimes in the form of required practices, standards or other

    directive documents. In this approach cost per error detection and total cost of

    ISVV are minimised structurally.

    ? The problem detection and resolution may be facilitated by the security

    offered by a non-competitive, independent, and stable agent. This should

    improve the error database and the management of the experience and

    therefore enhance the cost reduction above.

    ? Emphasis should be given within the ISVV to early analysis activities. This

    enhances integrity, and as an independent result, reduces actual software cost

    per function point since fixing errors is estimated at 40% of program actual

    costs.

    ? The potential should be developed for lightweight Verification and Validation

    techniques that could be applied to smaller programs at affordable cost and

    with significant integrity enhancement.

    ? Verification and Validation has to be seen also as an enabling technology to

    COTS introduction, especially in Ground Segment software, but eventually

    also in onboard software.

    ? The ISVVA should concentrate in improving suitably defined software error

    metrics for Aerospace programs initially, and then, as the domain engineering

    capabilities are obtained, into other fields.

    ? We highlight a distinction between Oversight Independent Verification and

    Validation that is a responsibility of the acquirer of a system, and the

    Verification and Validation that is responsibility of the supplier and can be

    Independent or not.

     Page 6

The Oversight ISVV function is initiated very early in the development

    process, and it is intended to confirm or certify the adequacy of finished

    intermediate or final products to preclude mission loss, in all possible

    circumstances, rather than to eliminate errors or generate detail evidence of

    the compliance with requirements of the specification.

    The Oversight Independent Verification and Validation offered by the Center

    will complement the Contractor Verification and Validation process. This is

    an alternate approach to a Supplier Independent Verification and Validation

    effort contracted by a Prime Contractor, on behalf of the Agency, to a third

    party other than the Prime itself.

    It is an advantageous alternative for various reasons.

    ? It is structurally more efficient since the ultimate prospect for the final

    customer of this activity is to reduce it to a minimum by improving the

    production processes and this interest is only partially shared by a

    specialised ISVV contractor.

    ? It is technically efficient since it accumulates experience faster than any

    other approach particularly regarding the Mission assurance aspects, by

    building on a strong database of critical errors and defects

    ? It is focused on avoiding mission loss and leaves the responsibility of

    meeting the specification with the contractor

    ? It reduces the cost of controlling software associated risks by balancing

    them, across the various programs within ESA, instead of isolating them at

    each program level.

    So it is recommended that at contract time, in agreement with the prevailing

    standards, the ESA or other Agencies do not include in the effort requested to

    the Prime an ISVV Package but rather turn it into an Oversight Independent

    Verification and Validation effort retained at ESA through this Center.

    3.2 Conclusions on the Center Activity

    We reach the following conclusions regarding the activity of the Center:

    ? The requirement for Supplier Independent Verification and Validation of

    Software as such, is restricted to highly critical software.

    ? The services potentially offered by the Center are:

    ? Provision of Oversight Verification and Validation Services

    The component processes of this Process are:

    ? Verification

    ? Verification Planning and Implementation

    ? Contract Verification

    ? Process Assessment

     Page 7

    ? Requirements Verification

    ? Design Verification

    ? Coding Verification

    ? Integration Verification ? Validation

    ? Validation Planning and Implementation

    ? Validation Execution ? Joint Technical Reviews

    ? Physical Audits

    ? Problem Resolution

    The Oversight ISVV Process is a combination of these

    processes and the corresponding methods designed to

    emphasise:

    ? Risk assessment and management

    ? Verification of Safety or Reliability Critical

    functions

    ? Validation under Boundary/marginal/stress and

    beyond specified conditions

    ? Quantification of failure probability

    ? Reduced cost with respect to a supplier

    independent verification and validation process The process should make use of contractor generated

    documentation regarding static and dynamic analysis,

    particularly with respect to coverage. It will produce

    independent analysis whenever critical functions are

    concerned and new test cases and results as required,

    particularly in search of potential failures or when testing

    dependability and fault tolerance issues. The process

    leverages on error guessing and failure statistics derived

    from its updated error and failure database.

    ? Provision of Supplier/Prime Contractor level ISSVA Service.

    The components of the process are as above, but in this

    case the emphasis is totally oriented at giving proof of

    performance. So it entails thorough verification and

    validation of all the functions in order to determine

    compliance with specification and fitness for the purpose.

    Although the technology involved is similar, the purpose

    is quite different to the above services since it is a Supplier

    Process instead of an Acquirer Process. It requires, for

     Page 8

    example, having a static and dynamic analysis of all the

    software in the system, with the specified coverage metrics.

    A plausible situation for performing this service at the

    VILSPA ISVVA Center is when ESA is partnering with

    industry in a program that could be, as previously

    indicated, a launcher future development, EarthWatch or

    other shared risk program. In this case ESA is contributing

    financially, technically, and endorsing the safety and the

    reliability of the system, and is ideally suited for the task.

    ? Production Process Assessment to qualify for a specific ESA

    or Non-ESA related work.

    Such service provides an assessment of the suitability of a

    potential contractor production process for a certain job. It

    is here as a service provided for example to the potential

    contractors seeking to bid successfully for work for an

    acquirer, for ESA Programs or for a Prime Contractor for

    which the Center might or may not be involved in ISVV.

    ? Certification of Products

    This Process is to certify at the request of the developer or

    a third party that a product does comply with a

    specification established by the developer itself or by

    another authority. This certificate may be, for example, for

    tools that have to be used in the construction of SW pieces

    that have to undergo a Verification and Validation Process.

    It may also become of great value, in the very next future,

    when certain component of the SW to be validated have

    undergone themselves a certification, resulting in lower

    effort accepted in a regulated ISVV Process. The role is,

    for example, as described for the independent Integrity

    Assurance Authority in ISO/IEC DIS 15026.

    ? Consultancy Services

    This service is to support primarily user organisations and

    administrations to organise the ISVV activities, establish

    co-operation links, and participate in common projects.

    ? Oversight ISVV is particularly useful when associated with a multilevel

    contractor situation like in ESA space programs. It should include the

    Independent Production Process Assessment or Verification.

    ? There is a potential requirement for Oversight Verification and Validation

    other than in Space and Aerospace sectors. However this remains an

    emerging requirement at this stage.

    ? The ISVV effort contains a very significant system analysis component. This

    component can only be credibly provided when domain specific system

    expertise is available to the ISVV provider. At the same time a strong

     Page 9

software expertise is necessary that is often lacking in most institutional

    oversight authorities.

    ? The technology required to establish an ISVVA Center is already available at

    ESA. Advances in the technology will come by progressively formalising the

    requirements, by further structuring system domain engineering and by

    enhancing software reusability by Object standardisation and the Verification

    and Validation techniques related to all of these aspects. The Center should

    play a major role in Research and Development associated with ISVV.

    In terms of cost:

    ? The cost of a Supplier Independent Verification and Validation program

    depends very much on how the extent of the effort is defined. There is quite

    some risk for an ISVV contractor in firm fixed pricing an individual program,

    because there may be many modifications and test repeats and many

    unknowns. This risk on a specific program is passed to the acquirer of the

    service, which has to pay for the sum of the risks in all the programs, instead

    of paying for the average risk in a consolidated activity.

    ? The cost of a practical individual program oversight action that is focused on

    mission critical software integrity should be estimated from 3 to 10% of the

    mission software development cost, depending on the specific contents of the

    ISVVA Plan that is adopted.

    ? A minimal effort could include Requirements analysis, Review

    support and Validation testing.

    ? A more balanced program could include additionally the full design

    verification, plus verification testing of critical functions. Programs

    which could allow larger cost contributions may be performed with

    more intensive testing activity and broader analysis, but often this is

    not practical as an oversight action.

    ? Instead small programs would have to be treated in a special manner

    to cope with minimum activity cost threshold.

    ? Certification of software products based in the use of well-structured

    software development documentation, and including validation testing,

    should cost in the low side of this range, so around the 3%.

    ? Highly critical thorough ISVV would be in the high side of the range.

    ? Costs would depend on how well implemented is the Center database

    and tools.

    ? The cost of the initial investment in a Center for ISVVA is not negligible.

    Important costs are involved in training, certification, new tools and tool

    integration and pilot program execution. Likewise there is a minimum time

    required to start an initial ISVV project that is evaluated at 4-6 month.

     Page 10

Report this document

For any questions or suggestions please email
cust-service@docsford.com