DOC

Performance Measures Audit Program (doc)

By Katie Thomas,2014-11-12 10:14
6 views 0
Performance Measures Audit Program (doc)

    Agency X - Internal Audit Section Performance Measures Data Integrity Audit/Review #XX-XXX

    Audit/Review Program ‘Template’

    Year XX

    Budget Completed W/P

    Hours Procedures by/Date Ref.

     Scope

     This audit will review (Key or Internal) Performance Measures reported data for FY XXXX.

     Objectives 1. Meet the requirement of OAR 125-700-0050.

    2. Determine accuracy/reliability of the data reported in the agencies (Key or

    Internal) Performance Measures report for FY XXXX.

    3. Help ensure Management that the agency’s performance measures are:

    ; Documented appropriate information behind the measures exists;

    ; Repeatable can be accurately re-created; and

    ; Consistent reported the same year to year.

     General Audit Steps

     1. During audit work be aware of management accomplishments and

    report as necessary.

    2. During audit work be aware of potential areas for fraud and report

    to management as necessary.

    3. Discuss the areas of greatest risk and exposure with management. Also, include suggested objectives and test procedures. Discuss with Management.

    4. Document and perform any additional audit steps deemed

    necessary.

     Fieldwork

     1. Conduct an entrance conference with the Performance Measure

    Coordinator and appropriate members of management.

    2. Obtain and review appropriate background information

    including:

    a) pertinent laws, rules, regulations and performance

    measure reporting guidelines

    b) Agency strategic plan

    c) Any previous audit reports of agency’s performance

    measures

    3. Obtain and review documentation of key performance

    measures to determine completeness. Each measure’s

    documentation should include:

    a) Definition of all components of the measure

    b) Identification of the source of the data

    c) Statement of all calculations involved in creating the final

    Page 1 of 4

    Budget Completed W/P

    Hours Procedures by/Date Ref.

    number

    d) Methodology used for calculations

    e) Definition of the timelines of data gathering and the frequency

    of reporting

    f) Reports include current performance data, trends, targets and/or

    required levels of performance and dates of performance

    4. Interview the Performance Measure Coordinator, data owners, and other pertinent personnel as necessary to gain an understanding of

    performance measurement processes. Request and review any

    available documentation of the performance measure reporting process including procedures, desk manuals of appropriate staff, or flowcharts.

    5. Determine which key performance measures to test using the

    following information:

    a) Risk assessment results

    b) Measures that represent the major business lines and

    their products and services, quality elements, outcomes

    and those of significant customer interest or input

    6. Identify testing methodology and sample plan for each data set

    to be tested.

    ; Determine number of reports to include in testing per each

    measure depending on length of reporting period. Include a

    reasonable represented sample for each measure

    recommended use at least 2 reporting periods for each measure

    (For example, if a measure is based on a population survey

    conducted every 2 years, you may include the last 4 reports in

    testing.) Sufficient data for each measure must be reviewed to

    allow a conclusion to be drawn.

    ; Review each measure’s definition and calculation for accuracy

    and determine the degree of deviation for each, if any.

    ; For each measure, determine how the data is kept and identify

    the source of documentation review manual (paper files) or

    automated (computer application or spreadsheet) system.

    7. Determine if adequate controls are in place for each measure being tested. Begin this review at the point of incoming information and end at the point when calculated performance information is reported.

    ; For a manual system: Determine if documentation exists to

    outline how and when performance information is first

    recorded, and for collecting and calculating the information.

    There should be consistencies in staff understanding and

    application of the procedures and staff should understand the

    source of the information and be familiar with any

    modifications. Are regular reviews taking place to ensure

    consistency?

    ; For an automated system: Ensure staff performing data entry

    have been trained and have access to guidelines and procedures.

    Page 2 of 4

    Budget Completed W/P

    Hours Procedures by/Date Ref.

    Perform testing to ensure the computer program being utilized

    is obtaining the correct data and making the correct

    calculations. System data, summary information and measure

    calculations should be reviewed for correctness on a regular

    cycle.

    9. Determine if the performance data can be recreated. Obtain

    performance measure reports and all supporting raw data for

    measures selected for review. Using the documentation for each

    measure, determine the method used to collect and calculate the

    performance data. Using the raw data, recalculate all of the

    performance measure information based on the defined calculations

    in the documentation for each of the measures. Document any

    miscalculations in the performance measure reports.

    10. Request a list of all data elements included in the performance

    measures to be tested or create a list based on the performance

    measure documentation already obtained. Verify with appropriate

    staff that the list is correct and all-inclusive. Obtain data to match

    the data elements.

    11. Test the source documentation for accuracy. Verify that

    performance measure documentation is complete and accurate. The

    definition should be adequate to minimize any misunderstanding.

    The data elements should mirror the data element list. The directions

    for the calculation of each performance measure should support the

    performance reported. The data sources(s) listed should be complete

    and accurate.

    12. Review testing results and conclude as to each performance

    measure’s verification category:

    ; Verified: The performance reported is consistently accurate within

    plus or minus five percent and adequate controls are in place to

    ensure consistency and accuracy in collection of all supporting data

    and subsequent reports.

    ; Verified with Qualifications: The performance reported is

    consistently accurate within plus or minus five percent, but adequate

    controls are not in place to ensure continued accuracy. The span of

    data is less than ideal or the performance measure definition is not

    followed but the calculation remained within the five percent error

    range.

    ; Factors Prevented Verification: When documentation is not

    available and controls are not adequate to ensure consistency and

    accuracy or the performance measure definition is not followed and

    the correct measure result cannot be determined.

    ; Inaccurate: The performance reported is commonly not within five

    percent of actual performance.

    ; Not Applicable: Lack of adequate data exists for review for a

    justifiable reason, i.e. a new measure.

    13. Review methodology behind measure calculations. Determine

    data type and verify appropriateness of methodology employed

    (reference guide).

    Page 3 of 4

    Budget Completed W/P

    Hours Procedures by/Date Ref.

    14. Prepare findings for results in the Verified with Qualifications,

    Factors Prevented Verification and Inaccurate categories.

    15. Conclude on the results including listing strengths and opportunities

    for process improvement in performance measure reporting

    processes regarding documentation, reports, calculations, data

    handling and verification category.

     Reporting

     16. Complete and index working papers.

    17. Prepare a preliminary draft of the report and forward to CAE,

    Performance Measure Coordinator, and appropriate Management for

    review.

    18. Have report and supporting work papers reviewed and clear any

    review notes.

    19. Hold exit conference and provide draft to agency; request response

    and action plan to implement recommendations, if appropriate.

    20. Incorporate response into the final report.

    21. Distribute final report to management; present to Internal Audit

    Committee; and submit summary to the Department of

    Administrative Services Internal Audit Section and Legislative

    Fiscal Office.

    22. Complete quality control worksheet.

    Page 4 of 4

Report this document

For any questions or suggestions please email
cust-service@docsford.com