DOC

NV-DOC

By Evelyn Hamilton,2014-12-23 20:42
7 views 0
NV-DOC

     NordVal

     c/o Department of Microbial Food Safety

     Danish Institute for Food and Veterinary Research

     Mørkhøj Bygade 19

     DK-2860 Søborg

     Denmark

    NV-DOC. D 2004-01-01

Protocol for the validation of alternative

    microbiological methods

Contents

     Page

    Scope 2

    Definitions 2

    Phases of validation 5

    Test characteristics 5

    Qualitative methods 5

     Method comparison study 5

     Collaboratory study 10

    Quantitative methods 13

     Method comparison study 13

     Collaboratory study 14

    References 15

Chairman: Sven Qvist Tel: +45 72347031 Fax: 45 72347698 E-mail: sq@dfvf.dk

     2

Scope

    This NordVal protocol describes the technical procedures for validation of alternative methods for microbiological analyses of food, water, animal faeces, feed, and food environmental samples in the Nordic countries.

Definitions

General terms:

Alternative method

    An alternative method is a method of analysis that demonstrates or estimates, for a given category of products, the same analyte as is measured using the corresponding reference method.

Analyte

    The analyte is the component demonstrated or measured by the method of analysis. It may be the microorganism, its components or products.

Collaboratory study

    Study of the performance of the alternative method using common samples in several laboratories and controlled by the expert laboratory.

Expert laboratory

    Laboratory having qualified staff and skills to perform the method comparison study and organise the collaborative study. The availability of an experienced statistician is essential for the analysis of the results.

Method comparison study

    Study performed by the expert laboratory of the alternative method against the reference method.

Negative deviation

    The alternative method presents a negative deviation if it gives a negative result when the reference method gives a positive result.

Positive deviation

    The alternative method presents a positive deviation if it gives a positive result when the reference method gives a negative result.

Qualitative method

    A qualitative method is a method of analysis whose response is either the presence or absence of the analyte in a certain amount of sample.

     3

    Quantitative method

    A quantitative method is a method of analysis whose response is the amount of the analyte measured either directly or indirectly in a certain amount of sample.

Relative accuracy

    The relative accuracy is the degree of correspondence between the response obtained by the alternative method and the reference method on artificially inoculated samples.

Relative detection level

    Documentation must be provided that the relative detection level of the alternative method is equal to that of the reference method.

Relative selectivity

    The relative selectivity is a measure of a) the inclusivity: detection of the target microorganism from a wide range of strains, and b) the exclusivity: the lack of interference from a relevant range of non-target microorganisms.

Relative sensitivity

    The relative sensitivity is the ability of the alternative method to detect the analyte compared to the reference method.

Relative specificity

    The relative specificity is the ability of the alternative method not to detect the target microorganism when it is not detected by the reference method.

Reference method

    A reference method is a method which is internationally recognised and accepted (e.g. NMKL, ISO, CEN and AOAC methods, and certain national standards of equivalent standing).

Validation of an alternative method

    The validation of an alternative method is the procedure to demonstrate if the results obtained by the alternative method are comparable to those obtained using the reference methods.

Statistical terms:

Outlier

    An outlier is an extreme value which normally appears randomly in less than 1% of tests, but more frequently, if abnormal situations occur. Statistical test procedures can be used to quantify this probability.

     4

    Relative standard deviation(RSD)

    The relative standard deviation is a measure of the precision of the method by demonstrating the RSD for repeatability and reproducibility.

Repeatability

    The repeatability is the closeness of agreement between successive and independent results obtained by the same method on identical test material under the same conditions ( apparatus, operator, laboratory and short intervals of time ).

Repeatability limit (r)

    The repeatability limit is the value less than or equal to which the absolute difference between two tests results obtained under repeatability conditions is expected to be with a probability of 95%.

     Note: If the difference between 2 results exceeds r, the results should be considered as suspect.

Reproducibility

    The reproducibility is the closeness of agreement between single test results on identical test material using the same method and obtained by operators in different laboratories using different equipment.

Reproducibility limit (R)

    The reproducibility limit is the value less than or equal to which the absolute difference between two test results obtained under reproducibility conditions is expected to be with a probability of 95%.

    Note: If the difference between 2 results from different laboratories exceeds R, the results must be considered suspect.

     5

    Phases of validation

The validation of qualitative and quantitative methods comprises two phases:

    A: A method comparison study of the alternative method against a reference method

     (performed by an expert laboratory).

    B: A collaboratory study of the alternative method

     (organised by an expert laboratory).

Test characteristics

    The test characteristics for alternative methods are shown in the following table:

     Qualitative methods Quantitative methods

    1. Relative selectivity 1. Relative selectivity

    2. Relative accuracy 2. Relative accuracy

    3. Relative detection level 3. Repeatability

    4. Relative sensitivity 4. Reproducibility

    5. Relative specificity 5. Relative standard deviation

Part 1 Qualitative methods

A. METHOD COMPARISON STUDY

1 - Relative selectivity

    The relative selectivity is a measure of a) the inclusivity: detection of the target microorganism from a wide range of strains, and b) the exclusivity: the lack of interference from a relevant range of non-target microorganisms.

a) inclusivity

    Select 50 pure cultures of target microorganisms relevant to the alternative method and the food matrices.

    From each test strain a growth medium is inoculated with a level of 10 to 100 times greater than the detection level. For both the alternative method and the reference method the complete protocol is used.

Results

     6

    Microorganism Alternative method +/- Reference method +/-

    Strain 1

    Strain 2

    Strain 3

     etc.

    TOTAL

b) exclusivity

    Select 30 pure cultures of non-target microorganisms chosen from both the strains known to cause interference with the target microorganism and from strains naturally present in the food matrices.

    From each test strain a growth medium is inoculated with a level similar to the greatest level of contamination expected to occur in the food matrices. For both the alternative method and the reference method the complete protocol is used.

Results

    Microorganism Alternative method Reference method

    Strain 1

    Strain 2

    Strain 3

     etc.

    TOTAL

    Note: Criteria for the selection of test strains for measurement of selectivity see Annex G in

     ISO 16140 : 2003 - Protocol for the validation of alternative methods.

2 - Relative accuracy

    The relative accuracy is the degree of correspondence between the response obtained by the alternative method and the reference method on identical samples.

    As far as possible naturally contaminated samples should be used. However, in the Nordic countries such samples are seldom available in sufficient numbers for extensive validations.

    Therefore, artificial contamination of samples is permitted. As a general rule three inoculation levels are used: 0= negative control, 1-10 cells per 25 g sample and 10-100 cells per 25 g sample usually 2 strains relevant for the matrix are selected.

    If validation is requested for all food matrices, at least 5 relevant food matrices are selected from the NordVal matrix-lists. Samples from a food matrix should be representative by taking samples from several of the sub-matrices indicated in NV-DOC: F. For other categories of matrices a relevant number of matrices are selected.

     7

    For each matrix at least 60 samples are analysed to give approximately 30 positive and 30 negative results by the reference method. Each sample is analysed once by the reference method and once by the alternative method.

    Identical samples should be used as far as possible. However, when different pre-enrichments are used, duplication of samples is necessary.

    Note: See guidance in Annex D in EN ISO 16140:2003 - Protocol for the validation of alternative methods.

Tabulate the data from the accuracy measurement after screening.

     Responses Reference method Reference method

    positive (+/ ) negative (-/ )

    +/+ positive agreement -/+ positive deviation Alternative method

    (PA) (PD) positive ( /+)

    +/- negative deviation -/- negative agreement Alternative method

    (ND) (NA) negative ( /-)

    PANA100;;Relative accuracy % PANAPDND

Tabulate the data from the accuracy measurement after confirmation.

     Responses Reference method Reference method

    positive (+/ ) negative (-/ )

    +/+ positive agreement -/+ true positives Alternative method

    (PA) (TP) positive ( /+)

    +/- false negatives -/- negative agreement Alternative method

    (FN)) (NA) + false positives (FP) negative ( /-)

    PANAFP100;;Relative accuracy % PANATPFNFP

Positive deviation (PD)

    The alternative method presents a positive deviation if it gives a positive result when the reference

    method gives a negative result.

     8

    A positive deviation must be confirmed. A positive deviation (PD) becomes a false positive (FP) result when the true result can be proven as being negative.

    A positive deviation is considered as a true positive (TP) result when the true result can be proven as being positive.

Negative deviation (ND)

    The alternative method presents a negative deviation if it gives a negative result when the reference method gives a positive result.

    A negative deviation becomes as a false negative (FN) when the true result can be proven as being positive.

    It is generally assumed that the reference method gives the true result. However, experience has shown that poor accuracy can be due to poor performance of either the alternative method or the reference method.

3 -Relative detection level

    Documentation must be provided that the relative detection level of the alternative method is equal to that of the reference method.

4 - Relative sensitivity

    The relative sensitivity is the ability of the alternative method to detect the analyte compared to the reference method.

    The calculation is made by using the data obtained in the accuracy measurement after confirmation.

    PATP100;;Relative sensitivity % PAFN

Interpretation: Generally, values above 95% are considered acceptable.

5 - Relative specificity

    The relative specificity is the ability of the alternative method not to detect the target microorganism when it is not detected by the reference method.

    NA100Relative specificity % NAFP

     9

    A summary of all calculation tests for relative accuracy, relative sensitivity and relative specificity

    should be tabulated as follows:

After screening:

     Relative Relative Relative Matrices PA NA ND PD Sum Accuracy sensitivity specificity

     AC (%) SE (%) SP (%)

     (PA+NA) x 100 PA PA x 100 NA NA x 100

     N PA+NA+PD+ND + PA+ND + PD+NA

    ND PD Food cat. 1

    Food cat. 2

    Food cat. 3

    Food cat. 4

    Food cat. 5

TOTAL

N = PA +NA + PD + ND

After confirmation:

     Relative Relative Relative Matrices PA NA FN TP FP Sum Accuracy sensitivity specificity

     AC (%) SE (%) SP (%)

     (PA+NA+FP)x 100 (PA+TP)x 100 NA x100

     N N PA+FN NA+FP Food cat. 1

    Food cat. 2

    Food cat. 3

    Food cat. 4

    Food cat. 5

TOTAL

N = PA +NA + FN + TP + FP

     10

B. COLLABORATORY STUDY

    The aim of the collaboratory study is to determine the variability of the results obtained by the alternative method in different laboratories using identical samples.

    The expert laboratory shall include the alternative method and the reference method in its study. Results from at least 8 laboratories with valid results must be available for the calculations. (Therefore, it is advisable to select 10-12 labs). In case of very expensive instruments results from 5 laboratories are considered acceptable.

    One relevant food matrix is selected from the NordVal matrix list to prepare the test samples. Artificial inoculation is used at 3 levels (O = negative control, 1-10 cells per 25 g and 10-100 cells per 25 g). Duplicates are prepared for each laboratory to be analysed by the alternative method. If the alternative method comprises more than one protocol a relevant food matrix for each protocol should be selected for the study.

    The expert laboratory shall determine which data from the participating laboratories are suitable for the calculation of the precision data.

    Note: Guidelines for the expert laboratory conducting the collaboratory study see: Annex H and L in EN ISO 16140:2003 Protocol for the validation of alternative methods.

    Calculation.

    Tabulate the data obtained with each method as follows:

    Positive results by the expert laboratory

    Laboratories Contamination level

     LLL0 1 2

    Reference method /2 /2 /2

    Alternative method /2 /2 /2

    L = negative control 0

    L = 1-10 cells per 25 g 1

    L = 10-100 cells per 25 g 2

Report this document

For any questions or suggestions please email
cust-service@docsford.com