Introduction
This guidance provides recommendations for investigation and reporting of test deviations during a validation exercise.
Out-of-specification (OOS) results and any other deviations that may impact the acceptability of the qualification/validation should be documented, investigated, root cause determined, corrective action taken and reported. Several examples are included.
Recommendations & Rationale
Deviations that occur during validation testing must be documented and investigated in accordance with site procedures. Additionally, a summary of all deviations, investigations or corrective actions associated with a specific validation activity need to be included, discussed, and cross-referenced in the validation report.
The documentation and investigation of a deviation and its resolution should address the impact of the deviation on the acceptability of the validation of the process or system in question. Typically, site deviation procedures are targeted towards the impact of the deviation on commercial product and may not be designed to address the specific issues that may arise during validation of a system or process. If the existing site deviation handling systems do not readily support validation deviations, it is recommended that deviations that occur during qualification/validation activities are documented separately.
The procedure for documenting validation deviations can be established in the Validation Master Plan, validation SOP or within the standard test document template. Deviation forms (see attached example in Appendix 1) can be included as part of the testing documentation.
Regardless of the system used to document a validation deviation, the principles remain the same and are summarized in the flowchart below:
Identification of deviation
Deviations may be identified by the tester during execution of the protocol or by a reviewer.
Where a protocol error is identified, it is recommended that the deviation procedure makes a clear distinction between typographical and other minor errors where the intent of the protocol is still clear, and errors that require correction to allow the test to be executed. It is considered acceptable to hand-amend (per applicable site documentation practices) typographical and other minor errors that have no impact on the test method or acceptance criteria. Where correction is required to allow the test to be executed, including determining whether the test passes or fails, then a deviation should be raised.
Document the deviation
The deviation should be documented according to the applicable procedure or protocol.
This should include assignment of a reference number to the deviation, the test section (and run number, where applicable), the test step (where applicable), a description of the deviation and the signature and date of the person recording the deviation.
Although errors may be grouped by type, repetitive protocol errors are the most practically grouped in this way. When deviations are grouped, care should be taken to ensure that the link between each individual issue identified, its investigation, and corrective actions is clear. In addition, all issues will need to be resolved (or transferred to a tracking system) to allow closure of the deviation.
Investigation
An investigation should be conducted to determine the root cause of the deviation. In many cases, this will not require any in-depth analysis. However, for system or process failures, a formal investigation including appropriate technical representatives may be required. Such a detailed investigation should be conducted according to applicable procedures.
The investigation may be used to identify the type of deviation; this can be documented descriptively on the form or by using check boxes. Examples of types of deviations could include:
- System/Process Error –An actual problem with the functionality of the system or a technical problem with the process, whereby the system or process does not meet the acceptance criteria when the test step is executed.
- Protocol Error –The test step instructions require addition, deletion, modification and/or clarification due to missing/incorrect/ambiguous test instructions. Or the acceptance criteria require addition, deletion, modification and/or clarification due to missing/incorrect /ambiguous acceptance criteria.
- Other –A deviation occurs that is not related to the test method, acceptance criteria or system, for example, operator error.
Protocol errors that require correction in order to allow the test to be executed as intended may include missing test instructions or incorrect acceptance criteria. In these cases, the investigation should include clear justification for the corrections, with references to any source documentation. Where a system or process error is identified as the root cause, the investigation should include an analysis of whether the system is acceptable without change.
Such a conclusion should be approved by appropriate technical and QA representatives, in addition to the system/process owner.
Identification of corrective actions
Once the investigation is complete, appropriate actions should be identified, if any. These should include both corrective actions and any preventive actions required to prevent a recurrence of the deviation.
Examples of corrective actions include:
- System/process documentation update – where the investigation identifies that the original system/process documentation from which the protocol acceptance criteria were derived was incorrect, or the process/system does not meet the acceptance criteria but is considered acceptable by appropriate technical and QA representatives, then the system/process documentation should be updated. Where applicable, documentation update should be completed according to the relevant change management procedure.
- Process/system changes – where the deviation investigation concludes that a change is required to the system or process, then this should be documented and controlled through the relevant change management process. Reference to the change management documentation should be made on the deviation form.
Repeat testing – where a system/process change has been completed to correct a fault, or where an operator error resulted in a test not being executed correctly, repeat testing is generally required. The level of testing to be repeated should be clearly identified and may range from the addition of another batch to a process validation study to the repeat of a single test step during system validation. In some cases, a new protocol maybe required and restarting of the validation activities may occur.
Assessment of impact on validation activities
Where the deviation and any corrective actions do not impact the intent of the original validation test, then testing should be allowed to proceed. If corrective actions identified impact ongoing validation activities (including moving to the next phase of validation), validation testing should be stopped, the actions should be implemented and confirmed and the deviation should be closed before continuing testing. Where testing that has already been completed is impacted, consideration of the repetition of those tests should be included in the corrective actions.
Retrospective and concurrent validation studies
Where a deviation occurs during a retrospective or concurrent validation study, the potential impact of the deviation on commercial product should also be evaluated. If there is a potential impact, then the impact on product should be managed through the applicable Deviation Reporting system.
Review and approval of investigation and corrective actions
Quality Team must review and approve deviation investigations, summaries, and corrective action plans prior to (or in parallel with) the approval of the associated completed validation study. It is recommended for this approval to take place prior to the corrective action implementation. However, approval may take place after implementation. Any corrective action plans not completed at time of approval of the validation study completion should be placed in a CAPA system so that they may be tracked to closure.
Implementation of corrective actions
The approved corrective action plan should be implemented as described on the deviation form. Documented evidence that each action has been completed should be attached to or referenced from the deviation form. Any repeat testing performed should be discussed and justified in the validation report.
Documentation of repeat testing
The level of documentation of repeat testing required will depend on the nature of both the validation exercise and the scope of re-testing. The inclusion of an additional batch in a process validation study may need only to be included in the collation of data in the final report. Re-testing of a system function will typically require a new test script. Where the results of the re-test need to be documented in a test script format, this may be created in several ways:
1. The test script may be created as an attachment to the deviation and approved as part of the corrective action plan.
2. A controlled copy of the relevant pages from the original approved protocol may be generated and annotated with the deviation reference and test run number.
3. A new or supplemental protocol may be generated and approved by the system owner and QA. It should be noted that where the test script created as a result of a deviation includes revised acceptance criteria and/or test methodology, the same functional approvals required for the initial protocol approval are applicable to the revised test script.
Deviation Closure
Once all actions have been completed and confirmed on the deviation form, closure of all deviations should be reviewed and approved by the appropriate QA representative prior to approval of the validation study. This approval may be by signing each deviation form, or through the final approval of the completed validation study. The approach taken should be documented. When a corrective action is not complete and has no impact on the release of a system or process for the next step in validation or for use, it is acceptable to use a follow-up system (for example, CAPA) to assure that the corrective action is completed and document the appropriate reference on the deviation form and in the Validation Report
Examples of Deviations
An out-of-specification (OOS) result generated by the laboratory is a deviation and should be investigated according to applicable procedure and procedures for laboratory OOS results. Another common deviation is missing or lost data. The impact of the missing or lost data should be evaluated to determine the criticality of the unavailable information and its impact on providing evidence of validation or qualification of the system or process.
Example 1
Deviation – Product temperatures are recorded during the validation run using 10 thermocouples placed in the product, evenly distributed throughout the load and linked to a data logger. At the completion of the cycle, it is discovered that one of the thermocouples has fallen out of the product container it had been placed in.
Investigation & corrective actions – The other 9 thermocouples have provided valid data. The location of the thermocouple that fell out of the product container is assessed to determine if this location is a ‘worst-case’ location within the load. If it is a worst-case location, then repetition of the study may be warranted. If not, then it may be possible to write a rationale that explains why the missing data point does not negate the validation.
Example 2
Deviation – In the process validation of a solid oral dosage product, 3 samples are to be taken from each shift (beginning, middle and end) during compression. The compression step for the proposed batch size takes on average 8 shifts to complete. During validation batch #2, the samples are forgotten during the night shift.
Investigation & corrective actions – A temporary corrective action is put in place to ensure that all samples from all shifts are collected for the remainder of validation batch #2 and #3. It is determined that a sufficient amount of data has been obtained to allow extrapolation of the data to cover the 3 missing samples from validation batch #2, particularly since samples were neither the start nor the end of the batch. Had the missing samples been from a potentially critical stage in the compression the deviation may have required a 4th batch to replace the missing samples. Note: -the principles of this example may also be applied to the taking of homogeneity samples for an API.
Example 3
Deviation – During system qualification for a computerized warehouse management system (WMS), the test to confirm that the WMS could receive returned goods failed.
Investigation & corrective actions – The investigation determined that the root cause was because material being returned is automatically given a Quarantine status and is received into a (virtual) adjustment warehouse in the WMS. Inventory from the same lot already existed in the adjustment warehouse, but with Approved status. As the system requirement is that a lot cannot exist with two different statuses in the same warehouse, the system functionality was correct. The test was re-executed, ensuring during the set-up that no inventory remained for that lot in the adjustment warehouse.
Example 4
Deviation – During system qualification of a thermoforming machine, the sealing plate temperature profile did not meet specification.
Investigation & corrective actions – Following the investigation, it was determined that the equipment installation was incorrect to achieve the required functionality. Under site change management procedures, the standoffs on top of the upper sealing tool were changed to stainless steel. The test was re-executed and met the specification.