Risk-Based Validation of Commercial Off-the-Shelf Computer Systems 2

For lower risk devices, only baseline validation activities may be conducted. As the risk increases, additional validation activities should be added to cover the additional risk.
The "FDA Part 11 Guidance on Scope and Application" states:
We recommend that you base your approach (to implement Part 11 controls, e.g., validation) on a justified and documented risk assessment and a determination of the potential of the system to affect product quality and safety, and record integrity.
The most specific advice for risk-based compliance of computer systems came from the Pharmaceutical Inspection Convention's, "Good Practices for Computerized Systems Used in Regulated Environments" (5). It has several recommendations related to risks: For critical GXP applications, it is essential for the regulated user to define a requirement specification prior to selection and to carry out a properly documented risk analysis for the various system options. This risk-based approach is one way for a firm to demonstrate that it has applied a controlled methodology, to determine the degree of assurance that a computerized system is fit for its intended purpose.
The inspector will consider the potential risks, from the automated system to product/material quality or data integrity, as identified and documented by the regulated user, in order to assess the fitness for purpose of the particular system(s). The business/GXP criticality and risks relating to the application will determine the nature and extent of any assessment of suppliers and software products (5).
Basically, this means the FDA and other agencies expect a risk assessment for each computer system, otherwise full validation is required. Companies without justified risk assessments will not be able to defend their selected level of validation. The real value in a comprehensive risk-based validation approach is in doing exactly the right amount and detail of validation for each system.

Figure 1: Risk vs. validation costs.

The principle is quite clearly illustrated in Figure 1. Costs for validation increase when going from no validation to 100% validation. Full validation for a COTS system would mean, for example, the testing of each function of the software under normal and high load, across and beyond the expected application range, and this for each possible system configuration. In addition, whenever the system is changed, may it be computer hardware, operating system, or application software, full revalidation would require that the same tests be rerun. In today's rapidly changing computer environment, this could possibly mean that the system would be used 100% for testing. At the same time that testing increases, the risk of unexpected system failure decreases, because errors found during testing can be corrected or work-around solutions can be found and implemented.
The optimum testing is, obviously, somewhere between zero and 100%. The range depends on the impact the software or system has on (drug) product quality. For example, a system used in early drug development stages will have a lower impact and require less validation than a system used in pharmaceutical quality control.
In the past, companies frequently have applied the principles of such risk-based validation, but the rationale behind it was not documented and the approach was not implemented consistently within a company. The extent of validation depended more on individual validation professionals than on a structured rationale. As explained earlier, in new guidance, the FDA suggests that industry base the extent of its validations on a 'justified and documented' risk assessment.
Most confusing to the industry has been finding a structured way to prioritize risks. The FDA has been asked frequently to prepare a matrix of regulated processes indicating the level of risk associated with each. The FDA has made it very clear that this will not happen, because each situation is different. However, they have released criteria to be used in making these determinations. These are defined as: impact on product quality and patient safety.
General advice came from FDA's John Murray when he answered questions concerning FDA's expectations at the Institute of Validation Technology (IVT) Computer System conference in May 2004: