System Validation Requirements

Kevin Lloyd is vice president, chief technology officer of State College, PA-based Centre Analytical Laboratories.

In 1997 the U.S. Food and Drug Administration, in response to requests from industry, issued a regulation specifying criteria for the acceptance of electronic records and electronic signatures as the legal equivalent to manual, paper- based systems. The citation for this regulation can be found in the federal register as 21CFR11.

Citation 21CFR11, also known as Part 11, offers many advantages to industry, including a shortened period for agency review, nearly instantaneous reconstruction of studies, and enhanced confidentiality through limited authorized system access.

In order for an organization to reap the benefits of Part 11, there are many requirements that must be met. Due to the wide scope of Part 11 this article will focus only on the validation requirement. In future articles we will cover the full scope of the regulation.

For an organization to be compliant with Part 11, a number of requirements must be met. See Table 1 (in the print version) for a list of these requirements.

Due to the wide scope of Part 11, this article will focus on system validation requirements only. According to information posted on the FDA’s web site, validation is the establishment of documented evidence that provides a high degree of assurance that a specific process will consistently yield a product meeting its predetermined specifications and quality attributes. Implied by this statement: “If you didn’t document it, you didn’t do it.”

At some point in time, the FDA will inspect every company operating within an FDA-regulated environment. Although the FDA has made it clear that inspections will not be conducted solely for Part 11 compliance, it is well within its purview to inspect for Part 11 compliance within the context of a general audit.

A review of www.fda.gov reveals some common FDA-483 citation letters that have been recorded at previously inspected sites. These citations pertain to absent or incomplete records and/or practices. From the 483s, the following assumptions can be made regarding what the FDA may be looking for when they audit a facility:

• High-level hardware/software documentation, including diagrams and narratives detailing all computer programs and their relationships to each other

• Comprehensive inventories, including operating systems software, terminal emulators and client PC configurations

• Change control and error reporting SOPs

• IT personnel training records

• Software versions and installation dates

• Error reporting logs

• Evidence of proper handling of raw data files and metadata

• Archiving/backup procedures and responsibilities

• Hardware maintenance records

• Software requirements/design/testing documents

• Problem tracking

• Environmental monitoring specifications

• Disaster recovery procedures

It is clear from these requirements that a comprehensive, systematic plan is necessary to address the general expectations listed above, the Part 11 regulations and any applicable predicate rules. This plan can be divided into four phases.

Phase 1: Management Buy-In
Management buy-in is essential to any significant project in any company. The role of management in the validation program is to foster company-wide support for compliance, to make personnel available for appropriate responsibilities and to approve the expenditures necessary for implementation. It is essential to the success of a validation project to not underestimate the commitment required. Our 80-person company has invested 2 FTEs (1.5 man-years) since Jan 1, 1999 to this effort. The project also required the creation of a new position, CSVS (computer software validation specialist), whose full-time job is to ensure Part 11 compliance.

It is also essential to identify an internal “champion” to ensure the success of the computer validation effort. This person should be management-level within the company and have the authority to make key decisions and commit resources to the project. This person should also have the authority to set and enforce policy.

Another requirement for a successful validation is input from disparate functional areas of the company, usually accomplished by the formation of a validation committee. Within this committee, all affected departments must be represented, including IT and quality assurance. All participants must be clearly identified and held accountable to the objectives of the group.

Finally, hire a consultant! This may be costly; however, the project is large, complex and dynamic. It is my opinion that the best way to get started is with the help of an experienced consultant. Our project was nearly 100% dependent on external help at the beginning. However, we have gained experience and confidence and expect to be completely self-sufficient by mid-2001.

Phase 2: Develop SOPs To Address Operation and Maintenance of the Data Servers and Network
Even before the advent of Part 11, most companies understood the need for reliable information backup strategies, the value of a disaster recovery plan and the need for system security. It is the first task of the validation committee to identify all SOPs currently in place that relate to a company’s computer systems. Specifically, the following areas must be addressed:

• Computer System Network Backup and Recovery

• Computer System Data Archive

• Computer System Disaster Recovery

• Computer System Security

• Computer System User Administration

• Computer System Server/Network Monitoring

• Computer System Media Lifecycle

• Computer System Maintenance

As stated, some of these SOPs may already exist. If so, they should be reviewed by the committee and re-evaluated as to their relevance to current and expected practices. If any SOP is out of date or absent, it must be revised or retired and replaced. Computer system SOPs must not only be written, they must also be followed. Regulatory compliance is not satisfied by a well-written SOP; it is the job of the computer software validation specialist to ensure that day-to-day activities are consistent with the SOPs once they are implemented.

Phase 3: Validation System Development
At this point, a validation program must be formally established through a series of SOPs. It would be unusual for pre-existing SOPs to adequately address all aspects of validation system development, although some current SOPs (change control, inventories, etc.) may already be in use. A consultant can be a valuable resource for this phase, providing sample industry standard validation practices that can serve as baseline documents. It is then the task of the validation committee to modify these baseline procedures into detailed practices and procedures that are functional for their specific company. The following SOPs must be written for a successful validation system.

• Computer System Validation Policy

• Computer System Validation Program

• Computer System Change Control Policy

• Development of Computer System Requirements Documentation

• Development of Computer System Design Documentation

• Development of Computer System Test Documentation

• Computer System Configuration Management Plan

• Computer System Software and Hardware Inventory

• Computer System Environments

• Computer System Vendor Evaluation

Phase 4: Conduct a “Gap” Analysis
Once all the policy statements and SOPs are in place, it is necessary to conduct a “gap” analysis. The purpose of this exercise is to represent visually all system validation requirements. Draw a grid with the developed requirements along the ordinate and the individual computer systems along the abscissa. Physically test each system according to the requirements. Any computer system that does not meet one or more of the requirements of the validation program must be considered a “gap.” Gaps require remediation. Compliance only exists when there are no gaps.

Before we look at specific required SOP elements, it is necessary to define the software life cycle and categories of software. These definitions are fundamental to the validation effort.

Software Life Cycle
The graphical representation below is a waterfall model. (There are other acceptable models, but this one, in my opinion, is the simplest and most effective.) The software life cycle is based on the concept that software moves through a series of steps: it is born, it is tested against specs, it is maintained and it is finally retired. The waterfall model depicts this as a series of successive events. At every step, there is documentation to verify compliance.

Categories of Software
According to information presented at the Good Automated Manufacturing Practices (GAMP) ’96 conference, there are five categories of software. Each category has its own validation requirements. Table 2 (in the print version) provides an example of each category.

Computer Validation Policy
The computer validation policy is a high-level SOP that spells out in very general terms the goals of the validation program. An analogy is the corporate mission statement. Most companies have a statement that identifies broad goals and a level of commitment required of each employee of the company. The computer validation policy is similar except that its scope is limited to computer validation.

The following is an example of a computer validation policy for an analytical testing laboratory. It is important to note that the system life cycle features prominently in this policy. This is a foundation concept, around which the rest of the validation program is built.

“To support our goal of establishing ourselves as a premier supplier of analytical laboratory services, XXX shall develop and maintain standards and procedures whereby all computer systems in the company shall be implemented and maintained in a validated state according to FDA computer systems validation guidelines. It shall be the policy of XXX that validation of computer systems used to generate, manipulate, store or transmit data related to all regulated products or services shall follow a System Life Cycle (SLC) approach. This approach shall include documentation of computer system requirements and design specifications, thorough and documented testing of the computer system for compliance, and documented procedures to ensure consistent operation and maintenance of the system in a validated state until it is retired.”

The computer system validation program is another core SOP. This document also has a high-level mission statement. An example is given below:

“At its most basic level, validation is the demonstration of control through documented evidence during development and routine operation of a computer system. This control provides a high degree of assurance that a computer system will consistently yield a product or result meeting its predetermined specification and quality attributes. The process by which computer systems are brought into a validated state and then maintained in that validated state is the facilities validation program.”

Following this general statement, the computer system validation program SOP should detail specific actions to be taken to bring the facility into compliance. The first step, the procedure development phase, sets in stone all the elements necessary for the validation program. There are two distinct types of supporting SOPs required: (1) procedures for the operation and maintenance of the data servers and network, and (2) computer validation procedures. Development can be a time- and resource-consuming task, as it typically requires the efforts of the validation committee as well as help from an external consultant, over multiple iterations of each SOP, until the SOPs reflect the needs of each department.

Once policy SOPs have been developed, it is necessary to conduct a system inventory of all new and legacy systems. Assign a unique number to identify each computer system. Inventory each system and enter the records into a database. All information about the system will be recorded, including hardware configuration such as hard drive size, CPU clock speed and amount of random access memory, as well as a detailed inventory of the software on the system, including operating system version and build number. All new systems records should also include the installation dates of each component.

Once the inventory is complete, a gap analysis can be conducted to identify discrepancies between requirements (the validation program) and what has actually been done for each computer system. Once the gap analysis is completed, the remediation can begin, based upon the remaining elements in the validation program.

Requirements Phase: A requirements definition must be written for each system to identify specific operating and use requirements for the computer system. Requirements are documented and approved by management. The level of detail should be sufficient to support the design phase, commensurate with the GAMP ’96 complexity model. In other words, once the requirements are defined, they will be used as the basis of the design phase. The testing phase will assure that all requirements elements have been accounted for in the design phase.

During the requirements phase, stakeholders from each department that may ultimately use the system meet to discuss features of importance to their respective groups. For example, consider a custom temperature-logging application: QA may be interested in the calibration features of the application; operations may be interested in the ease of use while management may be interested in the initial cost and cost of operation. Once again, it is imperative that all groups are represented and all requirements are defined in this phase.

Design Phase: The design phase follows the requirements phase. The purpose is to assure that each element defined in the requirements phase is addressed. Level of detail is commensurate with the GAMP ’96 complexity model, which states, for example, that there is a difference between commercial off-the-shelf (COTS) software and a fully customized system: the former requires much less validation than the latter.

Implementation Phase: During the implementation phase all elements of the system, as defined in the design phase, are brought together. Elements may include COTS or custom code, or both. The product is a fully functioning piece of software that meets the original requirements.

Testing Phase: During this phase, the computer system is tested against specifications in the requirements definition phase. Tests are structured to provide traceability to both design and requirements documentation. Testing has three elements:

• Installation Testing – e.g., environment, power, plumbing

• Operational Testing – e.g., each individual function

• Performance Testing – e.g., maximum users, maximum data flow

Once testing is complete (successful) and the system is put into production, it enters a new phase: routine operation and maintenance. During this phase, the IT department must provide support and consultation to end-users. A possible outcome of this interaction may be maintenance. If so, all maintenance must be performed in a very controlled manner, under the conditions of the change control SOP. Any changes to the system inventory would also be updated at that time.

Finally, there is a decommissioning phase. At some point, it becomes easier to replace rather than maintain the current system. Starting over can be as simple as a new version/upgrade of currently used software or it can be as radical as replacement of the current system with that of a competing vendor. In any case, all data is archived and/or migrated to the new system. The old system is then removed from operation and a full audit trail of the dates and reasons for decommissioning is kept.

Change Control: The final SOP in the validation program defines how changes to a validated computer system are managed and documented. Changes are tested on a non-production system before they are implemented on the target system. The computer systems validation specialist must evaluate the impact of the change on the validation system. If it is significant change, revalidation may be required. Minor changes may require only documentation.

It is my opinion that a custom database application is the best way to implement change control. This system would include the following steps, each designed to demonstrate authorization and provide a documentation trail:

• Initiation – Anyone within the company can identify, recommend or initiate a possible change

• Acknowledgement – IT acknowledges receipt of the initiation (by e-mail, for convenience)

• Pre-approval – IT reviews the request, evaluates initial feasibility

• Implementation – IT executes the change

• Resolution – IT documents resolution of the problem/ change

• Testing – Demonstration of compliance and documentation of results

• Final approval – Acknowledgement of the action/resolution by the initiator, network administrator and the computer software validation specialist

• Update – Manual entry to system inventories reflecting the changes

• Audit – Performed by the computer software validation specialist

This article has focused on the validation requirement of Part 11, specifically, the high-level procedural SOPs that make up an effective validation program. Future articles will address specific objectives of the requirements, design and testing SOPs, as well as specific requirements of the operation and maintenance of the data servers and network.