Using a Delphi Survey to Assess the Value of Pharmaceutical Process Validation Part 1: Survey Methodology 1

Despite the long history of pharmaceutical validation, process validation in pharmaceutical manufacturing continues to be topical. European regulations regarding process validation were renewed in autumn 2001, which again brought the subject under the spotlight. Many people working in pharmaceutical production are now reviewing the state of their compliance practices and posing the question: "How will we benefit from process validation?"
To estimate the value of process validation, a systematic evaluation of the collected opinions and experiences of it was performed using technology assessment (TA). TA is an evaluation process that aims to protect people/society from the consequences of rapid technological developments and attempts to identify all the possible impacts of a technology, not just the intended ones.1 Today, technology in this context refers not only to the logical products of science, but also to the attitudes, processes, apparatus and consequences associated with it, and in that wider meaning, the principles of TA are well suited to the evaluation of process validation as a tool for pharmaceutical manufacturing.

Changes in the Delphi survey

There are a number of different methods of TA; this study used the synthesis (the compilation and evaluation of all available knowledge1) method. Initially, a literature search was performed,2 which revealed a lack of European experts' comments. Therefore, during autumn 2001, an experimental survey was conducted amongst European experts in pharmaceutical fields of manufacturing, regulation and academia to unearth their opinions. Objectives The main objectives of the study were to explore the value of process validation and to ascertain the best tools to perform it. Additionally, the study provided an opportunity to test how the principles and methods of TA could be used in the field of pharmaceutical quality assurance. Methodology Given that there is no single solution to process validation and that the value of process validation cannot be evaluated by using solid empirical measurement, but rather by informed judgement, 3 a discussion group was found to be an effective method for collecting information. For this reason, the Delphi method was chosen.
Delphi method. The procedure used in the Delphi method aims at structuring and distilling the mass of information from a selected group of experts by means of a series of questionnaires based on a structured process with controlled feedback.4 Moreover, this method was chosen for the following benefits:

• it enables participants from various countries and different fields (industry, authorities, schools) to take part
• it allows anonymous participation; a benefit that was of special value because the survey intended to facilitate discussion between the industry and its authorities, and to obtain comments from different organizational levels
• participants can take part asynchronously; that is, one may choose when to participate
• participants can choose to contribute to areas in which they are best qualified.

For development in the Delphi survey, see sidebar "Changes in the Delphi survey."

Figure 1 and Figure 2.

The Internet and e-mail. To accelerate communication, the Internet and e-mail were used. Using the Internet was also beneficial because it offered supporting tools for group communication, such as the potential for online discussion.6 Furthermore, the Internet provides a better and more illustrative means of informing the participants of the survey's key elements. Principles of the Delphi technique Although TA synthesis methods are frequently used to predict future scenarios, they can also be employed to critically examine the state-of-the-art of a given field.1,7 One of the most popular tools of synthesis is the Delphi technique. The aim of most Delphi techniques is the reliable and creative exploration of ideas or the production of suitable information for decision-making.
The replies to one round of questions are summarized and used to construct the next questionnaire. This reiterative process is continued until consensus or clear disagreement is reached among the participants.
Locating experts One disadvantage of the Delphi method is the definition and selection of experts; that is, whom to regard as an expert and how to create a representative group.
For this survey, experts were defined as those people working on process validation in:

• the pharmaceutical industry
• pharmaceutical authorities
• pharmaceutical schools
• consultant companies.

Representatives of the pharmaceutical industry were chosen from quality assurance, production and product development positions; representatives of the authorities had to either evaluate pharmaceutical and chemical aspects of the marketing authorization applications or work in inspection; and the representatives of pharmaceutical education had to teach process validation. Further details were unspecified; educational background was disregarded and the objective was to obtain the widest possible representation of different organizational levels. The level of experience of pharmaceutical process validation would have been of interest, but because of the limited number of experts in the field, this issue was not used as an exclusion criterion.