Using a Delphi Survey to Assess the Value of Pharmaceutical Process Validation Part 1: Survey Methodology 2

Given the pharmaceutical industry's sensitivity concerning knowledge sharing and to get both the regulating authorities and the regulated industry involved in the survey, anonymity was regarded as essential.

Figure 3 and Figure 4.

The search for participants took 5 months and was the most challenging part of the survey, particularly contacting industry experts because their e-mail addresses could not be located. To find suitable people from the pharmaceutical industry, a snowball method was used.23 Additionally, addresses for quality assurance, product development and production experts were requested from the representatives of appropriate foreign companies in Finland and from the qualified persons of appropriate domestic companies. However, only a few participants from the foreign companies participated because the representatives did not know who the right people were to ask, or the company refused to participate because of time constraints. The only effective way of obtaining willing participants was finally found to be direct telephone contact; this method also worked well for all three pharmaceutical fields. The size of the expert group is also important to the outcome of a Delphi study, but it depends on the homogeneity of the expert population and whether the study searches for qualitative or quantitative results.23 Other Delphi surveys have varied in size from 10–15 up to 2000–3000.23,24 For this survey, the number of experts was limited and the study mainly searched for qualitative results. Thus, a group of approximately 30–50 participants was considered apposite.
The questionnaires An extremely important part of the Delphi method is the questionnaires, particularly the first round questionnaire (Q1). Q1 needs to be easy and clear, to motivate and encourage the respondents; otherwise they may lose interest. The design of the questionnaires was carefully discussed by the advisory group before the start. The objective was to form easy-to-use and fast-to-complete questionnaires. It was agreed to keep the number of questions to a minimum and to limit the use of open-ended questions. Not only would these measures avoid making the survey too time consuming, but they would deter those who would use these factors as reasons for abandoning the survey. The use of open-ended questions is, though, often found necessary to eliminate the possible external bias linked to the guiding role of the investigator.7 Instead of open-ended questions, the external bias was eliminated using a mixture of negatively and positively influenced arguments. The questionnaires were also pre-tested with a pilot study among the advisory group and external experts.
Although the aim was to identify the attitude on the usefulness of pharmaceutical process validation, a further aim was to establish whether there are differences in attitudes between European countries or the different parties.
Operationalization of the subject was done under five headings:

• How do you feel about process validation?
• What are the benefits of process validation?
• What are the negative aspects of process validation?
• How can we make process validation easier and more effective to get the most from it?
• What hinders positive thinking on process validation?

The first three questions were designed to measure the overall attitude towards process validation and the final two were mostly determined the reasons underlying the attitudes, and also attempted to find out if some specific tools of process validation were known among the participants.
The initial round started with a background information questionnaire, which also included one final question estimating the overall opinion on process validation. This questionnaire was to be completed before commencing Q1, and thus, the last question served as a control, measuring the attitude at the beginning of the study. The same question was repeated at the end of the second round questionnaire (Q2).

Figure 5: Pharmaceutical participant demography: gender, education and age.

Q1 consisted of 31 questions, most of which were multiple-choice. Only three of the questions were open-ended to increase the inclination and quality of personal expression. Q2 included some new or modified questions, but most were repeated in their original form together with a summary of the answers from Q1. In Q2, the opportunity to add comments was provided after every question and was encouraged by offering participants the option to answer in their native language. Also in this round, the questions were posed in a slightly different order, with the addition of one new heading - "Cost of validation." In Q1, questions concerning the cost of validation had been spread under other headings.
The Internet as an environment for the survey The WebCT (WebCT, Inc., Lynnefield, Massachusetts, USA) learning environment26 was chosen as the platform for the survey because it was already being used in Helsinki University and offered the required tools. Extranet homepages were constructed and the questionnaires, together with supporting communication tools and information pages, were available from the Internet public page. The participants were e-mailed the passwords required to access these pages.
Guidance and other information, including definitions of the critical terms, were offered on the homepages and an online forum was available for anonymous discussion. The discussion area was continuously available and an online forum was organized twice during the survey - before and after Q2. The surveys' Internet public homepage can still be viewed.