Basic Principles of Study Design

This brief overview of study design addresses several points of particular concern related to current methodology used in the study of women with epilepsy and pregnancy outcomes. The studies envisioned are observational; however, all of the issues presented below are important methodological considerations.

Study Types

The design of a scientific study is determined by the primary purpose of the study. Here we consider studies designed to assess pregnancy outcomes in women with epilepsy. The main hypotheses being tested are driven by that objective. In general, purposes may be divided into three main categories:

• Descriptive
• Analytic
• Both descriptive and analytic

Beyond descriptive and analytic, the specific questions being addressed will further determine details of the study design, conduct, and analysis.

Descriptive Research

Descriptive research provides rigorous accurate information about the frequency of conditions in the population being studied, as well as about demographic and related characteristics associated with those conditions. The key concern is the representativeness of a sample vis a vis a population of interest. The gold standard methods for this purpose are complete census and population-based random sampling. Other methods are often used for pragmatic reasons; however, these methods must be evaluated carefully to ensure that they yield a representative sample of the population. Where those included in the study population vary from a representative sample, this variance must be described as should the potential consequences of the variance on generalizing results. Implementation of the sampling method is important. A sampling method may be adequate by itself, but it will not yield the necessary effect if implemented poorly.

Analytic Research

Analytic research provides an internally valid assessment of statistical associations, which may represent causal associations. The key concern is the comparability of samples being compared. In this regard, confounding factors must be considered. The gold standard method for achieving comparability and reducing or eliminating the influence of confounding factors is randomization, which distributes those factors across the samples.

Observational Analytic Research

By definition, observational studies do not utilize randomization. The needs of observational analytic research studies emphasize sources of sampling and measurement bias. Biological understanding of the disease under study, the exposure, and their potential associations is necessary.

Study Design Errors

Errors in study design and conduct of a study can affect both types of research and result in inaccurate descriptive information, spurious associations, attenuated or exaggerated associations, and/or an inability to detect an association that is present in the source population. The key issues are:

• Sampling
• Measurement (of both health outcome and exposures)
• Beta error and lack of precision

Sampling Errors and Bias

Sampling error and biases occur when individuals with a particular characteristic are included more often in a study sample than those without that characteristic. This can occur through various mechanisms involving either the method of sampling, where certain individuals are more likely to be identified for inclusion in the study, or by response, where certain individuals are more or less likely to participate in the study when invited.

There are ways to avoid or minimize sampling errors and biases through the design and the conduct of the study.

• Study Design – Use methods that emphasize identifying a sample representative of the population of interest. This requires understanding the social, medical, and related systems with which and through which a study must operate.
• Conduct of Study – Consider approaches to maximize response rate and avoid selective refusal associated with important (relevant) characteristics. This requires understanding psychological and social factors that may influence an individual’s acceptance of study participation.

Measurement Bias

Measurement bias is an important consideration in observational studies. Assessment of exposure should be accurate, both sensitive and specific. The quality of exposure assessment should be entirely independent of the outcome of interest. Where possible, assessment of exposure should be done prior to occurrence of the outcome. When this cannot be done, the assessment should be made without knowledge of the outcome to the extent possible. Exposure assessments should be standardized and performed in exactly the same way for everyone, regardless of the outcome. Factors that might have even subtle effects on quality of measurement should be distributed equally among study participants, regardless of outcome. Such factors include, but are not limited to, clinicians making medical assessments, interviewers collecting information, and instruments used in making measurements.

Principles for Quality of Outcome Assessments

The principles for quality of outcome assessments are essentially the same as those for exposure assessment. Assessment of outcomes must be accurate. Again, this means they must be both sensitive and specific. In the case of teratology, some outcomes are quite obvious and may not require particularly specialized skill for their determination. For abnormalities (physical anomalies as well as behavioral and neuropsychological deficits) that are more subtle, it may require a systematic examination by a trained specialist.

Examinations should be performed in precisely the same way for all patients/subjects regardless of exposure status and without knowledge of exposure status by the examiner. Factors that may affect the examination subtly (examiner, instruments, etc.) should be distributed equally among exposed and unexposed individuals.

Beta Error and Lack of Precision

Beta error is the probability that the results of a study will miss an association that is present in the source population. Statistical power is the flip side of beta-error. It is the probability that a study will detect an association if it is there in the source population.

Statistical Power

Statistical power is primarily dependent on 4 factors:

• Sample size – the larger the sample size the better the power (the less the beta-error)
• Frequency of the exposure in the population – the relation of exposure frequency to power is complicated since it depends on other factors. In general, in a cohort study, power is lower when the exposure is very rare. If the exposure is a continuous measure, then its standard deviation is the driving factor (generally, a smaller standard deviation, better power)
• Frequency of the outcome – Again, the relationship of outcome frequency and power depends on other factors. Extremely rare outcomes pose considerable difficulties for power in cohort studies. For very common outcomes (e.g. >40%), however, power can also be compromised. This is not an issue for case-control studies because we sample on the outcome. When the outcome is a continuous measure, power is driven by the standard deviation of the outcome measure.
• Magnitude of association – Generally, the larger the effect in the source population, the better the power of a given study to detect it.

Lack of Precision

Precision, statistical significance and power are integrally related and represent different facets of the same quantities and principles. The relative risk, odds ratio, or risk difference derived from a study provides the study’s best estimate of that parameter in the population. That estimate is associated with an error quantified by a confidence interval. The confidence interval represents the likely range of population values, which (provided that certain key assumptions are valid) might have resulted in a study sample with the given estimate.

A narrow (precise) confidence interval is preferred, as it is more informative about the likely value in the population. A very wide confidence interval (imprecise) may confirm a statistically significant association, but may be compatible with an effect ranging from negligible to extremely large.

Back to top

Return to Articles and Publications