Aktuelle Version vom 19. März 2014, 23:09 Uhr

Design of Experiments

Introduction

Design of Experiments (DoE) defines methods of how to plan and arrange experiments. We explain the method using the example of a baker in a bakery. The baker wants to analyze the effect of different settings of his oven to the quality of the resulting bread by conducting several experiments.

Target Values

We define the targets of experiments (here to improve the quality of the bread) as target values.

Input Parameters

All sources of influence on the target values are referred to as factors. Three different groups of factors are distinguished:

1. noise factors: These factors cannot be controlled or controlling these factors is too expensive. As an example, the temperature and humidity in the bakery depends on the season and weather and cannot be controlled.
2. control factors: These factors can be controlled. However, their influence on the target value is not of interest and thus, these factors are set constant. For example, the baker wants to leave his recipe for the bread unchanged.
3. signal factors: These factors can be controlled. Their influence is of interest and their values are modified in the experiments.

DoE is important when we have to analyze a process with multiple signal factors, in order to determine their effect on the target value(s).

Note: In the following, we will only consider signal factors. For simplicity, they are referred to as factors.

Factor Levels

For each factor, we may want to analyze different values. For example, we want to analyze an oven temperature of 185, 190, 195, and 200^oC and a baking time of 45, 50, 55 and 60 minutes. These different values are referred to as factor levels.

Experiment

One experiment is one combination of different factor levels and the corresponding target values for the experiment. For example, one experiment may be a temperature of 190^oC and a baking time of 60 min. One experiment is also referred to as one factor level combination.

DoE Plan

The DoE plan defines which experiments are to be conducted and in which sequence they are conducted. The sequence may be important as environmental conditions like temperature may effect the target values as well as hysteresis effects.

DoE plans differ in number of total experiments and the order of the experiments. The number of experiments strongly influences the information gain and the trust in the experiments.

As references for DoE plans we recommend [ 1, 2 ].

DoE Plans

A DoE plan can be considered as a set of rules describing how to choose factors, the number of factor levels, factor level values or value distributions, etc.. In the following, two different DoE plans are outlined.

Full Factorial

A two factor Full Factorial plan. Each square represents one possible experiment. The "X" denote the actual experiments that will be scheduled.

For given numbers of factor levels for each variable, a limited number of different experiments can be conducted. A DoE plan with all possible experiments is referred to as Full Factorial plan: The number of factor levels may vary for the factors. All other plans use only a subset of all possible experiments.

Latin Hypercube

A two factor Latin Hypercube plan. Each square represents one possible experiment. The "X" denote the actual experiments that will be scheduled.

Compared to the Full Factorial plan, the Latin Hypercube (LHC) sampling dramatically reduces the number of experiments (see e.g. [ 3 ]). So it is most useful for expensive experiments, whether they be money or time intensive. Latin Hypercube requires the same number of factor levels for all factors. The number of experiments of a LHC plan is equal to the number of factor levels and is independent on the number of factors. This makes it very scalable (O(1)) in respect of number of factors.

The philosophy of LHC is to use every factor level exactly once. The figure on the right demonstrates a possible plan for 4 factor levels. As each factor level is only used once, each row and column in the figure must contain exactly one cross. Thus, LHC guaranties that for each factor, the entire range of factor levels is used.

Common Settings for all DoE Plans

The following list explains common settings for all DoE plans.

Number of Iterations

The testing of a set of experiments that a DoE plan proposes is referred to as one iteration. If we would schedule three iterations, each experiment (i.e. a certain factor level combination) in the set is tested exactly three times.

Using more than one iteration is only necessary if the system to analyze is noisy (i.e. the target values are noisy), may contain a drift while testing, or produces hysteresis.

Random Scheduling

Four different scheduling plans of six experiments resulting from DoE plan with three experiments and two iterations. The small double-bordered boxes outline the blocks.

Without random scheduling, the experiments are tested in a fixed order. With randomization activated, a random sequence for the experiment is set up. To what extend the randomization occurs depends on whether block building is active or not.

Random scheduling may help to analyze effects like hysteresis, i.e. the previous experiment effects the outcome of the current experiment. If we run multiple iterations, the effect of hysteresis decreases because of statistical reasons.

Block Building

If block building is activated, all experiments of one iteration are tested. Then, all experiments of the next iteration are tested. If block building is deactivated, experiments of different iterations may follow each other. Block building, like random scheduling, may reduce the effect that may occur because of the order in which the experiments get scheduled. While random scheduling may reduce the effect of subsequent experiments, activating block building may reduce the effect caused by external (long term) influences. These influences can be quality of the used materials, which may vary over time or depend on the supplier, human interaction with the machines or the accuracy of machines of the same type used for the experiments.

DoE in OpenDino

A description for DoE in OpenDino can be found in the module documentation (see Documentation/Modules/DoePlanner).

References

1 Wilhelm Kleppmann, Taschenbuch Versuchsplanung (4. Auflage) - Produkte und Prozesse optimieren, Carl Hanser Verlag Muenchen Wien, 2006

2 http://en.wikipedia.org/wiki/Design_of_experiments

3 http://en.wikipedia.org/wiki/Latin_hypercube