Design of Experiments
On this page
What is a DOE?
Design of Experiments (DOE) is a systematic approach to planning physical tests or computer simulations to study how inputs affect outputs. A DOE specifies the combinations of input settings to evaluate, with each combination representing a test or simulation run.
For example, a cooling-system DOE might vary coolant flow rate, inlet temperature, and channel geometry, then record the resulting maximum component temperature. The choice of design depends on the study's objective, input types, and available resources.
Why DOE Matters:
More Insight from Every Simulation and Test
Engineers need to understand how design choices, manufacturing tolerances, and operating conditions affect performance, but every simulation and physical test consumes time and resources. A well-designed DOE helps teams answer these engineering questions with informative data while keeping computational and testing costs under control.
Complex systems often have many interacting inputs and nonlinear responses. Changing one factor at a time can miss interactions, while testing every possible combination quickly becomes impractical as the number of variables grows. DOE addresses these challenges by distributing samples across the design space and varying factors together, providing the coverage needed to investigate sensitivities, interactions, and performance trade-offs.
The benefits extend beyond collecting data. A well-chosen DOE creates a stronger foundation for surrogate modeling, optimization, and uncertainty quantification. Engineers can identify influential parameters, explore alternative designs, and assess the effects of variability with less reliance on repeated expensive simulations. As a study progresses, additional runs can fill gaps or target regions that need more information, helping teams spend their next simulation or test where it offers the most value.
SmartUQ's DOEs: Unique Capabilities and Best in Class Performance
SmartUQ combines specialized space-filling designs with proprietary adaptive sampling and flexible tools for extending existing experiments. This combination addresses two central engineering needs: collecting enough information to represent a complex system accurately and controlling the number of costly simulations or physical tests required.
SmartUQ's adaptive DOEs use a fitted surrogate model and previous simulation results to select new samples that improve predictive accuracy. DOE augmentation offers a complementary approach: it fills gaps in an existing sampling pattern without requiring a fitted model, preserving the value of data already collected. In the adaptive example below, targeted sampling reduced model error by an order of magnitude compared with a one-shot design using the same number of simulation runs, illustrating the value of choosing samples based on what has already been learned.
Engineering studies also need to accommodate changing budgets, different simulation fidelities, and mixed input types. SmartUQ's sequential and multi-fidelity DOEs support staged sampling and coordinated studies across fidelity levels. Its parallel space-filling designs organize runs into batches, support combinations of continuous and discrete inputs, and help distribute work across available computing or testing resources.
Together, these capabilities give engineers a flexible path from initial exploration to model refinement. Teams can start with broad coverage, reuse existing experiments, and direct additional effort toward gaps in coverage or areas where predictions need improvement. The result is a DOE workflow built to make better use of engineering data, simulation budgets, and test capacity.
Space-Filling Designs
Factorial DOEs provide robust information gathering capabilities but they often require taking more samples, at greater expense, than are truly necessary for determining cause and effect relationships in highly non-linear systems. Modern space-filling designs are often the most efficient DOEs for simulations and complex physical experiments with highly non-linear responses.
Space-filling designs, such as Latin hypercube designs and optimal Latin hypercube designs, sample a series of representative input configurations evenly distributed to fill the design space. Research shows this type of DOE minimizes the number of runs while maximizing the potential learning gained from each simulation. Thus, space-filling designs offer high accuracy while reducing the total number of sampling points.
Sequential and Multi-Fidelity DOEs
Sequential DOEs use multiple DOEs, each optimized to fill gaps in the previous sampling pattern. This process can be repeated to increase emulator/surrogate model accuracy, helping minimize the total simulation time by only running the simulations necessary to achieve the desired level of accuracy. This DOE also allows new options for combining multi-fidelity simulations. Low-fidelity simulations can be used to generate an emulator/surrogate model and to quickly explore a design space. To ensure accuracy, the emulator/surrogate model can be calibrated using a smaller sampling of high-fidelity simulations. It is also possible to use this DOE to strategically run simulations and increase emulator/surrogate model accuracy as computational resources become available.
Example
In this example an initial 4-point DOE is generated. It is followed by a second 4-point DOE and a third 8-point DOE which sequentially add to the resolution of the initial DOE.
Parallel Space-Filling DOEs
In parallel DOEs, the sampling points are divided into separate parallel batches. This is particularly helpful when varying a mixture of continuous and discrete inputs: each batch can handle a separate level of discrete inputs. It is also possible to support multiple discrete inputs by building each batch such that it is composed of smaller parallel DOEs.
These DOEs are also useful for dividing computational and experimental work between multiple groups. For computer experiments, being able to divide the computational load of the full experiment among larger groups of computing devices can yield a significant reduction in clock time.
Example
This example shows an optimal 2-D, 20-point, 4-batch DOE suitable for handling two continuous inputs and one discrete input with four levels.
This example shows a 2-D, 36-point, doubly parallel DOE suitable for handling two continuous inputs and two discrete inputs with three levels each.
Adaptive DOEs
Adaptive Design allows the user to increase the accuracy of a fitted emulator/surrogate model (surrogate model, meta-model) by adaptively adding more points to the DOE used to create the emulator. In this process, an emulator/surrogate model is created from an existing DOE and the associated results data. This emulator/surrogate model is then used to determine where the next simulation run should be added in order to increase the accuracy of the emulator. This process can be repeated until the desired level of fitting accuracy is achieved.
Example
A test surface, shown here, with several difficult-to-fit features was emulated by our algorithms using two methods: a one-shot optimal Latin hypercube design and our Adaptive DOE.
The first DOE was a one-shot optimal Latin hypercube design. The emulator/surrogate model generated with this DOE yielded the model surface shown below.
The second DOE was our proprietary Adaptive DOE. Starting with an initial 20-point optimal LHD, SmartUQ’s adaptive design feature added five points to the design space, one by one. Each point targets specific areas of the design space chosen by our algorithms based on the previous emulator. This approach resulted in a much more accurate model, reducing error by an order of magnitude while using the same number of simulation runs.
DOE Augmentation
Design Augmentation allows the user to generate a DOE that combines with an existing DOE in such a way that the new points maximize the space-filling properties of the combined DOE. In practice, it is sometimes undesirable or infeasible to run all of the design points necessary for a large DOE in one simulation series. It is common to have existing DOEs from previous design space explorations, optimization procedures, and previous product development cycles. These DOEs might not have good space-filling properties but still contain valuable information. Design Augmentation is useful for determining the optimal simulation runs required to maximize the information that can be learned from a given number of new simulation runs while also taking advantage of previously collected data.
Design augmentation is different from Adaptive Design: In Design Augmentation the optimal points are generated based only on the properties of the existing DOE itself. Thus the new points are blind to the simulation results. In Adaptive Design, the optimal points are generated based on an emulator/surrogate model created using the previous simulation results as well as the properties of the DOE. Design Augmentation is more robust than Adaptive Design because the former is model free and works for simulations with any number of responses. Design Augmentation and Adaptive Design are both useful complimentary tools and the user should decide between these two options based on the properties of the specific case being studied.
Example
In this example an initial DOE of poor quality is augmented so that the resulting DOE better covers the full design space.
In this example a sparse grid with missing data because of failed simulation convergence is augmented to increase the DOEs space-filling properties. This feature is particularly useful when polynomial chaos based uncertainty analysis cannot be performed due to missing data.
Data Sampling
DOEs are typically used to collect new data from a system. In many cases, sufficient data has already been collected. Often in these scenarios, the data collected has been accumulated over long periods of time, and there is enough data that analysis is simply intractable. For example, health monitoring data from sensors on fielded components may capture live data continuously over the entire operating life of the component. SmartUQ’s data sampling tools can divide the data to mimic a space-filling DOE consisting of subsets of the full data set. Unlike DOEs which are developed before data collection, data sampling, such as subsampling and sliced sampling, takes existing input-output data pairs and selects the points that will represent the design space well.