Multifidelity methods leverage both low- and high-fidelity data in order to maximize the accuracy of model estimates, while minimizing the cost associated with parametrization. They have been successfully used in wing-design optimization, robotic learning, and have more recently been extended to human-in-the-loop systems, such as aerospace and transportation. They include both model-based methods, where a generative model is available or can be learned, in addition to model-free methods, that include regression-based approaches, such as stacked-regression. The approach used depends on the domain and properties of the data available, and is similar to the concept of metasynthesis, proposed by Judea Pearl.
The fidelity of data can vary along a spectrum between low- and high-fidelity. The next sections provide examples of data across the fidelity spectrum, while defining the benefits and limitations of each type of data.
In situations when there is not an adequate amount of high-fidelity data available to train the model, low-fidelity data can sometimes be used. For example, low-fidelity data can be acquired by using a distributedsimulation platform, such as X-Plane, and requiring novice participants to operate in scenarios that are approximations of the real-world context. The benefit of using low-fidelity data is that they are relatively inexpensive to acquire, so it is possible to elicit larger amounts of data. However, the limitation is that the low-fidelity data may not be useful for predicting real-world expert performance due to differences between the low-fidelity simulation platform and the real-world context, or between novice and expert performance.
High-fidelity data (HiFi)
High-fidelity data includes data that was produced by a person or Stochastic Process that closely matches the operational context of interest. For example, in wing design optimization, high-fidelity data uses physical models in simulation that produce results that closely match the wing in a similar real-world setting. In HITL situations, HiFi data would be produced from an operational expert acting in the technological and situational context of interest.
HiFi benefits and limitations
An obvious benefit of utilizing high-fidelity data is that the estimates produced by the model should generalize well to the real-world context. However, these data are expensive in terms of both time and money, which limits the amount of data that can be obtained. The limited amount of data available can significantly impair the ability of the model to produce valid estimates.
Multifidelity methods (MfM)
Multifidelity methods attempt to leverage the strengths of each data source, while overcoming the limitations. Although small to medium differences between low- and high-fidelity data are sometimes able to be overcome by multifidelity models, large differences can be problematic leading to decreased predictive performance when compared to models that exclusively relied on high-fidelity data. Multifidelity models enable low-fidelity data to be collected on different technology concepts to evaluate the risk associated with each concept before actually deploying the system.
