Introduction: The High Stakes of Simulation-Driven Compliance

In industries where failure is not an option—aerospace, medical devices, automotive, and nuclear energy—simulation has evolved from a complementary design tool into a primary source of regulatory evidence. The ability to accurately predict product behavior under a wide range of conditions using Computational Modeling and Simulation (CM&S) is invaluable for accelerating time-to-market and reducing physical prototyping costs. However, this speed demands heightened scrutiny. Regulatory bodies such as the FDA (Food and Drug Administration), FAA (Federal Aviation Administration), EASA, and the NRC (Nuclear Regulatory Commission) require irrefutable proof that a simulation model is not just sophisticated, but is demonstrably credible, accurate, and appropriate for its intended purpose.

The cost of insufficient validation is high. A regulatory submission rejected due to poor simulation evidence can delay a life-saving device by years. An unverified model guiding a critical spacecraft landing sequence could lead to catastrophic loss. This high-stakes environment demands a structured, rigorous, and transparent approach to simulation quality. This guide explores the best approaches for validating and certifying simulation quality for regulatory compliance, providing actional frameworks aligned with global standards while leveraging modern data management and automation strategies.

Understanding the V&V Framework: Verification vs. Validation

A rigorous Validation and Verification (V&V) framework forms the backbone of any credible simulation effort. While the terms are often used interchangeably, they represent distinct activities with specific goals. Confusing the two is a common pitfall that can undermine an entire regulatory submission. Standards such as the American Society of Mechanical Engineers (ASME) V&V 10 Standard for Computational Modeling and Simulation frame this distinction clearly and provide a foundational structure for building trust in simulation outputs.

Verification: Solving the Equations Right

Verification asks the question: "Did we build the model correctly?" It is a purely mathematical and software-oriented exercise. Verification activities focus on ensuring that the computational model faithfully represents the underlying mathematical model (the equations) without numerical or coding errors. This involves code verification, where discretization error is assessed using analytical benchmarks or manufactured solutions, and calculation verification, where numerical errors (such as time-step size or mesh density) are quantified and controlled. A verified model ensures that the software is solving the equations free of bugs and that the numerical solution is converging to the true solution of the model.

Validation: Solving the Right Equations

Validation asks the more challenging question: "Did we build the right model?" It assesses the agreement between the computational model's predictions and reality. This is accomplished by comparing simulation results to high-quality experimental data representing the physical system of interest. Validation is not a one-time event; it is a continuous process that defines the domain in which the model can be reliably used. The ASME V&V 20 standard outlines a specific approach for assessing the accuracy of a CFD code's predictions relative to experimental data. The difference between the simulation prediction and the experimental observation provides the validation error, which must be quantified and bounded to establish predictive capability.

Key Validation Techniques for Building Regulatory Evidence

To satisfy regulatory demands, validation must move beyond subjective comparisons of contour plots. It requires quantitative metrics, rigorous uncertainty quantification, and a clear connection to the intended use of the model. Below are the core techniques used to build a defensible validation case.

Quantitative Comparison with Experimental Data

Regulatory agencies expect objective, statistical measures of model accuracy. Relying on qualitative visual agreement is insufficient. Common quantitative metrics include the Root Mean Square Error (RMSE), the Sprague-Geers Metric (which separates magnitude and phase error), and the Normalized Error. These metrics provide a standardized way to evaluate how well the simulation matches reality. The key is to define acceptable error thresholds before running the validation simulations to avoid bias. Additionally, the experimental data used for validation must have its own associated uncertainty; a simulation cannot be validated against data that is itself unverified or has unknown error bounds.

Predictive Capability and Extrapolation

A validated model is only valid within its "validated domain"—the specific range of input parameters and scenarios tested. Regulatory bodies are understandably skeptical of models used far outside this domain. To earn trust in extrapolation, organizations must demonstrate a deep understanding of the underlying physics. This is where Uncertainty Quantification (UQ) becomes critical. By characterizing aleatory (natural variability) and epistemic (knowledge-based) uncertainties, teams can assign confidence bounds to predictions. The NRC and FDA often require explicit statements of predictive capability, including estimated bounds on predictions even when experimental data is unavailable.

Sensitivity Analysis for Model Robustness

Sensitivity analysis (SA) is a validation technique that investigates how changes in model inputs affect the outputs. It helps identify which parameters drive the simulation results the most. This is important for two reasons: first, it focuses validation testing on the most impactful parameters, and second, it reveals potential fragility in the model. Global sensitivity analysis methods, such as Sobol indices or the Morris method, are preferred over local "one-at-a-time" approaches because they account for interactions between parameters. A model that is hypersensitive to a poorly characterized input is likely a poor candidate for regulatory acceptance without extensive additional testing.

Benchmarking Against Industry Standards

Comparing a model to established benchmark cases validates the physics implementation and demonstrates alignment with industry best practices. Standards such as those published by the ASME V&V 10 consortium or the NAFEMS provide benchmark problems with known solutions. Successfully reproducing these results does not guarantee model validity for a specific application, but it builds confidence in the software tool's capability to handle relevant physics. Regulatory submission dossiers often include a section demonstrating that the simulation tools themselves have passed formal software quality assurance (SQA) standards (e.g., IEC 61508 or internal validation suites).

Certification is the formal process by which a regulatory body accepts the simulation evidence as part of the basis for product approval. The specific requirements vary dramatically by industry, but common themes include traceability, configuration management, and independent review.

Aerospace: DO-178C / DO-331 (Model-Based Development)

In aerospace, software developed for safety-critical systems must follow the DO-178C standard. For simulation models used in Model-Based Design (MBD), the DO-331 supplement applies. This standard requires that models be verified against their requirements and that the code generated from them be traceable back to those requirements. Certification involves rigorous independence: verification activities must be performed by someone other than the model developer. The industry strictly enforces "objectives" that must be satisfied for each Software Level (A through E), with Level A requiring the highest rigor.

Medical Devices: FDA In Silico Guidance

The FDA has been a leader in accepting simulation evidence, particularly under the Medical Device Development Tools (MDDT) program and the recent FDA Modernization Act 2.0. The agency expects a "Risk-Informed" framework, where the level of V&V rigor scales with the risk of the device. The ASME V&V 40 standard specifically guides the submission of computational models for medical devices. It emphasizes establishing a "Context of Use" (COU) and defining "Quantitative Prediction Goals" (QPGs). The model must demonstrate not just accuracy, but relevance to patient safety and clinical outcomes.

Automotive: ISO 26262 and ASPICE

The ISO 26262 standard governs functional safety for road vehicles. For simulation models used in developing Advanced Driver-Assistance Systems (ADAS) or powertrain controls, the standard requires a clear V-Model approach. Validation must confirm that the model behaves correctly under fault conditions and edge cases. ASPICE (Automotive SPICE) specifically audits simulation development processes for maturity and traceability. An automotive supplier must often prove that their simulation environment accurately reflects the vehicle dynamics and sensor behavior that will be encountered in the real world.

Nuclear Energy: NRC Regulatory Guides

The U.S. Nuclear Regulatory Commission (NRC) has historically required extremely detailed V&V for reactor safety analysis codes. The Evaluation Model Development and Assessment Process (EMDAP) outlined in NRC RG 1.203 is one of the most rigorous frameworks in existence. It demands a complete chain of evidence linking the modeled phenomena to the real physical behavior of a nuclear reactor. This includes explicit treatment of scaling (from test facilities to full-scale reactors) and a detailed assessment of how uncertainties propagate through the system.

Building a Robust Simulation Quality Management System (SQMS)

Producing compliant simulation evidence requires more than technical skill; it requires a system of processes that ensure consistency, traceability, and repeatability. An effective SQMS governs the entire lifecycle of a model, from initial request to final archival.

Documentation and Traceability

The single largest failure point in simulation certification is poor documentation. Every assumption, input parameter, boundary condition, and decision must be recorded. This includes not only the simulation results but also the reasoning behind modeling choices, the source of data, and the version of the software used. A traceability matrix linking simulation requirements to specific validation tests provides the "chain of evidence" that auditors and regulators require. Modern organizations use data management platforms to link simulation cases directly to the digital thread of the product.

Configuration Management

Simulation models evolve. A failure to manage these revisions can invalidate months of work. Configuration management ensures that the exact state of a model used for a specific regulatory submission can be retrieved and proven to be correct. This includes version control for the model file, the solver settings, the mesh, and the boundary conditions. It also includes the underlying platform data. Tools like Git for simulations, combined with controlled databases for results, prevent the common scenario of an engineer saying, "I can't remember exactly which settings generated those plots."

Independent Verification and Validation (IV&V)

Many regulatory standards, particularly DO-178C, require independence between the development team and the V&V team. IV&V provides an unbiased assessment of the model's fidelity and its suitability for its intended use. An independent reviewer is not emotionally invested in the model's success and is more likely to uncover hidden flaws. The results of IV&V activities—including discrepancy reports and corrective actions—form a core part of the certification package.

Automated Regression Testing

As models are updated to reflect new data or physics, there is a risk that changes will break previously validated behavior. Automated regression testing solves this by re-running a standard suite of validation cases whenever a model changes. This practice is borrowed directly from software engineering and applied to CM&S. If a new physics module reduces accuracy on a standard benchmark, the test fails, and the team is alerted immediately. Regression testing provides the safety net needed to continuously improve models without losing regulatory confidence.

Overcoming the Key Challenges in Simulation Certification

Even with the best processes, several technical and organizational challenges remain. Recognizing and mitigating these challenges is a sign of a mature simulation organization.

The Black-Box Problem: AI/ML and Neural Networks

The rise of machine learning in simulation presents a certification paradox. While neural networks can approximate complex physics orders of magnitude faster than traditional solvers, their lack of interpretability makes them difficult to validate against standards designed for deterministic code. Regulators are skeptical of "black boxes." To certify an AI/ML model, organizations must employ techniques like explainable AI (XAI), rigorous out-of-distribution detection, and uncertainty estimation. The model must demonstrate that its internal logic is consistent with physical laws, not just statistical correlations.

Balancing Model Fidelity with Computational Performance

High-fidelity models (LES, DNS) are incredibly accurate but computationally expensive. Lower-fidelity models (RANS, 1D) are fast but may miss critical physics. Certification requires finding the right balance. The chosen fidelity must be sufficient to make a reliable prediction about the specific regulatory question being asked. A rigorous mesh independence study and a clear justification for the modeling assumptions are necessary to defend the chosen fidelity level to a regulator.

Managing Legacy Models and Data

Many organizations possess decades of validated simulation models that are poorly documented or use outdated file formats. Bringing these models into a modern compliance framework can be daunting. A risk-based approach is best: high-risk legacy models requiring new regulatory submission should be prioritized for complete re-documentation and re-validation. Lower-risk models can be archived with a clear note of their limitations. A centralized data platform can help by abstracting the model details from the source file format.

Best Practices for Streamlining Regulatory Approval

Success in simulation certification hinges on proactive strategy, not reactive documentation. By embedding compliance into the daily workflow, organizations can accelerate approval and reduce rework.

Early Engagement with Regulatory Bodies

Waiting until the validation report is written to approach a regulator is a recipe for rejection. Most regulatory bodies offer mechanisms for early dialogue, such as the FDA’s Q-Submission program or the FAA's Issue Papers. Presenting the plan for V&V, the proposed context of use, and the acceptance criteria early allows for regulatory feedback that can save months of wasted effort. A pre-submission meeting ensures that both parties agree on what "good enough" looks like.

Risk-Based Validation Prioritization

Not every simulation case carries the same weight. A model driving a safety-critical implant (ISO 13485 / FDA Class III) requires far more rigor than a model guiding a non-critical trim component. Allocate validation resources based on risk severity and model novelty. High-risk, high-novelty cases demand full V&V including IV&V. Low-risk, well-understood cases may only require a simple benchmark check. This prioritization keeps budgets under control without sacrificing safety.

Leveraging a Centralized Data and Process Platform

The biggest bottleneck in simulation certification is the manual effort required to gather, organize, and trace data. A centralized platform acts as the source of truth for all simulation artifacts—models, boundary conditions, results, validation data, and reports. Platforms like Directus, acting as a headless CMS, can be configured to create a relational database that links every simulation run to its specific product requirement. This creates an immutable, queryable record that satisfies the most demanding auditor. Automation workflows can even trigger validation comparisons automatically when a simulation is completed, flagging discrepancies in real-time. This shift from siloed simulation files to a connected data ecosystem is the new standard for production-ready compliance.

Conclusion: The Evolution of Simulation in Regulatory Science

The validation and certification of simulation models is moving away from a final "check-box" activity and becoming an integrated, continuous thread throughout the product lifecycle. As Digital Twins become more prevalent, we will see a shift from static certification to "continuous validation," where a model is perpetually assessed against sensor data from the real-world product.

Organizations that invest in robust V&V frameworks, embrace quantitative validation metrics, and leverage centralized data management will be best positioned to navigate this complexity. By treating simulation certification as a continuous process of evidence building rather than a one-time audit, teams can unlock the full potential of simulation-driven design while maintaining the highest standards of safety and regulatory compliance.