
## Project Overview

This overarching project of many published journal articles addresses key challenges in sampling complex biological (and generically dynamical) systems. We integrated physics-based simulations with machine learning, particularly leveraging generative models to improve molecular phase space sampling (again, as well as sampling of dynamical systems generically) by generating initial conditions that are consistent with specific values of pertinent collective variables (CVs) of the system of interest. 

If these CVs were not known already, we identify them with manifold learning. We also show that enhanced sampling methods can be used directly with CVs in embedding (lower dimensional manifold) space. The manifold learning method used in these works to get to that lower dimensional embedding space are diffusion maps (and geometric harmonics to get from embedding space back to the ambient space). Additionally, we developed generative modeling frameworks that combine diffusion models and manifold learning to efficiently sample complex data distributions. 

I continued this work in the context of materials discovery for NASA’s Biological and Physical Sciences division, establishing a novel AI framework for generating alloy compositions with target properties Utilizing a framework that can be thought of as a VAE wrapper around a Diffusion model. 

The articles that comprise this work are shortly and informally described below. Please give some of them a further read if the summaries interest you.

---

## GANs and Closures

The seminal work in this project, this study explores how combining physics-based simulation methods with machine learning techniques, like conditional generative adversarial networks (cGANs), can enhance sampling in complex molecular systems by generating realistic high-dimensional states from simpler low-dimensional representations. The approach leverages known system characteristics or learns important features using tools like diffusion maps, using them to bias the generated states to specific regions in collective variable (CV) space. This is also the paper that shows you can bias enhanced sampling methods directly with embedded CVs. You do not have to bias in the physical/ambient space of the system!

[View details and external link to paper](../research/gans-closures)

---

## Micro-macro consistency in multiscale modeling: Score-based model assisted sampling of fast/slow dynamical systems

The second work extends the work of GANs and Closures to new classes of generative models, i.e. score-based diffusion models (abbreviated here as SGMs). This study explores how combining physics-based enhanced sampling techniques with machine learning tools like SGMs can help tackle the challenges of efficiently sampling multiscale dynamical systems.

[View details and external link to paper](../research/micro-macro)

---

## Generative Learning for Slow Manifolds and Bifurcation Diagrams

This paper proposes a framework leveraging conditional score-based generative models (cSGMs) and nonlinear dimensionality reduction techniques to efficiently approximate slow manifolds and construct bifurcation diagrams in multiscale dynamical systems. The approach circumvents fast transient dynamics and identifies steady states by directly initializing on the low-dimensional slow manifold of a system.

[View details and external link to paper](../research/gen-bifurcations)

---

## Generative Learning of Densities on Manifolds

This study introduces and validates a framework for generating data distributions on high-dimensional manifolds using modified score-based generative models (m-SGMs). The model leverages nonlinear dimensionality reduction methods to address challenges in high-dimensional uncertainty quantification. Applications in computational mechanics demonstrate the robustness and efficiency of the proposed approach over conventional methods.

[View details and external link to paper](../research/gen-densities)

---

## Inverse Design of Alloys via Generative Algorithms: Optimization and Diffusion within Learned Latent Space

Using the same principals for a NASA and materials science adjacent problem, this study introduces a generative AI framework using variational autoencoders (VAEs) and SGMs as inverse design methods to efficiently discover new alloy compositions with desired properties, tackling challenges like limited data, solution nonuniqueness, and complex design spaces. By enabling smooth mapping of properties and generating diverse compositions, the approach shows promise for accelerating the identification of desired alloy compositions.

[View details and external link to paper](../research/inverse-alloys)


A collection of papers demonstrating that generative model architectures (such as GANs and diffusion models) can efficiently sample and explore the phase space of various dynamical systems including molecular conformational space described by collective variables (that are found with diffusion maps or known beforehand), bifurcation diagrams, low dimensional manifolds of high dimensional systems, and more.

Generative Model Assisted Sampling of Dynamical Systems


# Generative Learning of Densities on Manifolds

## Research Summary

This study introduces and validates a framework for generating data distributions on high-dimensional manifolds using modified score-based generative models (m-SGMs). The model leverages nonlinear dimensionality reduction methods to address challenges in high-dimensional uncertainty quantification. Applications in computational mechanics demonstrate the robustness and efficiency of the proposed approach over conventional methods.

## Motivation

High-dimensional data encountered in multiscale computational mechanics, like stress-strain relationships in complex materials, presents challenges for accurate and efficient modeling. The framework developed in this paper aims to address:

1. The curse of dimensionality in simulations and sampling strategies.
2. Generative modeling of data constrained to manifolds defined by experimental and synthetic results.
3. Incorporation of physics-based features in generative approaches to ensure consistency with observed phenomena.

By employing manifold learning and score-based techniques, the paper seeks to improve the fidelity of synthetic data generation and its applicability to real-world computational mechanics problems.

## Methodology

### Framework Details

1. **Modified Score-Based Generative Models (m-SGMs)**:
   - Adapt score-based algorithms to directly model multiscale material systems.
   - Integrate physics-based parameters to maintain correspondence with real data.

2. **Double Diffusion Maps**:
   - Utilize these as a dimensionality reduction tool to parameterize manifolds, facilitating sampling in high-dimensional spaces.

3. **KL Divergence Performance Metrics**:
   - Statistical comparison of the generated distributions against ground truth to quantify model precision.

### Dataset Used

- 800-point simulation comparing tensile and bending test outcomes, structured into 449 feature dimensions.
- Dimensions include microscale, mesoscale, and macroscale properties.

### Techniques Applied

To generate additional realizations for high-dimensional synthetic datasets, the framework employs m-SGMs combined with double diffusion maps, surpassing typical generation techniques in efficiency and quality.

## Results

### Applications

1. **Synthetic Stress-Strain Data**:
   - Generated distributions accurately mimic observed stress-strain curves from laboratory tests.
   - Validated results include accurate reproductions under bending and tensile configurations.

2. **Dimensionality Reduction**:
   - Experimental data constrained within manifolds showed strong agreement with real-world outcomes.
   - Double diffusion maps provided significant enhancements to sampling precision.

3. **Performance Analysis**:
   - Among methods evaluated, m-SGM1 produced the lowest KL divergence, indicating the highest fidelity to true data distributions.

### Computational Metrics

- Improved runtime performance relative to standard Monte Carlo Simulations (MCS).
- Increased sampling precision across multiscale dimensional embeddings.

## Key Insights

- **Generative Modeling vs. MCS**:
   - Modified approaches reduce computational complexity while improving fidelity.
- **Physics-Based Integration**:
   - Ensures synthetic data aligns with physical constraints of observed phenomena.
- **Scalability**:
   - Applicability expands to larger, complex datasets by leveraging manifold-parametrization techniques.

## Future Work

1. Expanding to stochastic dynamics and chaotic systems.
2. Improving framework scalability for higher-dimensional contexts.
3. Creating publicly accessible benchmarks for generative models designed for physical systems modeling.


This paper introduces a generative framework integrating modified score-based models and manifold learning techniques for the approximation and generation of high-dimensional data distributions, particularly in multiscale computational mechanics.

Generative Learning of Densities on Manifolds


# Generative Learning for Slow Manifolds and Bifurcation Diagrams

## Research Summary

This paper proposes a framework leveraging conditional score-based generative models (cSGMs) and nonlinear dimensionality reduction techniques to efficiently approximate slow manifolds and construct bifurcation diagrams in multiscale dynamical systems. The approach circumvents fast transient dynamics and identifies steady states by directly initializing on the low-dimensional slow manifold of a system.

## Motivation

In systems characterized by multiple time scales, slow manifolds capture the long-term dynamics and are crucial for model reduction. Approximating these manifolds accurately improves the efficiency of dynamical simulations. Traditional numerical methods for identifying steady states or bifurcation points in such systems often require significant computational resources.

By using cSGMs conditioned on quantities of interest (QoIs), the proposed framework generates samples directly on the slow manifold, enabling faster and more precise model initialization. Additionally, the framework addresses challenges in filling missing segments of bifurcation diagrams and reduces computational effort in identifying steady states.

## Methodology

### Conditional Score-Based Generative Models (cSGMs)

- Trained cSGMs to generate consistent data samples conditioned on QoIs, which represent desired labels on the slow manifold or bifurcation surfaces.
- Utilized these samples to approximate slow-manifold geometries and steady-state distributions in dynamical systems.

### Diffusion Maps and Manifold Learning

- Implemented nonlinear dimensionality reduction via diffusion maps to reduce stochasticity and improve sampling in deterministic dynamical systems.
- Applied this method to identify QoIs that parameterize the slow manifold from system data.

### Framework for Initialization and Steady-State Identification

- Combined cSGMs and diffusion map outputs to initialize simulations directly on the slow manifold, bypassing transient state exploration.
- Demonstrated the framework's efficiency in constructing steady-state solutions and bifurcation diagrams under varying parameters.

## Results

### Applications and Validation

1. **Steady-State Identification**:
   - Efficiently approximated steady states in bifurcation diagrams for out-of-sample parameter values.
   - Reduced computational overhead compared to traditional initialization approaches.

2. **Bifurcation Analysis**:
   - Identified bifurcation points and geometric properties of reduced manifolds with high precision.

3. **Extrapolation Performance**:
   - Demonstrated scalability and generalization capabilities in dynamic systems with large parameter spaces.

4. **Application Examples**:
   - Validated the framework using a reaction–diffusion partial differential equation (PDE).
   - Explored its utility in a plug-flow tubular reactor system, highlighting improved initialization over classical numerical methods.

### Computational Efficiency

- Sidestepped transient phases in dynamical simulations, achieving significant reductions in simulation time after the model training phase.
- Enhanced phase space exploration through integration with physics-based simulations.

## Key Insights

- **Multiscale Advantages**: Facilitates switching between fine-scale and reduced-order coarse-scale simulations without needing extensive transient stabilization.
- **Manifold Parameterization**: Demonstrated the efficacy of combining data-driven and physics-based approaches in identifying low-dimensional embeddings.
- **Versatility**: Broad applicability to systems governed by ODEs and PDEs with separation of timescales.
- **Extrapolation Robustness**: Effectively fills in missing segments of bifurcation diagrams and handles parameter extrapolation.

## Future Work

- Extending the integration of cSGMs with other manifold learning techniques.
- Enhancing scalability for applications in high-dimensional systems.
- Exploring additional stochastic and chaotic dynamical systems.
- Releasing public benchmarks and datasets to enable broader adoption of the proposed methods.

**Dataset and Code**: Available at https://github.com/ecrab/generative_learning_slow_manifolds




This paper introduces a novel framework combining conditional score-based generative models and diffusion maps for efficient initialization on slow manifolds and identification of steady states in dynamical systems.

|

Data Scientist and Assistant Professor
Johns Hopkins All Children's Hospital and School of Medicine

Featured Work

Generative Model Assisted Sampling of Dynamical Systems

Generative Learning of Densities on Manifolds

Generative Learning for Slow Manifolds and Bifurcation Diagrams

|

Data Scientist and Assistant ProfessorJohns Hopkins All Children's Hospital and School of Medicine

Featured Work

Generative Model Assisted Sampling of Dynamical Systems

Generative Learning of Densities on Manifolds

Generative Learning for Slow Manifolds and Bifurcation Diagrams

Data Scientist and Assistant Professor
Johns Hopkins All Children's Hospital and School of Medicine