Generative Model Assisted Sampling of Dynamical Systems
Project Overview
This overarching project of many published journal articles addresses key challenges in sampling complex biological (and generically dynamical) systems. We integrated physics-based simulations with machine learning, particularly leveraging generative models to improve molecular phase space sampling (again, as well as sampling of dynamical systems generically) by generating initial conditions that are consistent with specific values of pertinent collective variables (CVs) of the system of interest.
If these CVs were not known already, we identify them with manifold learning. We also show that enhanced sampling methods can be used directly with CVs in embedding (lower dimensional manifold) space. The manifold learning method used in these works to get to that lower dimensional embedding space are diffusion maps (and geometric harmonics to get from embedding space back to the ambient space). Additionally, we developed generative modeling frameworks that combine diffusion models and manifold learning to efficiently sample complex data distributions.
I continued this work in the context of materials discovery for NASA’s Biological and Physical Sciences division, establishing a novel AI framework for generating alloy compositions with target properties Utilizing a framework that can be thought of as a VAE wrapper around a Diffusion model.
The articles that comprise this work are shortly and informally described below. Please give some of them a further read if the summaries interest you.
GANs and Closures
The seminal work in this project, this study explores how combining physics-based simulation methods with machine learning techniques, like conditional generative adversarial networks (cGANs), can enhance sampling in complex molecular systems by generating realistic high-dimensional states from simpler low-dimensional representations. The approach leverages known system characteristics or learns important features using tools like diffusion maps, using them to bias the generated states to specific regions in collective variable (CV) space. This is also the paper that shows you can bias enhanced sampling methods directly with embedded CVs. You do not have to bias in the physical/ambient space of the system!
View details and external link to paper
Micro-macro consistency in multiscale modeling: Score-based model assisted sampling of fast/slow dynamical systems
The second work extends the work of GANs and Closures to new classes of generative models, i.e. score-based diffusion models (abbreviated here as SGMs). This study explores how combining physics-based enhanced sampling techniques with machine learning tools like SGMs can help tackle the challenges of efficiently sampling multiscale dynamical systems.
View details and external link to paper
Generative Learning for Slow Manifolds and Bifurcation Diagrams
This paper proposes a framework leveraging conditional score-based generative models (cSGMs) and nonlinear dimensionality reduction techniques to efficiently approximate slow manifolds and construct bifurcation diagrams in multiscale dynamical systems. The approach circumvents fast transient dynamics and identifies steady states by directly initializing on the low-dimensional slow manifold of a system.
View details and external link to paper
Generative Learning of Densities on Manifolds
This study introduces and validates a framework for generating data distributions on high-dimensional manifolds using modified score-based generative models (m-SGMs). The model leverages nonlinear dimensionality reduction methods to address challenges in high-dimensional uncertainty quantification. Applications in computational mechanics demonstrate the robustness and efficiency of the proposed approach over conventional methods.
View details and external link to paper
Inverse Design of Alloys via Generative Algorithms: Optimization and Diffusion within Learned Latent Space
Using the same principals for a NASA and materials science adjacent problem, this study introduces a generative AI framework using variational autoencoders (VAEs) and SGMs as inverse design methods to efficiently discover new alloy compositions with desired properties, tackling challenges like limited data, solution nonuniqueness, and complex design spaces. By enabling smooth mapping of properties and generating diverse compositions, the approach shows promise for accelerating the identification of desired alloy compositions.