
# Generative Learning for Slow Manifolds and Bifurcation Diagrams

## Research Summary

This paper proposes a framework leveraging conditional score-based generative models (cSGMs) and nonlinear dimensionality reduction techniques to efficiently approximate slow manifolds and construct bifurcation diagrams in multiscale dynamical systems. The approach circumvents fast transient dynamics and identifies steady states by directly initializing on the low-dimensional slow manifold of a system.

## Motivation

In systems characterized by multiple time scales, slow manifolds capture the long-term dynamics and are crucial for model reduction. Approximating these manifolds accurately improves the efficiency of dynamical simulations. Traditional numerical methods for identifying steady states or bifurcation points in such systems often require significant computational resources.

By using cSGMs conditioned on quantities of interest (QoIs), the proposed framework generates samples directly on the slow manifold, enabling faster and more precise model initialization. Additionally, the framework addresses challenges in filling missing segments of bifurcation diagrams and reduces computational effort in identifying steady states.

## Methodology

### Conditional Score-Based Generative Models (cSGMs)

- Trained cSGMs to generate consistent data samples conditioned on QoIs, which represent desired labels on the slow manifold or bifurcation surfaces.
- Utilized these samples to approximate slow-manifold geometries and steady-state distributions in dynamical systems.

### Diffusion Maps and Manifold Learning

- Implemented nonlinear dimensionality reduction via diffusion maps to reduce stochasticity and improve sampling in deterministic dynamical systems.
- Applied this method to identify QoIs that parameterize the slow manifold from system data.

### Framework for Initialization and Steady-State Identification

- Combined cSGMs and diffusion map outputs to initialize simulations directly on the slow manifold, bypassing transient state exploration.
- Demonstrated the framework's efficiency in constructing steady-state solutions and bifurcation diagrams under varying parameters.

## Results

### Applications and Validation

1. **Steady-State Identification**:
   - Efficiently approximated steady states in bifurcation diagrams for out-of-sample parameter values.
   - Reduced computational overhead compared to traditional initialization approaches.

2. **Bifurcation Analysis**:
   - Identified bifurcation points and geometric properties of reduced manifolds with high precision.

3. **Extrapolation Performance**:
   - Demonstrated scalability and generalization capabilities in dynamic systems with large parameter spaces.

4. **Application Examples**:
   - Validated the framework using a reaction–diffusion partial differential equation (PDE).
   - Explored its utility in a plug-flow tubular reactor system, highlighting improved initialization over classical numerical methods.

### Computational Efficiency

- Sidestepped transient phases in dynamical simulations, achieving significant reductions in simulation time after the model training phase.
- Enhanced phase space exploration through integration with physics-based simulations.

## Key Insights

- **Multiscale Advantages**: Facilitates switching between fine-scale and reduced-order coarse-scale simulations without needing extensive transient stabilization.
- **Manifold Parameterization**: Demonstrated the efficacy of combining data-driven and physics-based approaches in identifying low-dimensional embeddings.
- **Versatility**: Broad applicability to systems governed by ODEs and PDEs with separation of timescales.
- **Extrapolation Robustness**: Effectively fills in missing segments of bifurcation diagrams and handles parameter extrapolation.

## Future Work

- Extending the integration of cSGMs with other manifold learning techniques.
- Enhancing scalability for applications in high-dimensional systems.
- Exploring additional stochastic and chaotic dynamical systems.
- Releasing public benchmarks and datasets to enable broader adoption of the proposed methods.

**Dataset and Code**: Available at https://github.com/ecrab/generative_learning_slow_manifolds




This paper introduces a novel framework combining conditional score-based generative models and diffusion maps for efficient initialization on slow manifolds and identification of steady states in dynamical systems.

Generative Learning for Slow Manifolds and Bifurcation Diagrams


# Generative Learning of Densities on Manifolds

## Research Summary

This study introduces and validates a framework for generating data distributions on high-dimensional manifolds using modified score-based generative models (m-SGMs). The model leverages nonlinear dimensionality reduction methods to address challenges in high-dimensional uncertainty quantification. Applications in computational mechanics demonstrate the robustness and efficiency of the proposed approach over conventional methods.

## Motivation

High-dimensional data encountered in multiscale computational mechanics, like stress-strain relationships in complex materials, presents challenges for accurate and efficient modeling. The framework developed in this paper aims to address:

1. The curse of dimensionality in simulations and sampling strategies.
2. Generative modeling of data constrained to manifolds defined by experimental and synthetic results.
3. Incorporation of physics-based features in generative approaches to ensure consistency with observed phenomena.

By employing manifold learning and score-based techniques, the paper seeks to improve the fidelity of synthetic data generation and its applicability to real-world computational mechanics problems.

## Methodology

### Framework Details

1. **Modified Score-Based Generative Models (m-SGMs)**:
   - Adapt score-based algorithms to directly model multiscale material systems.
   - Integrate physics-based parameters to maintain correspondence with real data.

2. **Double Diffusion Maps**:
   - Utilize these as a dimensionality reduction tool to parameterize manifolds, facilitating sampling in high-dimensional spaces.

3. **KL Divergence Performance Metrics**:
   - Statistical comparison of the generated distributions against ground truth to quantify model precision.

### Dataset Used

- 800-point simulation comparing tensile and bending test outcomes, structured into 449 feature dimensions.
- Dimensions include microscale, mesoscale, and macroscale properties.

### Techniques Applied

To generate additional realizations for high-dimensional synthetic datasets, the framework employs m-SGMs combined with double diffusion maps, surpassing typical generation techniques in efficiency and quality.

## Results

### Applications

1. **Synthetic Stress-Strain Data**:
   - Generated distributions accurately mimic observed stress-strain curves from laboratory tests.
   - Validated results include accurate reproductions under bending and tensile configurations.

2. **Dimensionality Reduction**:
   - Experimental data constrained within manifolds showed strong agreement with real-world outcomes.
   - Double diffusion maps provided significant enhancements to sampling precision.

3. **Performance Analysis**:
   - Among methods evaluated, m-SGM1 produced the lowest KL divergence, indicating the highest fidelity to true data distributions.

### Computational Metrics

- Improved runtime performance relative to standard Monte Carlo Simulations (MCS).
- Increased sampling precision across multiscale dimensional embeddings.

## Key Insights

- **Generative Modeling vs. MCS**:
   - Modified approaches reduce computational complexity while improving fidelity.
- **Physics-Based Integration**:
   - Ensures synthetic data aligns with physical constraints of observed phenomena.
- **Scalability**:
   - Applicability expands to larger, complex datasets by leveraging manifold-parametrization techniques.

## Future Work

1. Expanding to stochastic dynamics and chaotic systems.
2. Improving framework scalability for higher-dimensional contexts.
3. Creating publicly accessible benchmarks for generative models designed for physical systems modeling.


This paper introduces a generative framework integrating modified score-based models and manifold learning techniques for the approximation and generation of high-dimensional data distributions, particularly in multiscale computational mechanics.

Generative Learning of Densities on Manifolds


# Inverse Design of Alloys via Generative Algorithms: Optimization and Diffusion within Learned Latent Space

## Research Summary

Inverse design—generating material compositions that exhibit target properties—presents a promising avenue for accelerating novel material discovery. Artificial intelligence (AI) has emerged as a powerful tool for inverse learning. However, several challenges exist in developing AI tools for the inverse generation of novel alloys: limited availability of training data; nonuniqueness of solutions, where multiple compositions yield the same properties; and the complexity of navigating high-dimensional design space. This work proposes a novel generative AI framework that addresses these challenges by integrating a variational autoencoder (VAE) with two inverse design approaches: latent gradient optimization and latent diffusion model. The VAE establishes a compact low-dimensional latent space that enables smooth composition–property mapping, while both inverse methods generate compositions that match target properties. The optimization approach iteratively refines latent representations to search for optimal design, whereas the diffusion model progressively denoises latent variables to generate diverse and valid compositions. Both methods successfully generate alloy compositions with targeted properties, while the diffusion model outperforms in distribution-level reconstruction when both models are tested under the same conditions. Trained and validated on the Granta Alloys dataset, the proposed inverse generative framework provides a robust and scalable approach for discovering novel alloy composites and generating multiple feasible solutions.


This paper introduces a novel hybrid VAE/Diffusion Model framework for generating metal alloy compositions that are consistent with prescribed macroscopic properties.

Inverse Design of Alloys via Generative Algorithms: Optimization and Diffusion within Learned Latent Space


# Micro-macro consistency in multiscale modeling: Score-based model assisted sampling of fast/slow dynamical systems

## Research Summary

A valuable step in the modeling of multiscale dynamical systems in fields such as computational chemistry, biology, and materials science is the representative sampling of the phase space over long time scales of interest; this task is not, however, without challenges. For example, the long term behavior of a system with many degrees of freedom often cannot be efficiently computationally explored by direct dynamical simulation; such systems can often become trapped in local free energy minima. In the study of physics-based multi-time-scale dynamical systems, techniques have been developed for enhancing sampling in order to accelerate exploration beyond free energy barriers. On the other hand, in the field of machine learning (ML), a generic goal of generative models is to sample from a target density, after training on empirical samples from this density. Score-based generative models (SGMs) have demonstrated state-of-the-art capabilities in generating plausible data from target training distributions. Conditional implementations of such generative models have been shown to exhibit significant parallels with long-established—and physics-based—solutions to enhanced sampling. These physics-based methods can then be enhanced through coupling with the ML generative models, complementing the strengths and mitigating the weaknesses of each technique. In this work, we show that SGMs can be used in such a coupling framework to improve sampling in multiscale dynamical systems.


This work extends the work from GANs and Closures to score-based diffusion models, using them to initialize simulations of dynamical systems at specified regions that are defined with manifold learning.

Micro-macro consistency in multiscale modeling: Score-based model assisted sampling of fast/slow dynamical systems


# GANs and closures: Micro-macro consistency in multiscale modeling

## Research Summary

Sampling the phase space of molecular systems—and, more generally, of complex systems effectively modeled by stochastic differential equations (SDEs)—is a crucial modeling step in many fields, from protein folding to materials discovery. These problems are often multiscale in nature: they can be described in terms of low-dimensional effective free energy surfaces parametrized by a small number of “slow” reaction coordinates; the remaining “fast” degrees of freedom populate an equilibrium measure conditioned on the reaction coordinate values. Sampling procedures for such problems are used to estimate effective free energy differences as well as ensemble averages with respect to the conditional equilibrium distributions; these latter averages lead to closures for effective reduced dynamic models. Over the years, enhanced sampling techniques coupled with molecular simulation have been developed; they often use knowledge of the system order parameters in order to sample the corresponding conditional equilibrium distributions, and estimate ensemble averages of observables. An intriguing analogy arises with the field of machine learning (ML), where generative adversarial networks (GANs) can produce high-dimensional samples from low-dimensional probability distributions. This sample generation is what in equation-free multiscale modeling is called a “lifting process”: it returns plausible (or realistic) high-dimensional space realizations of a model state, from information about its low-dimensional representation. In this work, we elaborate on this analogy, and we present an approach that couples physics-based simulations and biasing methods for sampling conditional distributions with ML-based conditional generative adversarial networks (cGANs) for the same task. The “coarse descriptors” on which we condition the fine scale realizations can either be known a priori or learned through nonlinear dimensionality reduction (here, using diffusion maps). We suggest that this may bring out the best features of both approaches: we demonstrate that a framework that couples cGANs with physics-based enhanced sampling techniques can improve multiscale SDE dynamical systems sampling, and even shows promise for systems of increasing complexity (here, simple molecules).


This paper introduces a framework for using generative models to initialize simulations at locations in phase space prescribed by collective variables (CV) identified with manifold learning. We also show that traditional enhanced sampling methods can be directly biased by CVs constructed with diffusion maps.

GANs and closures: Micro-macro consistency in multiscale modeling


# Molecular Transport Behavior of CO2 in Ionic Polyimides and Ionic Liquid Composite Membrane Materials

## Research Summary

Ionic polyimides (i-PI) are a new class of polymer materials that are very promising for CO2 capture membranes, and recent experimental studies have demonstrated their enhanced separation performance with the addition of imidazolium-based ionic liquids (ILs). However, there is very little known about the molecular-level interactions in these systems, which give rise to interesting gas adsorption and diffusion characteristics. In this study, we use a combination of Monte Carlo and molecular dynamics simulations to analyze the equilibrium and transport properties of CO2 molecules in the i-PI and i-PI + IL composite materials. The addition of several different common ILs are modeled, which have a plasticization effect on the i-PI, lowering the glass transition temperature (Tg). The solubility of CO2 strongly correlates with the Tg, but the diffusion demonstrates more unpredictable behavior. At low concentrations, the IL has a blocking effect, leading to reduced diffusion rates. However, as the IL surpasses a threshold value, the relationship is inverted and the IL has a facilitating effect on the gas transport. This behavior is attributed to the simultaneous contributions of the increased i-PI plasticization at higher IL concentrations (facilitating gas hopping rates from cavity-to-cavity) and the increased IL continuity throughout the system, enabling more favorable transport pathways for CO2 diffusion.


This publication analyzes the transport behavior of CO2 in ionic polyimides and ionic polyimide + ionic liquid composites.

Molecular Transport Behavior of CO2 in Ionic Polyimides and Ionic Liquid Composite Membrane Materials


# Molecular analysis of selective gas adsorption within composites of ionic polyimides and ionic liquids as gas separation membranes

## Research Summary

The CO2 separation characteristics of ionic polyimides (i-PIs) are modeled using molecular dynamics simulations in combination with grand canonical Monte Carlo calculations. The performance of neat i-PI systems is evaluated, as well as composite structures containing both i-PIs and various ionic liquids (ILs). The i-PI + IL composites are based on combinations of 1-n-butyl-3-methylimidazolium ([C4mim+]) cations with three common molecular anions: (bis(trifluoromethylsulfonyl)imide ([Tf2N−]), tetrafluoroborate ([BF4−]), and hexafluorophosphate ([PF6−]). It is found that 50 mol% IL inclusion can increase CO2/CH4 selectivity by 16% in [BF4−]-based materials and by 36% in [PF6−]-based materials from mixtures of 5% CO2/95% CH4. While the [BF4−]-based system shows higher CO2/CH4 selectivity, the [Tf2N−]-based system shows higher CO2/N2 gas selectivity. A comprehensive structural analysis (fractional free volume (FFV), pore size distribution, surface area, etc.) is used to highlight the underlying differences among the different i-PI + IL systems that lead to the different adsorption properties.


This publication analyzes the selective gas adsorption behavior of CO2 in ionic polyimides and ionic polyimide + ionic liquid composites.

Molecular analysis of selective gas adsorption within composites of ionic polyimides and ionic liquids as gas separation membranes


# Solubility and diffusivity of CO2 in ionic polyimides with [C(CN)3]x[oAc]1−x anion composition

## Research Summary

Recently, mixtures of ionic liquids (ILs) containing acetate [oAc−] and tricyanomethanide [C(CN)3−] anions have demonstrated promising characteristics as solvents and as CO2 adsorbents. The anion composition can be optimized to obtain a significant improvement in the diffusivity while only slightly decreasing the solubility. Here, we explore a similar investigation applied to an ionic polyimide (i-PI) material in order to understand the molecular-level kinetic and thermodynamic characteristics that emerge when a polymeric material is used instead. The i-PI systems studied are part of an emerging class of “high-performance” ionene polymers that hold significant potential for applications in gas separation membranes. Here, neat i-PI systems with the two counter anions, [oAc−] and [C(CN)3−], at varying concentrations are modeled by a combination of molecular dynamics (MD) and grand canonical Monte Carlo (GCMC) simulations. Higher concentrations of [C(CN)3−] are predicted to improve CO2 diffusion, similar to the pure IL performance, while solubility remains relatively unchanged. The structural characteristics of the i-PI systems provide detailed insight into the effect that the anions have on the adsorption properties. The solubility is weakly related to the theoretical surface area, and the diffusivity is moderately correlated to the fractional free volume (FFV). Overall, the combination of different anions is predicted to be a viable strategy for improving the diffusivity throughout i-PI materials, but the behavior of pure ILs cannot be simply extrapolated to ion behavior in membranes.


This paper investigates the solubility of ionic polyimide materials in ionic liquid solvents and how this impacts adsorption capabilities.

Solubility and diffusivity of CO2 in ionic polyimides with [C(CN)3]x[oAc]1−x anion composition


# Molecular Simulation of Ionic Polyimides and Composites with Ionic Liquids as Gas-Separation Membranes

## Research Summary

Polyimides are at the forefront of advanced membrane materials for CO2 capture and gas-purification processes. Recently, ionic polyimides (i-PIs) have been reported as a new class of condensation polymers that combine structural components of both ionic liquids (ILs) and polyimides through covalent linkages. In this study, we report CO2 and CH4 adsorption and structural analyses of an i-PI and an i-PI + IL composite containing [C4mim][Tf2N]. The combination of molecular dynamics (MD) and grand canonical Monte Carlo (GCMC) simulations is used to compute the gas solubility and the adsorption performance with respect to the density, fractional free volume (FFV), and surface area of the materials. Our results highlight the polymer relaxation process and its correlation to the gas solubility. In particular, the surface area can provide meaningful guidance with respect to the gas solubility, and it tends to be a more sensitive indicator of the adsorption behavior versus only considering the system density and FFV. For instance, as the polymer continues to relax, the density, FFV, and pore-size distribution remain constant while the surface area can continue to increase, enabling more adsorption. Structural analyses are also conducted to identify the nature of the gas adsorption once the ionic liquid is added to the polymer. The presence of the IL significantly displaces the CO2 molecules from the ligand nitrogen sites in the neat i-PI to the imidazolium rings in the i-PI + IL composite. However, the CH4 molecules move from the imidazolium ring sites in the neat i-PI to the ligand nitrogen atoms in the i-PI + IL composite. These molecular details can provide critical information for the experimental design of highly selective i-PI materials as well as provide additional guidance for the interpretation of the simulated adsorption systems.


This publication analyzes the general adsorption behavior of CO2 in ionic polyimides and ionic polyimide + ionic liquid composites with both molecular dynamics and grand canonical Monte Carlo simulations.

|