Galactic Alchemy I:

Domain Transfer with Generative AI for Hydrodynamical Simulations

SKA research at
Zurich University of Applied Sciences (ZHAW)

Centre for Artificial Intelligence (CAI)
Institute for Business Information Technology (IWI)
Sept 4, 2024
contact_qr.png Philipp Denzel, Yann Billeter, Frank-Peter Schilling, Elena Gavagnin

Slides on my website

https://phdenzel.github.io/

talk_qr.png

Outlook

Motivation

  • teaching machines to emulate physics is cool!
    • benefit for fields like gravitational lensing
  • SKA-MID (0.35 GHz - 15 GHz, lower redshifts):
    • between 0.04" - 0.70" resolution (with baseline ~ 150km)
    • significant substructure in flux distributions
    • enable new perspective on star-formation as well as AGN

Cooganetal2023.jpg

Figure 1: simulation of ∼0.04 deg2 region of GOODS-North by Coogan et al. (2023)

Cooganetal2023_zoom.jpg

Figure 2: zoom of previous figure Coogan et al. (2023)

The old way of modelling galaxies

real_gal-inv.png

Figure 3: Mandelbaum et al. (2014)

What problems come with this

  • simple models work for simple galaxies, but we will often see:
    • no more blobs, no more Gaussian signals
  • not physical models:
    • difficult to infer physical properties
  • galaxy modelling has to evolve:
    • e.g., with data-driven methods

More advanced models

  • complex, realistic models
  • self-consistent dynamics
  • physics: on a wide range of scales
  • implicit models:
    • what if we want to fit them
      to an observation?

TNG300_compilation_with_radio_halos_2k.png

Figure 4: IllustrisTNG simulations

Multi-domain galaxy image dataset

Our goal:

"Infuse deep learning map-to-map translation models
with the physical model from simulations."

  • Question: can we infer unseen properties in observations?

Dataset from IllustrisTNG

  • projected TNG50-1 galaxies
  • 6 domains: dark-matter, stars, gas,
    HI, temperature, magnetic field
  • ∼ 2'000 galaxies, 6 snapshots,
    5 rotations in 3D, ∼ 360'000 images
  • each galaxy \(\ge\) 10'000 particles
  • scale: 2 baryonic half-mass radii

domains.png

Dataset from IllustrisTNG

  • projected TNG50-1 galaxies
  • 6 domains: dark-matter, stars, gas,
    HI, temperature, magnetic field
  • ∼ 2'000 galaxies, 6 snapshots,
    5 rotations in 3D, ∼ 360'000 images
  • each galaxy \(\ge\) 10'000 particles
  • scale: 2 baryonic half-mass radii

domains_directions.png

Generative Deep Learning

  • Image-to-image translation solves the inverse problem:
    \( \color{#f48193}{y} = A\color{#81f4a9}{x} + b \)
  • in Bayesian terms: \( p(\color{#81f4a9}{x}|\color{#f48193}{y}) \propto p(\color{#f48193}{y}|\color{#81f4a9}{x}) \,\, p(\color{#81f4a9}{x}) \)
  • \( p(\color{#f48193}{y}|\color{#81f4a9}{x}) \) is the data likelihood including the physics
  • \( p(\color{#81f4a9}{x}) \) is our prior knowledge on the solution.
  • MAP solution: \( \hat{x} = \arg \max_{x} \log p(\color{#f48193}{y}|\color{#81f4a9}{x}) + \log p(\color{#81f4a9}{x}) \)
  • explicitly sampling from the posterior distribution is difficult and expensive!

Generative Deep Learning architectures


Benchmark of generative models we're investigating:

Generative Deep Learning architectures


  • Inversion by Direct Iteration (InDI) models: similar to DDPMs,
    but more efficient at inference
  • Score-based diffusion models (SDMs): promising results,
    score gives direct access to the posterior likelihoods
  • Diffusion Mamba: the latest and greatest?

cGANs

pix2pix_schema.png

Figure 5: pix2pix scheme

DDPM

skais_indi_rnd_dm_no_formula.png

Main component: U-Net

U-Net.png

Figure 6: U-Net following Ronneberger et al. (2015)

Essential changes to U-Net blocks

"Attention is (almost) all you need!"

  • for better feature selection

attn_block.png

Results

  • all evaluated on a hold-out set
  • still somewhat preliminary…

Gas ⟶ DM: Massive halo

074baffb63a1.eval_batch.02.in.01.png

Figure 7: Input

074baffb63a1.eval_batch.02.pred.01.png

Figure 8: Output (pix2pix with Attention U-Net)

074baffb63a1.eval_batch.02.gt.01.png

Figure 9: Ground truth

Gas ⟶ DM: Spiral galaxy

074baffb63a1.eval_batch.12.in.00.png

Figure 10: Input

074baffb63a1.eval_batch.12.pred.00.png

Figure 11: Output (pix2pix with Attention U-Net)

074baffb63a1.eval_batch.12.gt.00.png

Figure 12: Ground truth

Gas ⟶ DM: Merger

074baffb63a1.eval_batch.14.in.01.png

Figure 13: Input

074baffb63a1.eval_batch.14.pred.01.png

Figure 14: Output (pix2pix with Attention U-Net)

074baffb63a1.eval_batch.14.gt.01.png

Figure 15: Ground truth

Profiles of DM column density

074baffb63a1_profiles_DM.png

074baffb63a1_cumulatives_DM.png

Profile residuals

074baffb63a1_residuals_DM.png

Gas ⟶ stars: High turbulence

f046843763c5.eval_batch.07.in.02.png

Figure 16: Input

f046843763c5.eval_batch.07.pred.02.png

Figure 17: Output (pix2pix with Attention U-Net)

f046843763c5.eval_batch.07.gt.02.png

Figure 18: Ground truth

Gas ⟶ stars: Mergers

f046843763c5.eval_batch.24.in.01.png

Figure 19: Input

f046843763c5.eval_batch.24.pred.01.png

Figure 20: Output (pix2pix with Attention U-Net)

f046843763c5.eval_batch.24.gt.01.png

Figure 21: Ground truth

Gas ⟶ stars: Irregular shape

diffusion_gas->dm_in_65681_a70c486921e405c6c534.png

Figure 22: Input

diffusion_gas->dm_pred_65681_5536c4565178d4c470a5.png

Figure 23: Output (standard DDPM)

diffusion_gas->dm_gt_65681_f72b986fed1618e14a84.png

Figure 24: Ground truth

"Abundance matching"

abundance_matching_074baffb63a1.f046843763c5.png

Figure 25: model using pix2pix+Attention

abundance_matching_074baffb63a1.f046843763c5_true.png

Figure 26: data

Gas ⟶ HI

e26dca2b6859.eval_batch.06.in.03.png

Figure 27: Input

e26dca2b6859.eval_batch.06.pred.03.png

Figure 28: Output (pix2pix with Attention U-Net)

e26dca2b6859.eval_batch.06.gt.03.png

Figure 29: Ground truth

Gas ⟶ HI: Massive halo

e26dca2b6859.eval_batch.00.in.09.png

Figure 30: Input

e26dca2b6859.eval_batch.00.pred.09.png

Figure 31: Output (pix2pix with Attention U-Net)

e26dca2b6859.eval_batch.00.gt.09.png

Figure 32: Ground truth

Profile residuals

e26dca2b6859_residuals_HI.png

Gas ⟶ B-field:

22186b9f64d6.eval_batch.24.in.00.png

Figure 33: Input

22186b9f64d6.eval_batch.24.pred.00.png

Figure 34: Output (pix2pix with Attention U-Net)

22186b9f64d6.eval_batch.24.gt.00.png

Figure 35: Ground truth

Next steps

  • paper in prep. (stay tuned)
  • test more architectures
  • improve observation mocks using Karabo
  • analogue with point clouds in 3D
    • problem: scaling to larger clouds

Contact

https://phdenzel.github.io/

talk_qr.png



Email: philipp.denzel@zhaw.ch

References

Created by phdenzel.