Gyeonghun Kang

Gyeonghun Kang강경훈 · 姜暻勋

Hi there! I am a fourth-year Ph.D. candidate in Statistical Science at Duke University, advised by Simon Mak and Xiang Cheng.

I study how neural network architectures and their components drive model performance across tasks such as language reasoning, image and video generation, and tabular prediction. I first formulate the task and network, then use experiments, mechanistic analysis, and causal interventions to understand what shapes the network's capabilities. These insights guide architectural design and adaptation.

I am currently developing new sampling methods for language models, building on my work on uncertainty quantification for neural operators.

Before joining Duke, I earned an M.S. in Statistics and Data Science and a B.A. in Economics from Yonsei University. Contact me about research or collaboration.

Contactgk149@duke.edu

Follow

Gyeonghun Kang standing in a library

News

Selected papers

All papers →
  1. Three-by-five grid of DiT generation samples: AAHQ, FFHQ, and AFHQ target rows; source (000), QK-only replacement (100), MLP-only replacement (001), VO+MLP replacement (011), and fully fine-tuned (111) columns.
    Component swaps after fine-tuning CelebA to AAHQ, FFHQ, or AFHQ (top to bottom). QK denotes query/key projections that set attention weights; VO denotes value/output projections that process attended features. Codes indicate source (0) or fine-tuned (1) weights in QK/VO/MLP order. Replacing MLPs shifts appearance toward the target domain.

    How attention and MLPs divide the work of denoising

    • Attention gathers relevant patch context, while MLPs predict clean patches from learned image representations.
    • This view leads to networks that share memory across layers while maintaining or improving generation quality in our experiments.
    • Fine-tuning experiments show that changes to MLPs drive shifts in appearance, while attention transfers across image domains with similar structure.

    These findings guide what to share and what to adapt when building and fine-tuning generative models.

  2. Image, Noise, and EDM clean-image denoising losses on ImageNet256, with Noise and EDM lower bounds under an input bottleneck.
    ImageNet256 denoising with an input bottleneck (epoch 200). Noise prediction tracks its theoretical lower bound; EDM (JiT scaling) alleviates the burden at high noise, approaching image prediction. Legend reports guided FID-10K (lower is better).

    Why image prediction can outperform noise prediction

    • A noise predictor must preserve the noisy input patch to cancel corruption. Architectural bottlenecks can discard this information and create unavoidable denoising error.
    • Our theory and ImageNet experiments explain the advantage of image prediction in JiT, how additional model width helps, and why EDM preconditioning alleviates this burden at high noise levels.

    These results provide principles for matching prediction targets, patch dimensions, and model capacity.

  3. Pretrained PFN and GP regression posterior predictive means and 5% and 95% quantiles for Sacramento housing prices.
    Posterior predictive means and 5th/95th percentiles for Sacramento housing prices. Pretrained PFN (top), GP regression (bottom); cyan × marks indicate context data.

    How network depth improves in-context Bayesian prediction

    • We show how transformers can perform Bayesian prediction in context: attention layers implement iterative gradient descent, and an MLP converts the results into a predictive distribution.
    • Our theory and experiments explain how depth and normalization help prior-data fitted networks (PFNs) make accurate predictions on datasets larger than those seen during pretraining.

    This gives depth a concrete role as additional inference steps and provides architectural principles for scaling tabular foundation models.