Selected Writing

A curated selection of my technical writing, organized by topic. For the full archive, see the blog.

3D Vision & Neural Rendering

• 3D Gaussian Splatting for Real-Time Radiance Field Rendering
• DreamFusion: Text-to-3D using 2D Diffusion
• DreamGaussian: Generative Gaussian Splatting for Efficient 3D Content Creation
• Instant Neural Graphics Primitives with a Multiresolution Hash Encoding
• pixelNeRF: Neural Radiance Fields from One or Few Images
• SIREN: Implicit Neural Representations with Periodic Activation Functions
• Neural Fields as Learnable Kernels for 3D Reconstruction

Language Models & Transformers

• A Primer on the Inner Workings of Transformer-Based Language Models
• Retrieval Head Mechanistically Explains Long-Context Factuality
• ShortGPT: Layers in Large Language Models are More Redundant Than You Expect
• Is DPO Superior to PPO for LLM Alignment? A Comprehensive Study
• The Geometry of Categorical and Hierarchical Concepts in Large Language Models
• Mixture-of-Depths: Dynamically Allocating Compute in Transformer-Based Language Models

Foundations & Methods

• The Mathematics and Philosophy Behind MSE
• Effects of Scale on Model Finetuning
• Universal Language Learning Paradigms — UL2

Essays

• On Writing to Think