Rohan Pandey

Twitter · GitHub · Google Scholar · Email

I Take the Bitter Lesson Seriously, so I'm teaching machines to do science at Periodic Labs.

At OpenAI, I helped train GPT-5 & GPT-6 class models. I graduated from Carnegie Mellon University in 2023 with an honors thesis on semantics in multimodal LLMs.

Experience

• OpenAI: Explored model architecture questions spanning Pre-training, RL, and Inference
• Reworkd (YC S23): Built evals & harnesses for computer-use agents
• Microsoft AI: Distilled & fine-tuned language models for on-device inference

Selected Publications

• gzip Predicts Data-dependent Scaling Laws (ArXiv 2024)
• Multimodal Learning Without Multimodal Data: Guarantees and Applications (ICLR 2024)
• Towards Vision-Language Mechanistic Interpretability: a Causal Tracing Tool for BLIP (ICCV 2023 - CLVL)
• Cross-modal Attention Congruence Regularization for Vision-Language Relation Alignment (ACL 2023)
• Syntax-guided Neural Module Distillation to Probe Compositionality in Sentence Embeddings (EACL 2023)
• A Family of Cognitively Realistic Parsing Environments for Deep Reinforcement Learning (NeurIPS 2021 - Deep RL)

Featured Projects

• LlamaGym: Fine-tune LLM agents with RLVR (6 months before o1)
• Tarsier: Vision utilities for agents to interact with the web
• veda.dev: Morphology visualizer for Sanskrit literature research & education

Fun Facts

• Run a biweekly Sanskrit reading group in San Francisco. Please reach out if interested!
• Worked on OCR for Sanskrit to immortalize the classical Indian literary canon in the training corpus for superintelligence
• Taught a Classical Indian Philosophy course at Carnegie Mellon University
• Conlanging in middle school led me to linguistics, and subsequently to NLP & Sanskrit