Rohan Pandey

Google Scholar · GitHub · Twitter · Email

I'm currently taking a short break from frontier AI research to solve OCR for Sanskrit. I feel it critical that the classical Indian literary canon be immortalized in the training corpus for superintelligence.

At OpenAI, I helped train GPT-5 and future models. I graduated from Carnegie Mellon University in 2023 with an honors thesis on semantics in multimodal LLMs.

Experience

• OpenAI: Explored model architecture questions spanning Pre-training, RL, and Inference
• Reworkd (YC S23): Built a multimodal web agent generating 5k lines of code weekly
• Microsoft AI: Fine-tuned language models to automate enterprise-scale data annotation

Selected Publications

• gzip Predicts Data-dependent Scaling Laws (ArXiv 2024)
• Multimodal Learning Without Multimodal Data: Guarantees and Applications (ICLR 2024)
• Towards Vision-Language Mechanistic Interpretability: a Causal Tracing Tool for BLIP (ICCV 2023 - CLVL)
• Cross-modal Attention Congruence Regularization for Vision-Language Relation Alignment (ACL 2023)
• Syntax-guided Neural Module Distillation to Probe Compositionality in Sentence Embeddings (EACL 2023)
• A Family of Cognitively Realistic Parsing Environments for Deep Reinforcement Learning (NeurIPS 2021 - Deep RL)

Featured Projects

• LlamaGym: Fine-tune LLM agents with online reinforcement learning
• Tarsier: Vision utilities for agents to interact with the web
• veda.dev: Morphology visualizer for Sanskrit literature research & education

Fun Facts

• Run a biweekly Sanskrit reading group in San Francisco. Please reach out if interested!
• Forked ved/acc from e/acc in 2023
• Lived at AGI House SF, a hacker house in Twin Peaks, for a year until September 2024
• Taught a Classical Indian Philosophy course at Carnegie Mellon University
• Conlanging in middle school led me to linguistics, and consequently to NLP & Sanskrit