Rohan Pandey

Undergrad ML Researcher &
10x Hackathon Winner

Hi! I'm a junior at Carnegie Mellon University with research interests in Natural Language Processing, Reinforcement Learning, & Brain-Computer Interfaces. My past work includes conversational dialogue systems, cognitively realistic syntax parsing, and multimodal brainwave classification. My forthcoming work primarily focuses on linguistic RL and semantic composition.

Beyond research, much of my focus currently goes towards building a ramen profitable micro-SaaS to secure my intellectual freedom. I also enjoy reading Sanskrit philosophical literature, especially on language and epistemology.

Feel free to reach out via Twitter, LinkedIn, or email!

Awarded by

9/2021 - Present

NLP Researcher @ CMU Language Technologies Institute

Investigating compositional representations in language models
Leveraging semantic graphs for natural language inference
Working with Uri Alon and Prof. Graham Neubig in NeuLab

12/2020 - 9/2021

RL Researcher @ Language, Logic, & Cognition Lab

Applying reinforcement learning for cognitively realistic syntax parsing
Extending work to semantic parsing and text-to-SQL
Optimizing linguistic RL library under Prof. Adrian Brasoveanu

1/2021 - 8/2021

NLP Researcher @ Natural Language & Dialogue Systems Lab

Developing conversational language flows and user behavior models
Training sequence classification models under Kevin Bowden and Prof. Marilyn Walker
Official co-author and team member for Amazon Alexa Prize 2021

8/2020 - 11/2020

ML Engineering Consultant @ Bunch

Implemented TensorFlow.js computer vision models in the browser
Built React web app to calculate force exertion of humans on video
Product to be deployed in gyms to replace >$20k in equipment

6/2020 - 8/2021

BCI Researcher @ Intheon

Developing deep learning models for NLP + BCI
Due to research confidentiality, further details upon request

6/2020 - 9/2020

NLP Engineer @ SapientX

Fine-tuned PyTorch NLP models for extractive question answering (F1 >0.9)
Implemented neural and classical information retrieval approaches
Productionized with Flask REST API for core company product

3/2020 - 6/2021

President @ NeuroTechSC

Lead 5 teams (25 members) in building Brain-Computer Interface
Project utilizes subvocal recognition for synthetic telepathy
Organized neurotech curriculum for students
Held weekly paper readings of cutting-edge neurotech research

1/2020 - 5/2020

NLP Researcher @ Applied ML Lab

Architected high-dimensional document attention model
Collected and cleaned >7 million textual data points from Internet
Benchmarked our model on a mental health sentiment analysis task
Examined relevant academic literature and wrote preprint under Prof. Narges Norouzi

7/2018 - 5/2019

Fullstack Web Dev Consultant @ Nevaka

Built React app that dynamically loads thousands of database objects
Coordinated migration from Google Realtime Database to Cloud Firestore
Integrated Google Firebase authentication

Projects

fbIRL

Won 1st @ Facebook SF Dev Hackathon 2019

Tomorrow's AR social network

Celery

Won 2nd & FinTech @ LA Hacks 2019

Big data forecasting for sustainable businesses

veda.dev

Deployed with active users

Morphology visualizer for Sanskrit literature research & education

sWEep

Won 1st @ SRC Code 2018

Cleaning neighborhoods with CV

Boolepathy

Won 1st in US @ NeuroTechX 2020

Non-invasive synthetic telepathy

We & You

Won Google Cloud @ BASEHacks 2018

Peer-to-peer mental health services for teens

Latent Space

Won 3rd @ HackMIT 2020

Domain-specific neural audio compression for virtual bands

Phil

Won Amazon & Blockchain @ CruzHacks 2019

Facilitating blockchain donations with Alexa skill art

Research

I'm interested in questions like...

How do humans perform semantic composition and how can we build systems that analyze language compositionally? Transformers have outpaced virtually all other architectures in NLP; is this just due to higher generalizability or is something about the self-attention mechanism inherently effective at expressing semantic composition?

How do humans ground language in their environment and how can we build systems that understand language in relation to the real world? The current approach of learning word representations from a large text corpus has gone a long way, but it falls into a trap that can only be avoided by grounding language. Could linguistic RL agents be a solution?

What is the underlying relationship between symbolic and statistical approaches? Why do some parts of nature seem so perfectly described by symbolic relations while others don't? Is reality fundamentally symbolic or are symbols a formalism that humans apply to our environment?

And a few miscellaneous ones: What makes specifically human brains so good at manipulating symbols, genetically, structurally, and culturally? How does the brain represent non-linguistic thoughts and is all perception symbolic at some level? How can classical theories from linguistics and philosophy of language aid modern research in NLP? Is internality an inherent property of matter?

Publications

Reinforcement Learning to Jointly Encode Prompts and Database Schemas for Text-to-SQL Semantic Parsing

Under Review at AAAI 2022

Rohan Pandey

A Family of Cognitively Realistic Parsing Environments for Deep Reinforcement Learning

Under Review at NeurIPS Deep RL Workshop 2021

Adrian Brasoveanu, Rohan Pandey, Maximilian Alfano-Smith

Athena 2.0: Discourse and User Modeling in Open Domain Dialogue

Proceedings of Amazon Alexa Prize Socialbot Grand Challenge 4

Omkar Patil, Lena Reed, Kevin K. Bowden, Juraj Juraska, Wen Cui, Vrindavan Harrison, Rishi Rajasekaran, Angela Ramirez, Cecilia Li, Eduardo Zamora, Phillip Lee, Jeshwanth Bheemanpally, Rohan Pandey, Adwait Ratnaparkhi, Marilyn Walker

Transfer Learning for Mental Health Evaluation from Natural Language

Preprint

Kamil Kisielewicz, Rohan Pandey, Shivansh Rustagi, Narges Norouzi

Fun Facts

I'm a philosophy enthusiast and consider myself a student of the Mīmāṃsā school of Indian thought.
I greatly value financial freedom and aim to produce $5k MRR from a micro-SaaS by the end of 2022.
I enjoy building constructed languages, including a 2-channel parallelized experimental lang and a modern descendant of Sumerian.
I've been teaching myself Mandarin Chinese and Classical Sanskrit for about 3 years. I'm fluent in English, Hindi, and Spanish.
I'm the former VP of UCSC's AI club, SCAI, where I taught deep learning, NLP, and computational genomics.
My interests are all over the place, but I try to concentrate on 3 areas and their intersections: ML, Linguistics, and Neuroscience.
My mathematics research knowledge is minimal but I'm fascinated by graph theory and topological spaces.
My current reading list includes Arthasaṁgraha (Bhāskara), Gödel, Escher, Bach (Hofstadter), and The Feeling of Life Itself (Koch).
The source code for this site is based on Andrej Karpathy's personal website.
My inbox is open for discussions!