Research Engineer

Responsibilities

• Own the evaluation infrastructure that tests competing modeling approaches, so that new tests can be set up quickly, run concurrently and automatically, and be monitored.
• Implement measurement pipelines exactly as specified by the scientists, with deterministic and traceable outputs.
• Maintain versioned evaluation datasets with held-out sets that are never used for training.
• Build and maintain data pipelines for human-participant data, external datasets, and model training data, including linkage of records belonging to the same participant across sources.
• Keep research code in step with the production product so that validated improvements ship promptly.
• Detect and prevent threats to the validity of evaluations.
• Track compute and API spend across experiments and their implications for production-stage costs
• Manage repositories, code review, documentation, and the reproducibility of the team's coding workflow.
• Identify implementation choices that can be made more correct or more efficient, and propose changes.

Qualifications

• Several years of software engineering experience, including experience supporting researchers in a lab or comparable setting.
• Experience translating specific scientific requirements and experiment designs into code.
• Experience building and operating large-scale data pipelines on human behavior data, and confirming that the pipelines matched the criteria and design the researchers specified.
• Strong Python, including the ability to read and audit code produced by agentic coding tools.
• Fundamentals of computing: file systems, software libraries, databases, SQL, tabular data, and Git-based workflows.
• Experience with cloud infrastructure and with running many jobs concurrently.
• Working understanding of machine learning, including how transformer-based language models behave, how model APIs are used, and what fine-tuning involves.
• A habit of writing deterministic, reproducible, cost-aware, and well-documented code.
• Hands-on experience engineering large language model systems: prompt pipelines, evaluation harnesses, embedding computation, agentic systems, and/or fine-tuning infrastructure.
• Experience fine-tuning open-weight language models on multi-GPU hardware.
• Experience serving open-weight models with an inference framework such as vLLM or SGLang, including batched inference and prefix caching for many requests that share a common prompt.
• Experience with experiment tracking and cost monitoring for machine learning workloads.
• Experience with survey or panel data and data collection platforms.
• Experience extracting and storing model activations for downstream analysis.
• Experience with experiment tracking and cost monitoring for machine learning workloads, including GPU provisioning across cloud providers.
• Prior work on generative agents, digital twins, or simulation of human respondents.

Preferred

Preferred Qualifications

• Preferred