Software Engineer | AI Training Data & Evals Lab ðŸ§
Role Overview
Broad builder role combining platform engineering with evaluation and experimentation infrastructure.
Design, build, and operate production systems used by researchers and operators.
The company is scaling pipelines, evaluation harnesses, and training environments.
Success in the first 30 to 90 days includes shipping a meaningful production improvement and becoming the owner of a core system.
Key Skills for This Role
Full Job Posting
Company overview
The client is a fast-growing applied research lab building the data layer for frontier AI.
It provides expert-generated datasets and evaluation and benchmarking for AI labs and enterprises.
The company is fully remote and works on training signals, software engineering agent evaluations, finance workflows, and measured data quality.
Founded in 2025 and backed by a CAD 30M Series A.
Role overview
Broad builder role combining platform engineering with evaluation and experimentation infrastructure.
Design, build, and operate production systems used by researchers and operators.
The company is scaling pipelines, evaluation harnesses, and training environments.
Success in the first 30 to 90 days includes shipping a meaningful production improvement and becoming the owner of a core system.
Responsibilities
- Build and maintain evaluation harnesses measuring model and agent performance on real tasks.
- Improve evaluation reliability, coverage, and signal quality through rubrics, task design support, and scoring.
- Ship reusable tools for researchers and operators to run experiments.
- Build APIs and backend services for human-in-the-loop workflows, task routing, and quality checks.
- Improve pipelines that convert expert work into structured training and evaluation data.
- Improve observability, scalability, and operability through logging, metrics, and debugging.
- Write maintainable code, take part in code reviews and design discussions, and document decisions.
- Build and run production systems that teams depend on.
Required qualifications
- Strong coding fundamentals in Node.js and TypeScript.
- Strong coding ability in Python and/or Go.
- Experience building and owning production APIs, services, and pipelines.
- Solid understanding of distributed systems and engineering trade-offs.
- Comfort with AWS or GCP, containers, and Kubernetes.
- Track record of shipping and maintaining systems others rely on, beyond prototypes.
- Strong written communication and comfort working asynchronously in a distributed team.
Bonus experience
- Evaluation frameworks, experimentation platforms, or ML tooling experience is a bonus.
- Data pipelines or workflow orchestration experience is a bonus.
- Building internal platforms for operators or research teams is a bonus.
- Early-stage or high-ownership B2B SaaS or platform team experience is a bonus.
Compensation and employment
- Full-time role.
- Compensation is listed as CAD 7,000–10,000 USD per month; the currency wording is inconsistent in the source.
Workplace and team
- Fully remote role with a LATAM focus and meaningful overlap with U.S. time zones.
- The team is lean and async-first and values clear writing, sound judgment, and follow-through.
Interview process
The process includes a take-home engineering and communication assignment, application review, a founding engineer screen, a real-work trial, and an offer stage.
Apply for this job in 1 click
Skip the repetitive application forms
Install the Base Career Chrome Extension and autofill job applications across major job boards with your profile.
Trusted by over 500,000 job seekers on Base Career