Builder 3

Role Overview

Core Responsibilities

• Define and enforce the standard for when an agent can access production data.
• Build an evaluation harness and integrate it into CI as a merge-blocking quality gate.
• Design guardrails for regulated and financial workflows, including human approval and rollback paths.
• Run design reviews, own agent incidents, make framework and model decisions, and mentor other builders.

Candidate Profile

• Candidates should have shipped an agentic system to real users and changed its architecture after an unexpected failure.
• Candidates should design state management, tool interfaces, failure semantics, reliability thresholds, and rollback paths.
• Candidates should be rigorous about evaluation results, failure modes, and production-grade handling of financial data.
• Candidates should be comfortable analyzing large undocumented legacy schemas and translating business rules into system constraints.

Technical Stack

• The stack includes Python, SQL with Postgres or SQL Server, TypeScript, and OpenAI or Anthropic models.
• The role may use pgvector or Qdrant, LangChain or LangGraph, DSPy, dbt, Apache Hop or Airflow, and Docker.
• The stack also includes Weights & Biases, OpenTelemetry, evaluation and tracing platforms, policy tooling, model routing, and caching.

Ownership Areas

• The guardrail layer includes validation, tool permissions, financial and PII handling, sandboxing, approval gates, kill switches, and blast-radius containment.
• The evaluation framework includes golden ERP datasets, offline and online evaluations, calibrated LLM judging, and regression detection.
• The cost and performance envelope includes model routing, caching, context budgets, latency budgets, and transaction-level cost tracking.

Success Indicators

Working Environment