{bc}
Apply
workday

Lead Software Development Engineer in Test

Alteryx
Remote, USA
Full-time
Lead
Remote
Discovered 5 days ago
alteryxjavascriptlangchainpythontypescript

Full Job Posting

Requirements

  • Experience evaluating AI agents, including multi-step task completion, tool selection, tool-call accuracy, and failure recovery.
  • Working knowledge of agent frameworks, preferably LangChain, with the ability to understand and debug context handling, tool execution, and agent state.
  • Hands-on experience with LangSmith or similar platforms for evaluation datasets, custom evaluators, experiments, and comparisons across agent changes.
  • Experience with agent observability, using traces and telemetry to investigate quality issues, latency, token usage, and cost.
  • Strong Python software engineering skills to build and extend evaluation tooling, metrics pipelines, and reporting systems. JavaScript or TypeScript experience is a nice-to-have.
  • Applied statistics and data analysis skills to assess variability, compare experiments, and distinguish meaningful improvements from noise; modeling experience is a plus.
  • Experience developing evaluation datasets and scoring methods, combining deterministic checks, human feedback, and LLM-based judges.
  • Ability to contribute at a technical lead level, guiding evaluation practices, mentoring engineers, integrating evaluations into CI/CD, and translating production failures into regression coverag
  • Compensation
  • Alteryx is committed to fair, equitable, and transparent compensation. Final compensation is determined by several factors, including but not limited to relevant work experience, education, certifications, skills, and geographic location.
  • A monthly Connectivity Plus stipend of $150 to support remote work-related expenses
  • An annual $200 home office reimbursement
  • Medical, dental, and vision coverage
  • 401(k) with company match
  • Paid parental leave, caregiver leave, and flexible time off
  • Mental health support and wellness reimbursement
  • Career development and education assistance
  • Find yourself checking a lot of these boxes but doubting whether you should apply? At Alteryx, we support a growth mindset for our associates through all stages of their careers. If you meet some of the requirements and you share our values, we encourage you to apply. As part of our ongoing commitment to a diverse, equitable, and inclusive workplace, we’re invested in building teams with a wide variety of backgrounds, identities, and experiences.
  • Alteryx has amazing benefits for all Associates which can be viewed here.
  • For roles in San Francisco and Los Angeles: Pursuant to the San Francisco Fair Chance Ordinance and the Los Angeles Fair Chance Initiative for Hiring, Alteryx will consider for employment qualified applicants with arrest and conviction records.
  • This position involves access to software/technology that is subject to U.S. export controls. Any job offer made will be contingent upon the applicant’s capacity to serve in compliance with U.S. export controls.

Preferred Qualifications

  • Strong Python software engineering skills to build and extend evaluation tooling, metrics pipelines, and reporting systems. JavaScript or TypeScript experience is a nice-to-have.
  • Applied statistics and data analysis skills to assess variability, compare experiments, and distinguish meaningful improvements from noise; modeling experience is a plus.
  • The base salary range for this role in the United States is $165,000-$185,000. This role is also eligible for a target annual bonus of 15% of base salary, based on individual and company performance.

Apply for this job in 1 click

Skip the repetitive application forms

Install the Base Career Chrome Extension and autofill job applications across major job boards with your profile.

Sarah M.James T.Maya R.

Trusted by over 500,000 job seekers on Base Career

Start Free Today

More from this employer

More jobs at Alteryx

Principal level enablement and product launch readiness across our Go-To-Market (GTM) organization

, USA

Discovered 2 days agoFull-time

Sales Enablement Professional – Sales Leadership

Colorado City, USA

Discovered 4 days agoFull-time

Senior Software Engineer - Cloud Platform & Data Plane Transport

, USA

Discovered 6 days agoFull-time

Software Engineer

Irvine, USA

Discovered 6 days agoFull-time

Staff Developer Platform Engineer

, USA

Discovered 1 weeks agoFull-time

Enterprise Account Executive (Montreal based/Quebec)

Québec, CAN

Discovered 1 weeks agoFull-time

Manager, Digital Sales - West

, USA

Discovered 1 weeks agoFull-time

Manager, Finance Analytics Engineering & Automation

, USA

Discovered 1 weeks agoFull-time