Senior Software Engineer, Machine Learning Infrastructure & Automation
Role Overview
Help fal's ML team move faster by building the automation, infrastructure, and developer tooling that makes developing, testing, and deploying generative AI models seamless.
You'll own and improve the CI/CD systems supporting our rapidly growing collection of ML models and inference pipelines.
Your focus will be on eliminating manual work, accelerating development cycles, and building reliable systems that allow ML engineers to ship new models and optimizations with confidence.
This is a high-impact engineering role where you'll work closely with our Applied ML and ML Performance teams.
You'll build everything from automated model validation and performance benchmarking to AI-powered development workflows that help engineers iterate faster.
The ideal candidate thinks beyond traditional CI/CD and sees automation as a force multiplier for the entire engineering organization.
What you’ll do:
• Own ML CI/CD infrastructure Design, build, and maintain automated testing, validation, and deployment pipelines for our ML models and inference services.
• Accelerate development cycles.Dramatically reduce CI execution times through intelligent parallelization, caching, test selection, and efficient use of compute resources.
• Build automated model validation.Develop systems that test model outputs, detect quality regressions, and validate changes across different models, GPU architectures, and configurations.
• Automate performance benchmarking. Build continuous performance testing that detects regressions in inference latency, throughput, GPU utilization, and cost.
• Build automated pricing and deployment checks. Ensure model pricing, billing configurations, API schemas, and deployments are validated automatically before reaching production.
• Extend our agentic engineering workflows. Develop AI-powered automation and agentic coding systems that automatically diagnose CI failures, identify regressions, propose fixes, and streamline engineering workflows.
• Improve deployment reliability. Build automated safeguards, deployment verification, rollback mechanisms, and monitoring to ensure new model releases are reliable.
• Eliminate engineering toil. Identify repetitive tasks across the ML team and build tools and systems that automate them, allowing engineers to focus on developing new models and improving performance.
Key Skills for This Role
Full Job Posting
About this role:
Help fal's ML team move faster by building the automation, infrastructure, and developer tooling that makes developing, testing, and deploying generative AI models seamless.
You'll own and improve the CI/CD systems supporting our rapidly growing collection of ML models and inference pipelines. Your focus will be on eliminating manual work, accelerating development cycles, and building reliable systems that allow ML engineers to ship new models and optimizations with confidence.
This is a high-impact engineering role where you'll work closely with our Applied ML and ML Performance teams. You'll build everything from automated model validation and performance benchmarking to AI-powered development workflows that help engineers iterate faster.
The ideal candidate thinks beyond traditional CI/CD and sees automation as a force multiplier for the entire engineering organization.
What you’ll do
Own ML CI/CD infrastructure Design, build, and maintain automated testing, validation, and deployment pipelines for our ML models and inference services.
Accelerate development cycles.Dramatically reduce CI execution times through intelligent parallelization, caching, test selection, and efficient use of compute resources.
Build automated model validation.Develop systems that test model outputs, detect quality regressions, and validate changes across different models, GPU architectures, and configurations.
Automate performance benchmarking. Build continuous performance testing that detects regressions in inference latency, throughput, GPU utilization, and cost.
Build automated pricing and deployment checks. Ensure model pricing, billing configurations, API schemas, and deployments are validated automatically before reaching production.
Extend our agentic engineering workflows. Develop AI-powered automation and agentic coding systems that automatically diagnose CI failures, identify regressions, propose fixes, and streamline engineering workflows.
Improve deployment reliability. Build automated safeguards, deployment verification, rollback mechanisms, and monitoring to ensure new model releases are reliable.
Eliminate engineering toil. Identify repetitive tasks across the ML team and build tools and systems that automate them, allowing engineers to focus on developing new models and improving performance.
Qualifications/Nice-to-haves:
- 5+ years of software engineering background with proficiency in Python and experience building production infrastructure and developer tooling.
- Experience designing and operating CI/CD systems using GitHub Actions or comparable technologies.
- Deep understanding of automated testing, build systems, dependency management, caching, and parallel execution.
- Experience working with containerized workloads, Docker, and cloud infrastructure.
- Ability to design reliable distributed systems and debug complex infrastructure failures.
- Strong understanding of observability, including logs, metrics, tracing, and automated alerting.
- A passion for developer productivity and a demonstrated ability to eliminate manual processes through automation.
- Comfortable working independently, identifying high-impact problems, and building end-to-end solutions.
- Experience with ML infrastructure, PyTorch, GPU workloads, or model-serving systems.
- Familiarity with NVIDIA GPU architectures and multi-GPU environments.
- Experience building automated inference benchmarks or ML quality evaluation frameworks.
- Experience with agentic coding tools such as Codex or Claude Code, or building custom AI engineering agents.
- Experience optimizing CI/CD pipelines at scale, including distributed test execution and ephemeral compute environments.
- Experience developing internal developer platforms or infrastructure-as-code tooling.
Tech Stack
Python, PyTorch, Docker, GitHub Actions
NVIDIA GPUs and distributed GPU infrastructure
Model serving and inference pipelines
Observability and performance monitoring tools
AI coding agents and automated engineering workflows
Access to fal's massive GPU cluster for testing and development
What we offer at fal
- Interesting and challenging work
- A lot of learning and growth opportunities
- Health, dental, and vision insurance (US)
- Regular team events and offsites
U.S. EQUAL EMPLOYMENT OPPORTUNITY INFORMATION
fal provides equal employment opportunities to applicants and employees without regard to race, color, religion, sex, sexual orientation, gender identity, national origin, protected veteran status, disability, or any other classification protected by applicable law.
About fal
Generative media platform for developers providing APIs and infrastructure for image, video, audio, and 3D models.
Visit company websiteJobs and hiring trendsApply for this job in 1 click
Skip the repetitive application forms
Install the Base Career Chrome Extension and autofill job applications across major job boards with your profile.
Trusted by over 500,000 job seekers on Base Career
More from this employer
More jobs at fal
Marketing Operations
San Francisco, USA
fal is the generative media ecosystem powering the next generation of AI products. We build the infrastructure, tools, and model access that teams need to move from idea to production, and do it at scale without compromi
Influencer Marketing Manager
San Francisco, USA
fal is the generative media ecosystem powering the next generation of AI products. We build the infrastructure, tools, and model access that teams need to move from idea to production, and do it at scale without compromi
Infrastructure Accounting Lead
San Francisco, USA
fal is the generative media ecosystem powering the next generation of AI products. We build the infrastructure, tools, and model access that teams need to move from idea to production, and do it at scale without compromi
Senior/Staff Software Engineer, Forward Deployed
San Francisco, USA
Software Engineer, Machine Learning-Backend
San Francisco, USA
fal is the generative media ecosystem powering the next generation of AI products. We build the infrastructure, tools, and model access that teams need to move from idea to production, and do it at scale without compromi
Marketing Operations
San Francisco, USA
Compute Capacity Acquisition
San Francisco, USA
Compute Deals
San Francisco, USA
Influencer Marketing Manager
San Francisco, USA
Infrastructure Accounting Lead
San Francisco, USA
Senior/Staff Software Engineer, Forward Deployed
San Francisco, USA
Software Engineer, Machine Learning-Backend
San Francisco, USA
Event Marketing Manager
San Francisco, USA