greenhouse

Senior Data Platform Engineer

Komodo Health

San Francisco, USA

Senior

Onsite

Discovered 2 days ago

Apply

Full Job Posting

Key Responsibilities

Build, operate, and optimize large-scale batch pipelines using Python, SQL, Spark, and Airflow on AWS
Make storage and data layout decisions (table formats, partitioning, file sizing, compaction) that keep large datasets fast to query and cost-efficient
Ingest data from many external sources and keep pipelines correct through frequent schema changes
Transform healthcare claims, EHR, and reference data into trusted, serving-ready data products for customers and internal teams
Strengthen reliability through data quality checks, validation, lineage, observability, and alerting
Debug complex data, system, and performance issues across compute-heavy workflows
Contribute to system design, code quality, testing, documentation, CI/CD, and rotational production support
What you bring to Komodo Health:
Strong hands-on experience building, owning, and debugging production data pipelines at multi-terabyte scale
Advanced Python, including performant, memory-efficient code for large-scale data processing, plus strong SQL
Deep production experience with Airflow, including operating and scaling it, not just writing DAGs
A solid understanding of how data is physically stored: columnar vs. row formats, and open table formats like Iceberg, Delta Lake, or Hudi (updates, compaction, small files, partitioning)
Experience with Spark or a comparable distributed processing framework
Proven experience designing and operating data systems in AWS
Strong instincts for data correctness, root-cause analysis, and production troubleshooting
The ability to communicate technical trade-offs clearly with engineering, product, and data partners
Comfort using AI-assisted engineering tools for coding, debugging, documentation, and technical exploration
AI-Augmented Engineering Expectations:
You will be expected to leverage AI-augmented engineering tools, such as ChatGPT, Gemini, or Claude, to improve productivity and technical decision-making. This may include using AI to generate and refine code, accelerate documentation, automate test case creation, debug complex issues, explore unfamiliar technical concepts, and assess architectural trade-offs and risks.
Additional skills and experience we’d prioritize (nice to have)...
Containers and orchestration (Docker, Kubernetes)
Strong core software engineering fundamentals and experience in more than one language
Data governance and lineage experience
Experience delivering external-facing data products through APIs, serving layers, or customer access patterns
Experience applying AI or agentic workflows to engineering, data quality, delivery, or operations
Success in high-growth or ambiguous environments
No healthcare background required. You’ll build that domain expertise here.
Open to US remote, OR SF/NYC hybrid
#LIRemote
The pay range for each job posting reflects a minimum and maximum range of annual base pay that we reasonably expect to pay for this position within the US. We carefully consider multiple business-related factors when determining compensation, including job-related skills, work experience, geographic work location, relevant training and certifications, business needs and market demands.
The starting annual base pay for this role is listed below. This position may be eligible for performance-based bonuses as determined in the Company’s sole discretion and in accordance with a written agreement or plan. This role may also be eligible for equity awards. In addition, this role is eligible for benefits including, but not limited to, comprehensive health, dental, and vision insurance; flexible time off and holidays; 401(k) with company match; disability insurance and life insurance; and leaves of absence in accordance with applicable state and local laws and regulations and company policy.
San Francisco Bay Area and New York City:
$196,000 — $230,000 USD
All Other US Locations:
$170,000 — $200,000 USD
Komodo's AI Standard
At Komodo, we're not just witnessing the AI revolution – we're leading it. This is a pivotal moment in time, where being first to market with AI transforms industries and sets the bar. We've already established industry leadership in leveraging AI to revolutionize healthcare, and we expect every team member to contribute. AI here isn't optional; it's foundational. We expect you to integrate AI into your daily work – from summarizing documents to automating workflows and uncovering insights. This isn't just about efficiency; it's about making every moment more meaningful, building on trust in AI, and driving our collective success.
Join us in shaping the future of healthcare intelligence. Where You’ll Work
Komodo Health has a hybrid work model with hubs in San Francisco, New York City, and Chicago. Roles vary — some can be performed from anywhere in the country, others are scoped to a specific region, and some are based near one of our hubs. For hub-based Dragons, we're building intentional in-office rhythms alongside the flexibility that's core to how we work. Whatever your setup, expectations will always be clear before you join.
Equal Opportunity Statement
Komodo Health provides equal employment opportunities to all applicants and employees. We prohibit discrimination and harassment of any type with regard to race, color, religion, age, sex, national origin, disability status, genetics, protected veteran status, sexual orientation, gender identity or expression, or any other characteristic protected by federal, state, or local laws.
By submitting your application, you acknowledge that you have read and understand Komodo Health’s Privacy Notice for Employees and Contractors.
This notice explains how we collect, use, and retain applicant data.
Join us in shaping the future of healthcare intelligence.
Where You’ll Work

Qualifications

Experience with Spark or a comparable distributed processing framework

Apply for this job in 1 click

Skip the repetitive application forms

Install the Base Career Chrome Extension and autofill job applications across major job boards with your profile.

Trusted by over 500,000 job seekers on Base Career

Start Free Today

More jobs at Komodo Health

Account Executive - AVP, Sales (Emerging)

, USA

Executive

Discovered 1 weeks agoView →

Sales Development Representative

New York City, USA

Junior

We Breathe Life Into Data At Komodo Health, our mission is to reduce the global burden of disease. And we believe that smarter use of data is essential to this mission. That’s why we built the Healthcare Map — the indust

Discovered 2 weeks agoView →

Senior Salesforce Solutions Consultant

, USA

Senior

Discovered 2 weeks agoView →

Principal Product Manager, AI Products

, USA

Manager

Discovered 2 weeks agoView →

Staff Software Engineer, Query Engine [Rust]

, USA

Senior

Discovered 3 weeks agoView →

Senior AI Engagement Manager

, USA

Senior

Discovered 3 weeks agoView →