Staff Software Engineer, Data
Role Overview
Own the company data platform, including its AWS data lake, transformation layer, business-system integrations, and Kubernetes-based infrastructure.
Build data engineering pipelines for research teams across audio, video, image, and 3D, converting raw media into training-ready datasets.
The role is hands-on and has substantial ownership; it is not a data-preparation or analyst position.
Key Skills for This Role
Full Job Posting
Team and platform context
The Data team builds and runs systems that make company data usable for business operations and model training.
The team works with Finance, Marketing, SRE, Security, Product, and Research, emphasizing outcomes, technical access, and ownership.
The stated stack includes AWS, Iceberg, dbt, Postgres, Terraform, Helm, GitHub Actions, QuickSight, Stripe, Auth0, HubSpot, Customer.io, PostHog, OpenTelemetry, and SLURM or similar technologies.
Role overview
Own the company data platform, including its AWS data lake, transformation layer, business-system integrations, and Kubernetes-based infrastructure.
Build data engineering pipelines for research teams across audio, video, image, and 3D, converting raw media into training-ready datasets.
The role is hands-on and has substantial ownership; it is not a data-preparation or analyst position.
Company data platform responsibilities
- Own the S3/Athena/Glue data lake, dbt project and daily build, business-system and product-event ingestion, and Postgres serving layer for dashboards and AI agents.
- Integrate billing, CRM, ERP, identity, and telemetry systems so revenue, usage, cost, and customer identity reconcile automatically.
- Design warehouse dimensional models, product event data contracts, quality tests, lineage, and a data catalog.
- Enforce PII tiering through access grants in partnership with Security and Legal.
- Process billions of telemetry rows and terabyte-scale log streams; manage Iceberg tables, partitioning, and cost control.
- Own orchestration and infrastructure on EKS, Helm, and Terraform, including CI/CD, monitoring, alerting, and business-hours on-call for daily pipelines.
- Deliver reporting for Finance and leadership on revenue by business line, credits and prepaid liability, and cloud cost by department.
Research data responsibilities
- Source, license-check, ingest, and version large training datasets for audio, video, image, and 3D research.
- Build and scale generation, filtering, deduplication, and captioning pipelines that produce training-ready shards from raw or synthetic media.
- Run data workloads on SLURM and cloud object storage.
- Maintain dataset lineage, quality metrics, and documentation so researchers can reproduce training sets.
- Lead design reviews, mentor teammates, write runbooks, and partner across company-wide data initiatives.
Experience
- At least 6 years building and running production data platforms, including architecture ownership.
- Experience integrating SaaS business systems such as Stripe, HubSpot, an ERP, and an identity provider into a warehouse.
- Production experience preparing large multimodal training datasets and running batch workloads on HPC or cloud schedulers such as SLURM or Kubernetes.
- Comfort working independently in a small team with substantial ownership.
Skills and expertise
- Expert SQL and Python, with strong software engineering practices including tests, code review, and CI.
- Warehouse and lake modeling experience with Iceberg or Delta on S3, Athena/Trino, Snowflake or BigQuery, and dbt.
- Production experience with an orchestrator such as dbt, Airflow, or Argo and distributed processing such as Spark or equivalent.
- AWS, Kubernetes/EKS, Helm, and Terraform experience, including troubleshooting data jobs and deployments.
Additional strengths
- Prior 0-to-1 platform ownership at a startup is a plus.
- Research-team experience or hands-on exposure to audio, video, image, or 3D data is a plus.
- FinOps and cloud cost reporting experience is a plus.
- Familiarity with SOC 2 or similar data-access compliance requirements is a plus.
Benefits and compensation
- Health coverage is offered for employees and their families, with PPO and HMO options; select plans cover 100% of premiums for eligible employees and dependents.
- The source also begins a benefit item describing an HSA.
Apply for this job in 1 click
Skip the repetitive application forms
Install the Base Career Chrome Extension and autofill job applications across major job boards with your profile.
Trusted by over 500,000 job seekers on Base Career