Senior Software Engineer - Python, Data Engineering, AI

About Zenoti

Role overview

Responsibilities

• Design, build, and operate batch and incremental ETL/ELT pipelines in Python using Glue Python-shell, PySpark, AWS Batch/Docker, or Lambda.
• Model curated and Iceberg tables; tune Trino/Athena SQL and manage lake partitioning, compaction, and cost/performance.
• Build external API ingestion with incremental anchors, retries, and idempotent writes.
• Extend data-quality and anomaly-detection checks; own alerting and on-call for pipeline health.
• Lead Databricks-on-Azure lakehouse work with Delta Lake, Databricks Workflows/Jobs, Unity Catalog, and migration of existing curation logic.
• Ship tested changes through pull requests, infrastructure as code, and CI/CD; participate in code and design reviews.
• Partner with Product, Finance, and Customer Success analysts on metrics and expose documented datasets to QuickSight and the AI/MCP layer.
• Mentor junior engineers and raise standards for testing, observability, and documentation.

Required qualifications

• 5–7 years of production data engineering experience building, deploying, and operating pipelines used by other teams; analyst, BI-developer, and drag-and-drop ETL-only experience does not count.
• Strong Python 3, including pandas/pyarrow, packaging, virtual environments, testable modules, and shared libraries.
• At-scale Apache Spark/PySpark skills, including DataFrame API, partitioning, joins/skew, shuffle tuning, and Parquet.
• Advanced distributed-engine SQL, including window functions, CTEs, incremental/merge patterns, and query-plan-level tuning.
• Lakehouse knowledge of Parquet, partitioning, and hands-on Apache Iceberg or Delta Lake experience, including schema evolution, time travel, compaction/OPTIMIZE, or MERGE INTO.
• AWS or Azure data-platform experience covering object storage, managed/serverless compute, a catalog, and IAM/RBAC basics.
• DAG orchestration with dependencies, retries, and failure alerting; incremental API/database ingestion; Git, pull requests, and CI/CD.
• Data quality and observability practices, plus clear written communication for design documents, runbooks, and pull requests.

Preferred qualifications

• Databricks hands-on experience, including relevant lakehouse, catalog, workflow, or platform capabilities, is good to have.
• Azure data-stack experience and deeper AWS platform experience are good to have.
• Databricks Data Engineer Associate or Professional certification is a plus.