Associate Process Manager

Key Responsibilities:

• Design, develop, and maintain robust ETL/ELT pipelines using Databricks (Spark) and Python (PySpark). • Develop and optimize complex transformations using SQL (joins, window functions, CTEs, query tuning). • Build curated datasets and data models to support reporting, dashboards, and advanced analytics use cases. • Implement pipeline reliability best practices: data quality checks, monitoring, alerting, and reconciliation. • Optimize Databricks workloads for performance and cost (cluster sizing, partitioning strategies, caching, file formats). • Work with structured and semi-structured data (JSON, CSV, Parquet/Delta) and handle schema evolution. • Collaborate with stakeholders to understand business KPIs and deliver data solutions aligned to retail/eCommerce metrics (sales, orders, returns, inventory, customer cohorts). • Follow engineering best practices for version control (Git), documentation, reusable code patterns, and testing. • Good to have: Support or migrate Alteryx workflows into Python/Databricks pipelines.
• Must-Have Skills & Qualifications:
• 7-13 years of experience in Data Engineering / Data Warehousing / Big Data. • Strong hands-on experience with Databricks (Jobs/Workflows, notebooks, cluster concepts, Spark tuning fundamentals). • Strong programming skills in Python (PySpark preferred). • Excellent SQL skills, including performance tuning and writing complex analytical queries. • Experience building scalable pipelines and working with large datasets in distributed environments. • Strong understanding of data engineering concepts: ETL/ELT, orchestration, data validation, and observability. • Familiarity with modern data storage formats and practices (Delta/Parquet, partitioning, incremental loads).
• Good-to-Have Skills:
• Retail / eCommerce domain knowledge (customer behavior, funnel metrics, pricing/promotions, inventory, catalog, order lifecycle). • Alteryx (workflow development, optimization, scheduling, or migration to Databricks). • Experience with Lakehouse patterns and Delta Lake features (e.g., MERGE, OPTIMIZE, Z ORDER). • Experience with orchestration tools (e.g., Airflow, ADF, Databricks Workflows). • Cloud experience: AWS / Azure / GCP (S3/ADLS/GCS, IAM basics, security controls). • CI/CD exposure for data pipelines, code reviews, and automated deployments. Preferred Traits • Strong problem-solving skills and a mindset for root-cause analysis. • Ownership and accountability for production-grade pipelines. • Ability to communicate with both technical and non-technical stakeholders. • Comfort working in fast-paced environments with evolving requirements.
• Design, develop, and maintain robust ETL/ELT pipelines using Databricks (Spark) and Python (PySpark).
• Develop and optimize complex transformations using SQL (joins, window functions, CTEs, query tuning).
• Build curated datasets and data models to support reporting, dashboards, and advanced analytics use cases.
• Implement pipeline reliability best practices: data quality checks, monitoring, alerting, and reconciliation.
• Optimize Databricks workloads for performance and cost (cluster sizing, partitioning strategies, caching, file formats).
• Work with structured and semi-structured data (JSON, CSV, Parquet/Delta) and handle schema evolution.
• Collaborate with stakeholders to understand business KPIs and deliver data solutions aligned to retail/eCommerce metrics (sales, orders, returns, inventory, customer cohorts).
• Follow engineering best practices for version control (Git), documentation, reusable code patterns, and testing.
• Good to have: Support or migrate Alteryx workflows into Python/Databricks pipelines.
• 7-13 years of experience in Data Engineering / Data Warehousing / Big Data.
• Strong hands-on experience with Databricks (Jobs/Workflows, notebooks, cluster concepts, Spark tuning fundamentals).
• Strong programming skills in Python (PySpark preferred).
• Excellent SQL skills, including performance tuning and writing complex analytical queries.
• Experience building scalable pipelines and working with large datasets in distributed environments.
• Strong understanding of data engineering concepts: ETL/ELT, orchestration, data validation, and observability.
• Familiarity with modern data storage formats and practices (Delta/Parquet, partitioning, incremental loads).
• Retail / eCommerce domain knowledge (customer behavior, funnel metrics, pricing/promotions, inventory, catalog, order lifecycle).
• Alteryx (workflow development, optimization, scheduling, or migration to Databricks).
• Experience with Lakehouse patterns and Delta Lake features (e.g., MERGE, OPTIMIZE, Z ORDER).
• Experience with orchestration tools (e.g., Airflow, ADF, Databricks Workflows).
• Cloud experience: AWS / Azure / GCP (S3/ADLS/GCS, IAM basics, security controls).
• CI/CD exposure for data pipelines, code reviews, and automated deployments. Preferred Traits
• Strong problem-solving skills and a mindset for root-cause analysis.
• Ownership and accountability for production-grade pipelines.
• Ability to communicate with both technical and non-technical stakeholders.
• Comfort working in fast-paced environments with evolving requirements.