Data Engineer (6 months contract)

Overview

Full-Time

Role Overview

Key Responsibilities

• Design, build, and maintain scalable batch and streaming data pipelines (ETL/ELT) that move data reliably from source systems to analytics and AI workloads.
• Build and optimize data models, databases, data warehouses, and lakehouse architectures (e.g., Databricks, Microsoft SQL Server) for performance and cost.
• Develop and tune distributed processing jobs using Spark/PySpark, including cluster management and optimization.
• Ensure data quality and data governance across the full data lifecycle (collection, access, storage, transformation, retention).
• Productionize pipelines with CI/CD, orchestration (e.g., Airflow, Azure Data Factory), monitoring, and alerting.
• Build the data foundation for analytics and AI use cases, collaborating closely with Data Scientists and AI Engineers (e.g., feature engineering, RAG data processing).
• Translate client business requirements into data architecture decisions and communicate trade-offs to technical and non-technical stakeholders.
• Maintain technical documentation, data lineage, and delivery artifacts across Design, Build, UAT, Production & Hypercare.

Candidate Requirements

• Demonstrated experience delivering AI solutions in an enterprise or client-facing/consulting environment.
• Prior experience at management consulting firms and/or Big Tech is an advantage.
• Strong programming in Python and SQL with solid software engineering fundamentals.
• Hands-on experience with Spark/PySpark and big-data processing at scale.
• Proven experience building production ETL/ELT pipelines.
• Practical knowledge of a major cloud data stack — Azure, AWS, or GCP.
• Experience with a modern data platform (Databricks, Snowflake, or equivalent).
• Familiarity with orchestration, CI/CD, and DevOps (e.g., Airflow, Git, Docker).
• Understanding of data warehousing concepts, data modeling, data governance, and security best practices.
• Experience with dashboard or frontend engineering is a plus.
• Strong collaboration and communication skills, with the ability to work directly with clients.

Qualifications

• Bachelor's or Master’s degree in Computer Science, Engineering, Information Technology, or a related field.
• 3–6 years of experience in data engineering, building and operating data pipelines at scale.
• Preferred certifications: cloud data certifications such as Azure Data Engineer, AWS Data Engineer, or Databricks Data Engineer.