Data Test Engineer – Databricks Delta Lake

About This Role

Data Test Engineer – Databricks Delta Lake

Experience: 3–6 Years

Location : Abu Dhabi , UAE (onsite)

Job Summary

We are looking for a skilled Test Engineer with strong experience in data validation, ETL testing, and Delta Lake on Databricks. The candidate will be responsible for ensuring data quality, integrity, and performance across data pipelines and analytics layers.

Key Responsibilities

Perform data validation and testing on Delta Lake tables

Validate ETL pipelines built using Databricks (batch and incremental loads)

Write and execute complex SQL queries to identify data issues, duplicates, and inconsistencies

Validate incremental load logic, partitioning, and data refresh processes

Perform source-to-target data reconciliation

Identify and troubleshoot data quality issues (duplicates, nulls, mismatches)

Collaborate with data engineers and BI teams to resolve defects

Validate data models used in reporting tools such as Power BI

Conduct performance testing on large datasets

Create and maintain test cases, test plans, and test reports

Required Skills

Strong SQL skills (mandatory)

Hands-on experience with Databricks and Delta Lake

Experience in ETL/Data Warehouse testing

Solid understanding of:

Data pipelines

Incremental load concepts

Partitioning and indexing

Experience in data validation techniques

Familiarity with Azure ecosystem (preferred)

Good to Have

Experience with Apache Spark

Knowledge of Power BI testing

Exposure to automation tools (Python / PySpark)

Key Testing Scenarios

Duplicate data handling in Delta tables

Incremental load validation (StartRange / EndRange issues)

Late-arriving data scenarios

Partition refresh validation

Schema evolution testing

Data consistency across layers (Bronze → Silver → Gold)

Job Types: Full-time, Contract

Pay: AED7,000.00 - AED10,000.00 per month

Application Question(s):

• Current Monthly Salary ?
• Expected Monthly Salary ?
• In how many days you can join us if shortlisted?
• How many years of experience do you have in Data Quality and Data testing role ?
• How many years of experience do you have in SQL coding or Testing ?
• How many years of experience do you have in testing Delta Lakes?
• Please rate your English communication skills out of 10 (1 is lowest and 10 is highest)?
• Have you gone through the JD and ok to perform all the responsibilties mentioned?
• Please write down your technical skills which makes you a fit for this role from the JD ?
• What is your Visa Status ?

Work Location: In person