Databricks Platform Engineer
Job Fit Check
Base Career helps you apply smarter for this job.
Key skills for this role
About the Role
Location: On-site in Abu Dhabi Seeking a highly skilled and motivated Data Platform Engineer to join our client's core platform team. This role focuses on building advanced platform capabilities, optimizing and maintaining robust data infrastructure, and enabling data-driven decision-making across the organization.
Key Skills for This Role
Full Job Posting
On-Site In Abu Dhabi
Seeking a highly skilled and motivated
Data Platform Engineer
to join our client's core platform team.
This role focuses on building advanced platform capabilities, optimizing and maintaining robust data infrastructure, and enabling data-driven decision-making across the organization.
The ideal candidate has deep expertise in
Databricks, PySpark, and Python
, and demonstrates strong communication skills - articulate, precise, and concise.
Key Responsibilities
- Collaborate with stakeholders during requirements clarification and sprint planning to ensure alignment with business objectives
- Design and implement technical solutions on the
- Lakehouse platform (Databricks)
- , including:
- *Prototyping new Databricks capabilities*
- *Exposing these capabilities to support Data Products strategy and Data & AI ecosystem*
- Integrate data platforms with enterprise tools, including:
- *Incident and monitoring systems (e.g., ServiceNow)*
- *Identity management solutions*
- *Data observability tools (e.g., Dynatrace)*
- Develop and maintain unit and integration tests to ensure quality and resilience
- Support QA teams during acceptance testing
- Act as a third-line engineer for production incidents, ensuring system stability and uptime
- Collaborate with cloud and infrastructure teams to deliver secure, scalable, and reliable solutions
Experience
- Minimum 8 years in IT/Data-related roles
Technical Expertise
- Expert knowledge of
Databricks
- Proficient in
Pyspark
- for distributed computing
- Strong
Skills For Library Development
- Advanced
Sql
- skills for complex query optimization (e.g., Oracle, MS SQL)
- Experience with Git for version control
- Familiarity with monitoring tools (e.g., ServiceNow, Prometheus, Grafana)
- Knowledge of scheduling tools (e.g., Stonebranch, Control-M, Airflow)
- Proficiency in data quality frameworks (e.g., Great Expectations, ideally Monte Carlo)
- Solid understanding of cloud infrastructure fundamentals (DNS, certificates, identity, load balancing)
- **Note:**
- Primary focus is building platform capabilities rather than writing ETL pipelines
Agile & Collaboration
- Comfortable working in Agile environments (sprint planning, stand-ups, retrospectives)
- Skilled in Azure DevOps for project management
Problem-Solving & Communication
- Strong debugging and troubleshooting skills for complex data engineering issues
- Exceptional written and verbal communication skills
- Ability to explain technical concepts to non-technical stakeholders
Preferred Experience
- Understanding of the investment data domain
Apply for this job in 1 click
Skip the repetitive application forms
Install the Base Career Chrome Extension and autofill job applications across major job boards with your profile.
Trusted by over 500,000 job seekers on Base Career
More from this employer
More jobs at Toptal
Data Specialist
Abu Dhabi, UAE
Provide L2 production support for enterprise Lakehouse data platforms, resolving incidents, performing diagnostics, and coordinating with engineering teams. Requires strong DataOps experience and hands-on Lakehouse platf
Data Specialist
Abu Dhabi Emirate, UAE
Toptal is an elite network of the world's top talent, connecting the best and brightest in business, design, and technology with leading organizations worldwide. We deliver freelance opportunities on an independent contr
Python Solutions Engineer DUBAI
Dubai, UAE
Job Description Summary This applied research facility transforms theoretical AI into practical tools for aviation, healthcare, and logistics, with a core focus on advanced robotics. By prototyping everything from autono
Quality Assurance Engineer
Abu Dhabi, UAE
Join Toptal’s Elite Network of Freelance Talent Toptal is an exclusive network of the world’s top freelance software developers, designers, finance experts, product managers, and project managers. As one of the fastest-g
Data Engineer
Abu Dhabi, UAE
Location: On-site in Abu Dhabi We’re looking for an experienced, hands-on Data Engineer capable of designing, building, and operating data pipelines and models that power analytics and applications across investment team