Databricks Data Platform Engineer/Architect
Skills
About This Role
Databricks Data Platform Engineer/Architect
Project description
We are seeking a highly skilled Databricks Platform Engineer with strong experience in data engineering.
The candidate will have a deep understanding of both data platforms and software engineering, enabling them to effectively integrate and operate the platform within a broader IT ecosystem.
This role requires a hands-on individual contributor who takes full ownership of deliverables end-to-end, including design, development, testing, deployment, and ongoing support.
Responsibilities
- Manage and optimize Databricks data platform including workspace setup, cluster policies, job orchestration, Unity Catalog, cost controls, multi-tenancy.
- Design, write and maintain APIs used for Platform automation, Serverless workflows, Deployment pipelines, release management and repository management
- Ensure high availability, security, and performance of data systems which includes access control, secrets management, RBAC, monitoring, alerting, RLS, incident handling, performance tuning.
- Provide valuable insights about the data platform (Databricks) usage which includes cost attribution, usage analytics, workload patterns, telemetry.
- Implementing new features of Databricks, including serverless, Declarative Pipelines, Agents, lakebase , etc.
- Design and maintain system libraries (Python) used in ETL pipelines and platform governance (Databricks).
• Optimize ETL Processes
- Enhance and tune existing ETL processes for better performance, scalability, and reliability.
Skills
- Must have
- Minimum 10 Years of experience in IT/Data.
- Minimum 5 years of experience as a Databricks Data Platform Engineer.
- 3+ years of experience in designing, writing, and maintaining APIs used for Platform automation, Serverless workflows, Deployment pipelines, release management and repository management
- Bachelor's in IT or related field.
- Infrastructure & Cloud: Azure, AWS (expertise in storage, networking, compute).
- Programming: Proficiency in PySpark for distributed computing.
- minimum 4 years of Python experience for ETL development.
- SQL: Expertise in writing and optimizing SQL queries, preferably with experience in databases such as PostgreSQL, MySQL, Oracle, or Snowflake.
- Data Warehousing: Experience working with data warehousing concepts and Databricks platform.
- ETL Tools: Familiarity with ETL tools & processes
- Data Modelling: Experience with dimensional modelling, normalization/denormalization, and schema design.
- Version Control: Proficiency with version control tools like Git to manage codebases and collaborate on development.
- Data Pipeline Monitoring: Familiarity with monitoring tools (e.g., Prometheus, Grafana, or custom monitoring scripts) to track pipeline performance.
- Data Quality Tools: Experience implementing data validation, cleaning, and quality frameworks, ideally Monte Carlo.
- Nice to have
- Containerization & Orchestration: Docker, Kubernetes.
- Infrastructure as Code (IaC): Terraform.
- Understanding of Investment Data domain (desired).
Your resume, rewritten
for this exact role.
Sign up free — Base Career tailors your CV to this job description in 60 seconds.
01 / 05
Resume Tailored to This Job
Your keywords, structure, and story — rewritten to match this exact role and pass ATS filters.
Free · No card · 60 seconds
02 / 05
Cover Letter for This Role, Done
Job-specific cover letters written in Gulf professional tone — ready in seconds, not hours.
Free · No card · 60 seconds
03 / 05
See How Well You Fit This Role
AI match score with clear reasons — know your fit before investing time in the application.
Free · No card · 60 seconds
04 / 05
Apply in One Click
Autofill any application form on Workday, LinkedIn, Bayt, Greenhouse — with your tailored content.
Free · No card · 60 seconds
05 / 05
Track It. Follow Up at the Right Time.
Visual pipeline for every application with AI-timed follow-up reminders so nothing slips.
Free · No card · 60 seconds
Similar Jobs
Databricks Data Platform Engineer/Architect
Luxoft · Abu Dhabi
Project description We are seeking a highly skilled Databricks Platform Engineer with strong experience in data engineering. The candidate will have a deep understanding of both data platforms and software engineering, e
Skills
1 months ago
Apply Now↗Apply Now ↗2.2K+
Cover Letters & Follow-ups
1.8K+
Resumes Tailored
190.5K+
Jobs Tracked
Trusted by professionals at
Stop applying blindly.
Start getting hired.
Base Career automates the hardest parts of job searching — apply smarter, not harder.
AI Resume in 60s
Your resume rewritten for this exact role using the job description as the brief.
ATS-Optimized
Get past automated screening filters with the right keywords matched to each job.
Application Tracker
Track every job, follow-up, and interview in one visual kanban board.
Free plan · No credit card required