{bc}
Apply
Company Site

Senior Databricks Data Platform Engineer

Luxoft
Abu Dhabi, UAE
Full-time
Senior · 10+ years experience
Onsite
Discovered 1 weeks ago
• Minimum 10 Years of experience in IT/Data. • Minimum 3 years of expertise in Databricks from a platform enablement perspective. • 3+ years of experience in designing, writing, and maintaining APIs used for Platform automation, Serverless workflows, Deployment pipelines, release management and repository management • Advanced Python skills for production-grade library and service development. • Solid understanding of distributed processing concepts (PySpark knowledge beneficial, but not limited to ETL use cases). • Experience designing and integrating REST APIs. • Advanced SQL skills for troubleshooting and performance optimisation (e.g., Oracle, MS SQL). • Experience with Git and CI/CD workflows. • Familiarity with monitoring and observability tools (e.g., ServiceNow, Prometheus, Grafana). • Knowledge of scheduling and orchestration tools (e.g., Stonebranch, Control-M, Airflow). • Understanding of data quality and governance frameworks (e.g., Great Expectations). • Strong grasp of cloud infrastructure fundamentals: o DNS o Certificates o Identity & access management o Networking concepts o Load balancing

Full Job Posting

Responsibilities

  • Platform Capability Engineering • Collaborate with stakeholders during requirements clarification and sprint planning to ensure alignment with platform strategy and enterprise standards. • Design and implement technical solutions on ADIA’s Lakehouse platform (Databricks), including: o Prototyping and productising new platform capabilities. o Developing reusable Python libraries, SDKs, and internal frameworks. o Exposing platform capabilities to support ADIA’s Data Products strategy and broader Data & AI ecosystem. • Build automation for provisioning, governance controls, configuration management, and operational standards. Enterprise Integration & APIs • Integrate the Lakehouse platform with enterprise systems, including: o Incident and monitoring systems (e.g., ServiceNow). o Identity and access management solutions. o Observability platforms (e.g., Dynatrace, Prometheus, Grafana). • Design and implement secure API-based integrations using enterprise authentication and authorisation mechanisms. Reliability, Quality & Operations • Develop and maintain unit and integration tests to ensure quality and resilience of platform components. • Support QA teams during acceptance testing of platform features. • Act as a third-line engineer for complex production incidents affecting platform stability. • Collaborate with cloud and infrastructure teams to deliver secure, scalable, and reliable platform architecture.
  • Platform Capability Engineering
  • Collaborate with stakeholders during requirements clarification and sprint planning to ensure alignment with platform strategy and enterprise standards.
  • Design and implement technical solutions on ADIA’s Lakehouse platform (Databricks), including:
  • o Prototyping and productising new platform capabilities.
  • o Developing reusable Python libraries, SDKs, and internal frameworks.
  • o Exposing platform capabilities to support ADIA’s Data Products strategy and broader Data & AI ecosystem.
  • Build automation for provisioning, governance controls, configuration management, and operational standards.
  • Enterprise Integration & APIs
  • Integrate the Lakehouse platform with enterprise systems, including:
  • o Incident and monitoring systems (e.g., ServiceNow).
  • o Identity and access management solutions.
  • o Observability platforms (e.g., Dynatrace, Prometheus, Grafana).
  • Design and implement secure API-based integrations using enterprise authentication and authorisation mechanisms.
  • Reliability, Quality & Operations
  • Develop and maintain unit and integration tests to ensure quality and resilience of platform components.
  • Support QA teams during acceptance testing of platform features.
  • Act as a third-line engineer for complex production incidents affecting platform stability.
  • Collaborate with cloud and infrastructure teams to deliver secure, scalable, and reliable platform architecture.

Skills

  • Minimum 10 Years of experience in IT/Data. • Minimum 3 years of expertise in Databricks from a platform enablement perspective. • 3+ years of experience in designing, writing, and maintaining APIs used for Platform automation, Serverless workflows, Deployment pipelines, release management and repository management • Advanced Python skills for production-grade library and service development. • Solid understanding of distributed processing concepts (PySpark knowledge beneficial, but not limited to ETL use cases). • Experience designing and integrating REST APIs. • Advanced SQL skills for troubleshooting and performance optimisation (e.g., Oracle, MS SQL). • Experience with Git and CI/CD workflows. • Familiarity with monitoring and observability tools (e.g., ServiceNow, Prometheus, Grafana). • Knowledge of scheduling and orchestration tools (e.g., Stonebranch, Control-M, Airflow). • Understanding of data quality and governance frameworks (e.g., Great Expectations). • Strong grasp of cloud infrastructure fundamentals: o DNS o Certificates o Identity & access management o Networking concepts o Load balancing

Qualifications

  • Must have
  • Minimum 10 Years of experience in IT/Data.
  • Minimum 3 years of expertise in Databricks from a platform enablement perspective.
  • 3+ years of experience in designing, writing, and maintaining APIs used for Platform automation, Serverless workflows, Deployment pipelines, release management and repository management
  • Advanced Python skills for production-grade library and service development.
  • Solid understanding of distributed processing concepts (PySpark knowledge beneficial, but not limited to ETL use cases).
  • Experience designing and integrating REST APIs.
  • Advanced SQL skills for troubleshooting and performance optimisation (e.g., Oracle, MS SQL).
  • Experience with Git and CI/CD workflows.
  • Familiarity with monitoring and observability tools (e.g., ServiceNow, Prometheus, Grafana).
  • Knowledge of scheduling and orchestration tools (e.g., Stonebranch, Control-M, Airflow).
  • Understanding of data quality and governance frameworks (e.g., Great Expectations).
  • Strong grasp of cloud infrastructure fundamentals: o DNS o Certificates o Identity & access management o Networking concepts o Load balancing

Preferred Qualifications

  • Containerization & Orchestration: Docker, Kubernetes.
  • Infrastructure as Code (IaC): Terraform.
  • Understanding of Investment Data domain (desired).

Apply for this job in 1 click

Skip the repetitive application forms

Install the Base Career Chrome Extension and autofill job applications across major job boards with your profile.

Sarah M.James T.Maya R.

Trusted by over 500,000 job seekers on Base Career

Start Free Today

More from this employer

More jobs at Luxoft

Regular NetReveal AML Developer

Chennai, IND

Discovered TodayFull-time

Senior Python Developer

Bengaluru, IND

Discovered TodayFull-time

Flowable Developer

Pune, IND

Discovered TodayFull-time

Senior L2/L3 Support Engineer

, USA

Discovered TodayFull-time

Senior Avaloq Data Migration & Integration Specialist

Bengaluru, IND

Discovered TodayFull-time

Senior Avaloq Data Migration & Integration Specialist

Bengaluru, IND

Discovered TodayFull-time

Senior Python Developer

Pune, IND

Discovered TodayFull-time

Working student Field test Engineer

, USA

Discovered TodayFull-time