Data Platform Engineer

Overview

• _Job Summary_
• The Data Platform Engineer is responsible for designing, building, operating, and optimizing the enterprise data platform that enables analytics, AI, reporting, and automation across the organization.
• The role focuses on scalable data pipelines, lakehouse and warehouse environments, integration frameworks, platform reliability, security, and performance.
• This position ensures enterprise data platforms are governed, resilient, and capable of supporting real-time and batch workloads for business users, analytics teams, AI teams, and automation functions.
• _Job Responsibilities 1_ 1.
• Data Platform Architecture & Infrastructure
• Design, build, and maintain enterprise data platforms including data lakes, lakehouses, and data warehouses.
• Manage cloud-based data environments such as Databricks, Snowflake, BigQuery, Synapse, or equivalent platforms.
• Ensure scalability, reliability, resilience, and security of platform infrastructure across environments.

2. Data Pipelines & Integration Engineering

• Build and maintain batch and streaming ingestion frameworks for structured and unstructured data sources.
• Develop reusable ETL / ELT frameworks, templates, and standards for enterprise data movement.
• Integrate data from ERP, CRM, APIs, flat files, operational systems, and streaming sources.

3. Workflow Automation & DevOps

• Implement orchestration tools such as Airflow, ADF, Dagster, or similar workflow schedulers.
• Establish CI/CD pipelines for data engineering deployments and automated testing processes.
• Promote infrastructure-as-code, containerization, and automation best practices.

4. Data Governance & Security Enablement

• Implement access controls, encryption, and platform security standards for enterprise data assets.
• Enable metadata management, lineage capture, data cataloging, and discovery capabilities.
• Support governance frameworks, compliance requirements, and data quality monitoring controls.

5. Performance Optimization & Reliability

• Monitor platform performance, workload usage, storage efficiency, and query responsiveness.
• Optimize compute utilization, pipeline execution times, and operational costs.
• Troubleshoot platform incidents and ensure high availability with minimal disruption.
• _Job Responsibilities 2_ 6.
• Backup, Recovery & Environment Management
• Manage Dev, Test, and Production environments with controlled release practices.
• Implement backup, disaster recovery, retention, and recovery procedures for critical data assets.
• Ensure continuity planning for data platform operations and service resilience.

7. Support for Analytics, AI & Automation

• Deliver curated datasets and optimized data structures for reporting, dashboards, AI models, and automation use cases.
• Enable feature-ready datasets for machine learning and predictive analytics workloads.
• Support both real-time and scheduled consumption models for enterprise users.

8. Continuous Improvement & Stakeholder Collaboration

• Work closely with architects, analysts, AI teams, automation teams, and business stakeholders to improve platform capabilities.
• Recommend new technologies and engineering practices that enhance speed, scalability, and value delivery.
• Contribute to platform roadmaps, standards, and long-term modernization initiatives.

Additional Responsibilities 3 _Job Knowledge & Skills_