Data Platform Engineer - Security (m/f/d)

Role Purpose

• Real-Time Security Data Pipelines

• Design and implement real-time data ingestion pipelines for security events
• Integrate pipelines with:
• + SIEM platforms (e.g., Splunk, Sentinel, QRadar)
• + SOAR tools and security monitoring systems
• Ensure high availability, scalability, and low-latency data processing
• Handle large-scale event streaming and log ingestion from diverse sources

• Threat Intelligence Correlation Engine

• Develop and maintain threat intelligence correlation engines to support SOC operations
• Integrate external and internal threat feeds to:
• + Enrich security events
• + Improve detection accuracy
• Enable real-time correlation and contextualization of threats
• Work closely with SOC teams to refine detection logic and use cases

• Cyber Data Lake Architecture

• Design and manage a centralized cyber data lake for:
• + Security logs
• + Incident data
• + Forensic and audit records
• Ensure the platform supports:
• + Scalable storage and retrieval
• + Data retention and lifecycle policies
• + Compliance with regulatory and audit requirements
• Optimize data structures for:
• + Investigations
• + Reporting and analytics
• + Machine learning use cases

• Data Engineering & Platform Optimization

• Build robust ETL/ELT processes for structured and unstructured security data
• Ensure data quality, consistency, and governance across pipelines
• Optimize performance and cost efficiency of cloud-based data platforms
• Implement monitoring, logging, and alerting for data pipeline health

• Security Analytics Enablement

• Enable advanced analytics capabilities for:
• + Threat detection
• + Incident response
• + Behavioral analytics
• Support integration with:
• + BI tools (e.g., Power BI)
• + Machine learning models
• Provide datasets and structures optimized for SOC reporting and dashboards

• Compliance, Audit & Governance

• Ensure data platforms meet:
• + Internal security policies
• + Regulatory and audit requirements (banking environment)
• Maintain proper data lineage, traceability, and audit trails
• Support audit requests and forensic investigations with reliable data access

Education

• Bachelor’s or Master’s degree in:
• + Data Engineering
• + Computer Science
• + Information Systems or related field

Experience

• 8–10 years of experience in:
• + Data engineering / big data platforms
• + Cloud-based data architecture (Azure / AWS)
• Hands-on experience working with:
• + High-volume, real-time data pipelines
• + Security or operational data systems
• Experience in cybersecurity or SOC environments (highly preferred)
• Exposure to regulated industries (banking/financial services) is an advantage

Technical Skills

• Strong expertise in:
• + Azure Data Services (Data Factory, Synapse, Event Hub, etc.)
• + Databricks (mandatory experience preferred)
• Experience with:
• + Streaming technologies (Kafka, Spark Streaming, or equivalent)
• + SIEM/SOAR integrations
• Proficiency in:
• + SQL, Python, or Scala
• Knowledge of:
• + Data lake architectures (Delta Lake, Lakehouse models)
• + Security data schemas and log formats
• Familiarity with:
• + Cloud platforms (Azure, AWS)
• + Data governance and security best practices

Soft Skills

• Strong analytical and problem-solving capabilities
• Ability to manage large-scale data environments
• Effective collaboration with cybersecurity and SOC teams
• Strong documentation and communication skills

Ideal Candidate Profile

• Experienced data engineer with strong cloud and streaming expertise
• Proven ability to handle high-volume, real-time security data
• Familiar with SOC operations and cybersecurity data use cases
• Capable of designing platforms that support analytics, investigations, and compliance
• Strong balance of engineering depth and operational reliability
• Data Platform Engineer - Security in Abu Dhabi, United Arab Emirates