Data Engineer

• Data Lakehouse Architecture (Supply Chain)

• ERP Integration:
• Architect and deploy ingestion pipelines to extract high-volume transactional data from SAP S/4HANA, Ariba, and PLM systems, ensuring near real-time availability for forecasting models.

• External Feed Integration

• Unified Data Model

• Unstructured Data Pipelines (LeverEDGE)

• Complex Ingestion:
• Engineer pipelines to ingest and process unstructured technical data, including PDF tender documents, CAD metadata, and historical CONOPS, transforming them into vector-ready formats for RAG (Retrieval-Augmented Generation) applications.

• Vector Database Management

• Digital Thread Implementation

• Governance & Security

• Defense-Grade Security:
• Implement Role-Based Access Control (RBAC), audit logging, and data redaction policies to ensure compliance with export controls and strict on-premise security requirements.
• Quality Control:
• Deploy automated data quality frameworks to validate BOM (Bill of Materials) completeness and cost data accuracy before it reaches AI models.
• Infrastructure Optimization:
• Optimize pipelines for on-premise GPU clusters and air-gapped environments, ensuring efficiency within existing infrastructure limits.

Technical Requirements

• Core Stack:
• Expert proficiency in Python, SQL, and modern data engineering frameworks (Apache Spark, Kafka, Airflow).
• Enterprise ERP:
• Strong experience extracting data from complex ERP environments, specifically

Sap S/4hana

SAP Ariba

• .
• Familiarity with SAP BTP is a plus.
• Database Technologies:
• Deep understanding of Data Lakehouse architectures (Databricks/Delta Lake), Relational Databases (PostgreSQL), and Vector Databases (Weaviate/Milvus).

Experience

• building pipelines for RAG solutions, Conversational agents and classical ML models with tools like dbt, dagster, or prefect
• DevOps/DataOps:
• Proficiency with containerization (Docker, Kubernetes) and CI/CD pipelines for deploying data workflows in secure environments.