Senior Software Engineer - E3 -1.2 -Data Engineering

Key Responsibilities

• Data Processing & Engineering – Build and optimize scalable ETL pipelines using PySpark & Databricks for large-scale automotive datasets; implement CI/CD pipelines for robust deployment.
• Cloud & Big Data – Work with Azure Data Lake, Synapse, ADF, and Databricks for data ingestion, transformation, and storage.
• Analytics & Insights – Perform statistical analysis, predictive modeling, and trend identification on vehicle test drive data, environmental parameters, and system KPIs.
• Dashboard & Reporting – Develop interactive dashboards with Elasticsearch, Power BI, KPI visualizations, and real-time analytics for engineering teams and stakeholders.
• Collaboration & Documentation – Partner with data scientists, vehicle engineers, and cross-functional stakeholders, ensuring clear documentation of data pipelines and processes.
• Must-Have Skills
• Data Engineering & Cloud:
• PySpark, Databricks, Azure Data Lake, Azure Synapse, Azure Data Factory.
• Proficiency in ETL pipeline design, optimization, and CI/CD implementation.
• Analytics & Visualization:
• Power BI, Elastic Search with expertise in KPI creation, Kibana and real-time reporting.
• Ability to generate actionable insights from large-scale ADAS datasets.
• Automotive Data Expertise:
• Strong experience with vehicle trace data analysis (e.g., CAN, LIN, Ethernet).
• Hands-on with CANoe or similar automotive tools for ECU communication protocol analysis.
• Knowledge of ECU communication protocols (CAN, LIN, FlexRay, Ethernet)
• Collaboration & Documentation:
• Strong communication skills to work with cross-functional teams and document solutions effectively.
• Nice-to-Have Skills
• AI/ML in ADAS domain:
• Exposure to AI/ML model training, deployment, and performance monitoring.
• Familiarity with Azure Machine Learning services.
• Big Data & Streaming Analytics:
• Knowledge of Kafka, Event Hubs, or real-time data streaming for automotive use cases.
• Automation & DevOps:
• Experience with CI/CD pipelines (Azure DevOps, GitHub Actions, Jenkins) for data workflows.
• About CARIAD (Car, I Am Digital)
• CARIAD is an automotive software company that bundles together Volkswagen Group's software competencies and further expands them, building upon a heritage of bringing automotive innovation to everyone. CARIAD is building a unified technology and software platform, including a vehicle OS, vehicle cloud platform and a new unified architecture for all of Volkswagen Group’s brands. The brand profile can be found on its website - https://cariad.technology
• CARIAD India operates under a brand licensing agreement with Germany-based CARIAD SE. So, here’s a chance to innovate and transform automotive mobility!
• You'll join a team of 360 skilled experts and developers at CARIAD India and around 6,000 CARIDIANs globally, who bring with them a wealth of experience and knowledge. The areas of work involve development of platform components such as firmware, boot loaders, AUTOSAR, Linux/QNX BSP, Service oriented architecture-based middleware components.
• To explore an exciting & progressive career at CARIAD India:
• Visit us at: www.embitel.com
• Follow us at LinkedIn: https://www.linkedin.com/company/268681/
• Working with Embitel • Freedom to challenge norms, make mistakes, learn and grow • There is no ‘I’ in word ‘Team’ and no hierarchy at Embitel • Employees of Embitel are partners of our excellence and growth. And they are at the core of organization’s Mission and Vision
• To explore an exciting & progressive Career development at Embitel India: • Visit us at: www.embitel.com • Follow us at LinkedIn: https://www.linkedin.com/company/embitel-technologies-india
• Working with Embitel
• Freedom to challenge norms, make mistakes, learn and grow
• There is no ‘I’ in word ‘Team’ and no hierarchy at Embitel
• Employees of Embitel are partners of our excellence and growth. And they are at the core of organization’s Mission and Vision
• To explore an exciting & progressive Career development at Embitel India:
• Follow us at LinkedIn: https://www.linkedin.com/company/embitel-technologies-india