Data Platform Engineering Lead - AITDS
Role Overview
This is a hands-on technical lead role building AI-ready enterprise data platforms.
The platform includes scalable pipelines, lakehouse architectures, and governed data foundations for model training, evaluation, and inference.
The role also develops backend services and integration layers exposing curated datasets, feature stores, and data products for AI/ML and LLM applications.
Key Skills for This Role
Full Job Posting
Role Overview
This is a hands-on technical lead role building AI-ready enterprise data platforms.
The platform includes scalable pipelines, lakehouse architectures, and governed data foundations for model training, evaluation, and inference.
The role also develops backend services and integration layers exposing curated datasets, feature stores, and data products for AI/ML and LLM applications.
Key Responsibilities
- Design batch and streaming data pipelines using Python and Spark/PySpark.
- Build AI-ready datasets, feature pipelines, and training-data foundations.
- Develop Python backend services and APIs, such as FastAPI, to expose data products.
- Implement microservices and event-driven data ingestion, processing, and serving architectures.
- Mentor engineers on data engineering patterns and best practices.
- Develop pipelines for model training, evaluation, and inference.
- Support LLM and AI integrations including RAG, embeddings, inference APIs, vectorization, and retrieval.
- Collaborate with infrastructure teams across Azure, AWS, or GCP.
- Integrate data lakes, lakehouses, SQL/NoSQL systems, containerized, and serverless architectures.
- Implement CI/CD, observability, data quality, metadata, lineage, governance, and platform optimization.
Required Capabilities and Experience
- 8+ years in data engineering and backend development.
- Strong Python, APIs, and microservices experience.
- Deep Spark/PySpark or equivalent experience.
- Expertise in ETL/ELT pipelines and lakehouse architectures.
- Experience with feature stores and AI-ready datasets.
- Experience with Azure, AWS, or GCP.
- Familiarity with Docker, Kubernetes, and CI/CD.
- Familiarity with LLM/GenAI integration patterns.
- Familiarity with metadata and governance tooling.
- Strong system design and problem-solving skills.
Apply for this job in 1 click
Skip the repetitive application forms
Install the Base Career Chrome Extension and autofill job applications across major job boards with your profile.
Trusted by over 500,000 job seekers on Base Career
More from this employer
More jobs at Cognizant
Azure Business Data Analyst (Hybrid)
Toronto, CAN
Cognizant is seeking an experienced Business Data Analyst to bridge banking stakeholders and data technology teams on cloud data engineering, migration, analytics, and modernization initiatives. Candidates need 5+ years
Salesforce Business Analyst
Toronto, CAN
Cognizant is seeking an experienced Salesforce Business Analyst to help deliver Salesforce projects with business and technical teams. The role covers requirements, solution delivery support, testing, integrations, repor
Workday Reporting & Prism Principal Consultant
London, GBR
Cognizant is seeking a senior functional consultant to lead Workday Reporting and Prism Analytics solution design and delivery. Applicants need five or more years of Workday implementation experience, deep reporting and
Consulting Manager, P&C Insurance- Claims transformation
Mississauga, CAN
Cognizant is seeking a consulting manager to lead claims-transformation work for property and casualty insurers, from initiative roadmaps and business cases through solution design and delivery readiness. Applicants need
Cyber Security Solutions Sales Specialist (Canada)
Halifax, CAN
Cognizant is seeking a Cyber Security Solutions Sales Specialist to grow cybersecurity revenue, develop client opportunities, and lead complex pursuits. The role requires 15+ years of overall experience, including 10+ ye
Senior Business Analyst
, CAN
Cognizant seeks a Senior Business Analyst to connect business needs with technology solutions through requirements analysis, process improvement, stakeholder collaboration, and Agile delivery support. Candidates need a r
Salesforce Business Analyst
Toronto, CAN
Cognizant seeks an experienced Salesforce Business Systems Analyst to gather requirements, shape solutions, and support Salesforce project delivery with business and technical teams. The role includes user stories, proce
Salesforce Business Analyst
Toronto, CAN
Cognizant seeks a Salesforce Business Analyst to gather requirements and support Salesforce solution delivery, testing, and production validation. The role requires 5+ years as a BA/BSA and 4+ years of Salesforce experie
Azure Business Data Analyst (Hybrid)
Toronto, CAN
Salesforce Business Analyst
Toronto, CAN
Workday Reporting & Prism Principal Consultant
London, GBR
Consulting Manager, P&C Insurance- Claims transformation
Mississauga, CAN
Cyber Security Solutions Sales Specialist (Canada)
Halifax, CAN
Senior Business Analyst
, CAN
Salesforce Business Analyst
Toronto, CAN
Salesforce Business Analyst
Toronto, CAN