AI Data & Knowledge Engineer – Senior Manager

AI Data & Knowledge Engineer – Officer

About the Role

We are seeking an AI Data & Knowledge Engineer who can independently design and deliver scalable data pipelines and knowledge systems for enterprise AI use cases. This role combines core data engineering responsibilities such as building ETL/ELT pipelines, integrating large-scale datasets, and powering backend data platforms - with AI-specific capabilities like RAG, GraphRAG, and knowledge graph enablement. The ideal candidate can build reliable, governed, production-grade data foundations that support AI applications at scale.

Key Responsibilities

• Design, build, and maintain scalable batch and streaming data pipelines to ingest, transform, and serve data for AI and analytics use cases.
• Own end-to-end ETL/ELT workflows for domain data, including ingestion from enterprise systems, cleansing, transformation, enrichment, and publishing.
• Build AI-ready datasets and retrieval pipelines that support RAG, GraphRAG, semantic search, and knowledge-driven applications.
• Develop and manage knowledge graphs, ontologies, metadata layers, and data contracts for assigned domains.
• Implement and support vector indexing, chunking, hybrid retrieval, and backend data preparation workflows for AI services.
• Own integration patterns between source systems, data platforms, and AI application layers, including MCP server integrations where relevant.
• Monitor pipeline health, data quality, latency, throughput, and reliability; troubleshoot production issues and optimize performance.
• Mentor junior engineers and contribute to best practices for data engineering, data governance, and AI data enablement.

Required Qualifications

• Bachelor’s or Master’s degree in Computer Science, Data Engineering, Information Systems, or a related field.
• 5–10 years of experience in data engineering, platform engineering, or backend data systems.
• Strong hands-on experience with Python and SQL.
• Experience building production ETL/ELT pipelines and scalable data integration workflows.
• Experience with big data and distributed processing frameworks such as Spark, Databricks, Hadoop ecosystem components, or equivalent platforms.
• Familiarity with workflow orchestration tools such as Airflow or similar.
• Experience working with cloud data platforms, warehouses, or lakehouses such as Snowflake, Databricks, BigQuery, Redshift, or equivalent.
• Understanding of data contracts, metadata management, data lineage, and data governance practices.
• Hands-on exposure to RAG, GraphRAG, vector databases, or knowledge graphs in enterprise settings is preferred.
• Experience with streaming/event-driven patterns using Kafka, Kinesis, or similar tools is a plus.
• Cloud certifications in AWS and/or Azure are a plus.
• undefinedWork Schedule On-premiseKeywords Data Engineering, Big Data, ETL, ELT, Data Pipelines, Batch Processing, Streaming, Spark, Databricks, Airflow, Kafka, Snowflake, Data Lake, Lakehouse, RAG, GraphRAG, Knowledge Graphs, Ontology, Data Contracts, Vector Database, MCP Server

About State Street

Across the globe, institutional investors rely on us to help them manage risk, respond to challenges, and drive performance and profitability. We keep our clients at the heart of everything we do, and smart, engaged employees are essential to our continued success.

We are committed to fostering an environment where every employee feels valued and empowered to reach their full potential. As an essential partner in our shared success, you’ll benefit from inclusive development opportunities, flexible work-life support, paid volunteer days, and vibrant employee networks that keep you connected to what matters most. Join us in shaping the future.

As an Equal Opportunity Employer, we consider all qualified applicants for all positions without regard to race, creed, color, religion, national origin, ancestry, ethnicity, age, disability, genetic information, sex, sexual orientation, gender identity or expression, citizenship, marital status, domestic partnership or civil union status, familial status, military and veteran status, and other characteristics protected by applicable law.

Discover more information on jobs at StateStreet.com/careers

Read our CEO Statement