Senior Analyst - Data Engineering

Role overview

Experience and qualifications

• Strong programming skills in Python and SQL.
• Experience supporting AI/ML, GenAI, RAG, semantic search, feature engineering, or model monitoring.
• Experience with unstructured data such as documents, logs, images, text, transcripts, and embeddings.
• Hands-on experience with BigQuery, Spark, Dataflow, Dataproc, Airflow/Cloud Composer, DBT, or Dataform.
• Experience with MLOps, pipeline observability, production incident management, and scalable batch and streaming pipelines.
• Knowledge of DataOps, Git, CI/CD, automated testing, version control, APIs, microservices, event-driven architectures, streaming, and real-time analytics.
• Experience migrating legacy Hadoop or on-premises platforms to cloud-native environments.
• Cloud certification in GCP, AWS, or Azure; experience mentoring junior engineers and contributing to engineering standards.

Responsibilities

• Translate business, analytics, and AI use cases into scalable data engineering solutions and build production-grade data products.
• Design, develop, and maintain batch and streaming pipelines and ETL/ELT pipelines using Python and SQL.
• Use cloud-native data technologies including BigQuery, Dataflow, Dataproc, Spark, Airflow/Cloud Composer, DBT, and Dataform.
• Implement data quality validation, monitoring, metadata management, lineage, and observability.
• Prepare feature datasets, vector-ready datasets, and governed data assets for AI/ML and GenAI teams.
• Optimize performance, storage, cloud cost, and reliability; apply DataOps practices and manage release processes.
• Support governance, security, privacy, access control, and regulatory compliance.
• Collaborate with technical and business teams; troubleshoot production issues and improve platform stability.

Employment