Data Engineer-Senior Software Engineer

What You'll Do

Who You Are

• 4–7 years of overall experience, with at least 3+ years in a Data Engineering or Cloud AI role.
• Graduate degree in Computer Science, Engineering, or equivalent.
• Mandatory: Hands-on expertise with GCP data and AI services:
• BigQuery — data modeling, optimization, partitioning, and clustering
• Vertex AI — model training, deployment, and MLOps pipelines
• Cloud Dataflow / Dataproc — batch and streaming data processing
• Cloud Composer (Apache Airflow) — pipeline orchestration
• Cloud Storage, Pub/Sub, and Bigtable — data storage and real-time streaming
• Mandatory: Experience building and operationalizing AI/ML models in GCP (Vertex AI, BigQuery ML, or Generative AI APIs).
• Proficiency in Python and SQL for data engineering and AI/ML workflows.
• Familiarity with PySpark for large-scale data transformations.
• Strong understanding of data pipeline architecture — batch, micro-batch, and streaming patterns.
• Experience with version control and CI/CD tools (GitHub, Cloud Build, Jenkins).
• Proficiency in Linux shell scripting and automation.
• Good-to-Have Skills
• Experience with LLM integration or Generative AI on GCP (Gemini APIs, Model Garden, or similar).
• Familiarity with data governance, data cataloging (Dataplex), and data lineage on GCP.
• Exposure to dbt (data build tool) for transformation and documentation.
• Knowledge of enterprise integration patterns and event-driven architectures.
• Experience with BI and reporting tools such as Looker, Power BI.
• Strong communication and stakeholder collaboration skills in cross-functional, global teams.

Responsibilities

• Design, develop, and maintain GCP-based data pipelines integrating multiple business applications and data sources.
• Build and deploy AI/ML models and data products using GCP AI services (Vertex AI, BigQuery ML, Gemini APIs).
• Develop data transformation logic using PySpark, SQL, and Python on GCP platforms (Dataflow, Dataproc, BigQuery).
• Work within a DevOps environment with CI/CD pipelines, automated deployments, and cloud-native monitoring.
• Participate in code reviews, uphold coding standards, and ensure data quality and pipeline reliability.
• Perform root cause analysis on data issues and implement robust, long-term solutions.
• Support business stakeholders with data requirements, analytics insights, and reporting needs.
• Contribute to improving automation, testing, and observability practices across the data platform.