{bc}
Apply
oracle

Data Engineer II

American Express
Gurugram, IND
Full-time
Mid · 5+ years experience
Hybrid
Discovered 1 weeks ago
airflowgcpgithadoophivekafka

Full Job Posting

Minimum Qualifications

  • Bachelor's or master's degree in computer science, Engineering or a related technical discipline, or equivalent work experience.
  • 5+ years of hands-on experience in software development or data engineering, preferably with large-scale Big Data and cloud-based solutions.
  • Strong hands-on expertise in SQL, Python, GCP Big Query and Apache Airflow, including developing and optimizing complex data pipelines.
  • Experience with Spark, Scala, Hadoop/Hive or equivalent distributed data-processing technologies.
  • Strong data analysis and problem-solving skills, including data profiling, complex SQL, source-to-target validation, data reconciliation and root cause analysis.
  • Experience in ETL/ELT, data modeling, data transformation, workflow orchestration and performance optimization.
  • Practical hands-on experience with Generative AI tools such as GitHub Copilot, ChatGPT Enterprise or equivalent AI coding assistants for development, debugging, testing or automation.
  • Working knowledge of LLMs, prompt engineering and responsible AI practices, particularly secure handling of enterprise and sensitive data.
  • Experience with Git/GitHub, CI/CD pipelines, automated testing and software development best practices.
  • Understanding of data-quality controls, data governance, privacy and production support.
  • Strong communication skills, ability to collaborate across teams and independently resolve complex technical problems.

Preferred Qualifications

  • Experience with GCP Bigtable, Pub/Sub, Dataflow, Kafka or real-time data-processing frameworks.
  • Exposure to Adobe Experience Platform, Adobe Journey Optimizer or Customer Journey Analytics.
  • Experience building or contributing to AI-enabled data engineering solutions, such as automated RCA, data-quality monitoring, intelligent data discovery, lineage or test-data generation.
  • Familiarity with LLM APIs, Retrieval-Augmented Generation (RAG), agentic AI frameworks or AI workflow automation.
  • Experience developing enterprise data products supporting Marketing, Banking, Lending or customer engagement.
  • GCP Professional Data Engineer or equivalent cloud certification.

Responsibilities

  • We are looking for a skilled and motivated Engineer II with strong hands-on experience in Data Engineering, Big Data technologies, GCP, Python, SQL and AI-assisted software development.
  • Data Engineering & Development: Design, develop and maintain scalable data pipelines using GCP BigQuery, Python, SQL, Spark and Apache Airflow for large-scale data ingestion, transformation and processing.
  • Data Analysis & Quality: Perform data discovery, profiling, complex SQL analysis, source-to-target reconciliation, data validation and root cause analysis to ensure trusted, high-quality data.
  • AI-Assisted Engineering: Leverage approved Generative AI tools, GitHub Copilot and LLM-based solutions to accelerate code development, testing, troubleshooting, documentation and engineering automation.
  • Performance & Reliability: Optimize data-processing performance, automate monitoring, troubleshoot production issues and implement preventive measures to improve platform stability.
  • Data Product Enablement: Build reusable customer, product, campaign and engagement data capabilities supporting enterprise Marketing, targeting, personalization and analytics.
  • Engineering Excellence: Follow software engineering best practices, including CI/CD, GitHub, code reviews, automated testing, data governance and production-readiness controls.
  • Cross-Functional Collaboration: Partner with Product Owners, Business stakeholders, architects and upstream/downstream engineering teams to understand requirements, resolve dependencies and deliver business capabilities.
  • Continuous Improvement: Proactively identify opportunities for automation, technical debt reduction, AI adoption and engineering productivity improvements.

Apply for this job in 1 click

Skip the repetitive application forms

Install the Base Career Chrome Extension and autofill job applications across major job boards with your profile.

Sarah M.James T.Maya R.

Trusted by over 500,000 job seekers on Base Career

Start Free Today

More from this employer

More jobs at American Express

Associate-Tech Ops Engineering

Bengaluru, IND

Discovered YesterdayFull-time

Lead Analyst

Gurugram, IND

Discovered YesterdayFull-time

Senior Associate-Tech Ops Engineering

Bengaluru, IND

Discovered YesterdayFull-time

Vice President, TCE US Travel Network

Phoenix, USA

Discovered YesterdayFull-time

Credit Specialist – Financial Support Team (Start Date: Monday 1st February 2027)

Brighton, GBR

Discovered YesterdayFull-time

Senior Manager, Frameworks and Procedures, U.S. Issuing Financial Crimes Center of Excellence

New York City, USA

Discovered YesterdayFull-time

Manager, Organic Social Strategy

New York City, USA

Discovered YesterdayFull-time

Senior Manager - Digital Product Operations

New York City, USA

Discovered YesterdayFull-time