Data Architect - Data Engineering 4D

About the Role

What You'll Do

Core platform

Metadata-driven engineering (critical)

Platform engineering / DataOps

Governance & collaboration

Nice to Have

• Experience with Delta Live Tables or Databricks Streaming Tables (declarative SQL ingestion)
• SnowFlake, Iceberg
• Exposure to multi-tenant data platforms serving several internal business-unit consumers with differing retention/access needs
• What Success Looks Like in the First 6 Months
• New source onboarding (landing → central) reduced to a config-driven task, not custom pipeline development
• Partition-based retention live and operating automatically across central repository schemas
• Routing-rules table operational, with at least the initial set of downstream consumption views generated from it rather than hand-written
• Clear, current documentation of the ingestion → central → archive → consumption architecture

What Success Looks Like in the First 6 Months

Qualifications

• Bachelors - Business Analytics, Bachelors - Computer Science, Bachelors - Statistics, Masters - Data Science
• Certifications
• Certified Data Management Professional - RiversandRiversand, Databricks Certified Associate Developer for Apache Spark 3.0 - Databricks AcademyDatabricks Academy, Databricks Certified Associate Developer for Apache Spark - Databricks AcademyDatabricks Academy, Microsoft Certified: Azure Data Engineer Associate - MicrosoftMicrosoft, Oracle Database 12c Certified Implementation Specialist - OracleOracle
• Agile Methodology, Change Management, Consumer Packaged Goods (CPG), Databricks Platform, Data Engineering, Snowflake GenAI
• Language
• English
• Language Proficiency -
• Proficient - C2
• Additional Job Location -
• Job Type
• Regular
• Master Skill List -
• Data Engineering
• Remote Type -
• Office
• Work Shift -
• Flex Time (India)
• Why join Genpact?
• Lead AI-powered transformation – Drive innovation and solve real-world business challenges that matter
• Make an impact – Help global enterprises solve business challenges that matter
• Accelerate your career – Gain hands-on experience, mentorship, and world-class learning opportunities to stay ahead
• Work with the best – Join 140,000+ bold thinkers and problem-solvers who push boundaries every day
• Thrive in a values-driven culture – Our courage, curiosity, and incisiveness - built on a foundation of integrity and inclusion - allow your ideas to fuel progress
• Come join the 140,000+ coders, tech shapers, and growth makers at Genpact and take your career in the only direction that matters: Up.
• Let’s build tomorrow together.
• Genpact is an Equal Opportunity Employer and considers applicants for all positions without regard to race, color, religion or belief, sex, age, national origin, citizenship status, marital status, military/veteran status, genetic information, sexual orientation, gender identity, physical or mental disability or any other characteristic protected by applicable laws. Genpact is committed to creating a dynamic work environment that values respect and integrity, customer focus, and innovation.
• Furthermore, please do note that Genpact does not charge fees to process job applications and applicants are not required to pay to participate in our hiring process in any other way. Examples of such scams include purchasing a 'starter kit,' paying to apply, or purchasing equipment or training.

Required Skills

• Agile Methodology, Change Management, Consumer Packaged Goods (CPG), Databricks Platform, Data Engineering, Snowflake GenAI
• Language
• English
• Language Proficiency -
• Proficient - C2
• Additional Job Location -
• Job Type
• Regular
• Master Skill List -
• Data Engineering
• Remote Type -
• Office
• Work Shift -
• Flex Time (India)
• Why join Genpact? • Lead AI-powered transformation – Drive innovation and solve real-world business challenges that matter • Make an impact – Help global enterprises solve business challenges that matter • Accelerate your career – Gain hands-on experience, mentorship, and world-class learning opportunities to stay ahead • Work with the best – Join 140,000+ bold thinkers and problem-solvers who push boundaries every day • Thrive in a values-driven culture – Our courage, curiosity, and incisiveness - built on a foundation of integrity and inclusion - allow your ideas to fuel progress Come join the 140,000+ coders, tech shapers, and growth makers at Genpact and take your career in the only direction that matters: Up. Let’s build tomorrow together. Genpact is an Equal Opportunity Employer and considers applicants for all positions without regard to race, color, religion or belief, sex, age, national origin, citizenship status, marital status, military/veteran status, genetic information, sexual orientation, gender identity, physical or mental disability or any other characteristic protected by applicable laws. Genpact is committed to creating a dynamic work environment that values respect and integrity, customer focus, and innovation. Furthermore, please do note that Genpact does not charge fees to process job applications and applicants are not required to pay to participate in our hiring process in any other way. Examples of such scams include purchasing a 'starter kit,' paying to apply, or purchasing equipment or training.

Responsibilities

• Build and operate Auto Loader–based ingestion pipelines (batch and incremental) from Unity Catalog volumes into managed Delta tables, using MERGE patterns for upsert/dedup logic
• Design and maintain metadata-driven pipeline generation — where dataset onboarding (new source, new schema, new retention rule) is a config change against a rules table, not new pipeline code per dataset
• Implement and maintain partition-based retention (year partitioning; rolling window of current + N prior years) and the archive extraction process for datasets requiring longer history
• Build and maintain a dataset routing-rules engine: a governed metadata table (splittable, target_systems[], row_filter_expr, column_mask_expr) that programmatically generates consumption-layer views per downstream system
• Own Unity Catalog governance: catalog/schema/storage-location design, access grants, lineage, and audit trail across landing, central, archive, and consumption layers
• Manage Databricks Asset Bundles (DAB) or equivalent IaC for repeatable environment and job deployment across dev/test/prod
• Monitor pipeline health, checkpoint integrity, schema drift/evolution, and job SLAs; build alerting for ingestion failures and data-freshness breaches
• Collaborate with upstream source-system owners to define data contracts (e.g. ensuring source_changed_at semantics are honored) and with downstream teams to translate their access/filter requirements into routing rules
• Document pipeline architecture, retention policy, and routing logic for audit and onboarding purposes
• Required Skills & Experience
• Azure Databricks and Unity Catalog (catalogs, schemas, managed/external volumes, managed/external tables, storage-location hierarchy, grants)
• Strong PySpark and SQL, including Structured Streaming / Auto Loader (cloudFiles, foreachBatch, trigger(availableNow=True)), MERGE INTO, and Delta Lake internals (transaction log, OPTIMIZE, VACUUM, ZORDER)
• Practical experience with partitioning strategy and retention management (partition-drop vs. rolling delete tradeoffs) at production data volumes
• Demonstrable background designing or maintaining metadata/config-driven code generation — e.g. a rules or registry table that drives dynamically generated SQL, views, jobs, or DDL, rather than hardcoded per-object pipelines
• Comfort with templated/parameterized pipeline patterns (e.g. DAB with variable substitution, dbt-style codegen, or custom Python/SQL generation from a config table)
• Understanding of how to keep generated artifacts (views, jobs) in sync with a source-of-truth ruleset, including versioning and change auditability
• Azure fundamentals: ADLS Gen2, Azure Data Factory or equivalent orchestration, networking/private endpoints, cost management basics
• CI/CD for data pipelines (Azure DevOps or GitHub Actions), ideally with Databricks Asset Bundles or Terraform for Databricks
• Git-based workflow discipline; comfortable owning pipeline code as a shared, reviewed codebase rather than notebook-only development
• Monitoring/observability for data pipelines (job alerting, data quality checks, schema drift detection)
• Working knowledge of data governance concepts: access control models, data lineage, retention/compliance-driven deletion, PII masking
• Experience working with multiple downstream consumers with differing data needs — building filtered/masked views rather than granting broad table access
• Clear written documentation habits; able to explain pipeline and governance logic to both technical and non-technical stakeholders