Senior Software Engineer II
Role Overview
Serve as technical owner and steward of Nielsen’s corporate Databricks Lakehouse, driving strategy, architecture, and operational maturity.
Build a secure, scalable, cost-efficient, governed, observable, and audit-ready data platform.
Develop AI-native platform capabilities and use AI-assisted and agentic engineering practices to build and operate the platform.
Key Skills for This Role
Full Job Posting
Strategic Mandate
Serve as technical owner and steward of Nielsen’s corporate Databricks Lakehouse, driving strategy, architecture, and operational maturity.
Build a secure, scalable, cost-efficient, governed, observable, and audit-ready data platform.
Develop AI-native platform capabilities and use AI-assisted and agentic engineering practices to build and operate the platform.
Core Goals and Responsibilities
- Own Lakehouse architecture, workspace topology, cluster policies, compute strategy, and multi-cloud deployment patterns across AWS, Azure, or GCP.
- Design and enforce global governance for catalogs, schemas, access controls, lineage, and audit logging through Unity Catalog.
- Set standards for Delta Lake, Delta Live Tables, medallion architecture, and performance optimization.
- Partner with CloudOps and DevOps on infrastructure as code, CI/CD, and monitoring, with a target of 99.99% platform availability.
- Own cost transparency and optimization, including cluster sizing, workload isolation, chargeback/showback, and budget alerts.
- Translate stakeholder requirements into platform capabilities and self-service patterns.
- Govern rollout of AI use cases and native AI features, including Databricks Genie, AI/BI Dashboards, and Mosaic AI, within access controls.
- Lead a developer community of practice; publish standards, deliver enablement sessions, and remove delivery friction.
Experience and Expertise
- 8–10+ years in data engineering, data platform architecture, or distributed systems, including 3+ years architecting and operating Databricks environments at scale.
- Hands-on experience building, managing, running, and scaling production-grade Databricks data platforms end to end.
- Experience with Unity Catalog, Delta Lake, Delta Live Tables, Databricks Workflows, Databricks SQL, and cluster/compute policy design.
- Experience architecting feature stores, vector search/indexes, and GenAI-ready pipelines, and using AI-assisted or agentic engineering practices.
- Deep hands-on Apache Spark expertise for batch and structured streaming, large-scale data processing, and performance tuning.
Technology and Platform Scope
- Databricks, Unity Catalog, Delta Lake, Delta Live Tables, Databricks Workflows, Databricks SQL, and cluster/compute policies.
- Terraform or Databricks Asset Bundles, CI/CD, monitoring, and multi-cloud deployment patterns across AWS, Azure, or GCP.
- AI platform capabilities include feature stores, vector search, GenAI-ready pipelines, Databricks Genie, AI/BI Dashboards, and Mosaic AI.
About Nielsen
Nielsen provides audience engagement insights to support client decisions and operates in the media measurement and analytics sector.
Apply for this job in 1 click
Skip the repetitive application forms
Install the Base Career Chrome Extension and autofill job applications across major job boards with your profile.
Trusted by over 500,000 job seekers on Base Career
More from this employer
More jobs at The Nielsen Company
Senior Product Manager II (multi-country quantitative research)
Bengaluru, IND
•Define product requirements, work with engineering teams, and deliver solutions that meet user needs. Scope includes driving transformational growth across our Sports Research and Consulting products and solutions, as w
Sales Lead, Solutions Consulting (Pre-Sales)
New York City, USA
Serves as a senior strategic partner to prospects and clients, leading complex, high-value consultative sales engagements to close new business and expand Nielsen’s relationships with a diverse set of accounts Requires s
Bilingual Spanish Field Sales Representative
San Francisco, USA
As a Field Sales Representative, you'll be the face of Nielsen, recruiting households to participate in Nielsen ratings. Our field sales representatives spend 75% or more of their week in the field driving to and recruit
Senior Financial Planning & Analysis Partner I (MDG Finance)
Mumbai, IND
About the Role As a Senior Analyst in Master Data Governance within our FP&A organization, you will serve as the critical bridge between finance operations, commercial/supply chain teams, system architecture, and strateg
Senior Legal Counsel, Privacy and Data Protection
, GBR
This position can be done remotely within the UK. A central-London office is also available for hybrid work if preferred. You will be part of a friendly, international team. Job Summary The Senior Legal Counsel, Privacy
Senior Product Manager, Audience Measurement & Media Data
, USA
The Senior Product Manager, Audience Measurement & Media Data will lead a critical area of measurement innovation across streaming, broadcast, and cable television. This role will shape the products, platforms, and data
Senior Product Manager I
Bengaluru, IND
The Senior Product Manager I will help shape and advance reporting capabilities for Nielsen's Ads and Outcomes measurement portfolio. These products help clients understand advertising performance, audience delivery, and
Senior Legal Counsel, Privacy and Data Protection
, GBR
This position can be done remotely within the UK. A central-London office is also available for hybrid work if preferred. You will be part of a friendly, international team. Job Summary The Senior Legal Counsel, Privacy
Senior Product Manager II (multi-country quantitative research)
Bengaluru, IND
Sales Lead, Solutions Consulting (Pre-Sales)
New York City, USA
Bilingual Spanish Field Sales Representative
San Francisco, USA
Senior Financial Planning & Analysis Partner I (MDG Finance)
Mumbai, IND
Senior Legal Counsel, Privacy and Data Protection
, GBR
Senior Product Manager, Audience Measurement & Media Data
, USA
Senior Product Manager I
Bengaluru, IND
Senior Legal Counsel, Privacy and Data Protection
, GBR