Senior Data Engineer (Databricks/AWS)
Role Overview
The Senior Data Engineer designs, delivers, and evolves enterprise data platforms and applications for critical regulatory and financial reporting.
The hands-on role uses AWS, Databricks, and the Cloudera Big Data ecosystem to build high-throughput batch and streaming pipelines.
The engineer partners with technology teams, platform providers, business users, and senior stakeholders to deliver secure, governed, reliable solutions.
Key Skills for This Role
Full Job Posting
Role overview
The Senior Data Engineer designs, delivers, and evolves enterprise data platforms and applications for critical regulatory and financial reporting.
The hands-on role uses AWS, Databricks, and the Cloudera Big Data ecosystem to build high-throughput batch and streaming pipelines.
The engineer partners with technology teams, platform providers, business users, and senior stakeholders to deliver secure, governed, reliable solutions.
Responsibilities
- Lead implementation of regulatory and financial reporting solutions; manage risks and complex performance issues to ensure accurate, timely submissions.
- Support application system development, feasibility studies, time and cost estimates, IT planning, risk technology, and model development.
- Oversee analysis, design, construction, testing, and implementation, and provide user and operational support.
- Troubleshoot and optimize Spark jobs, Hive and Impala queries, and HBase operations to meet deadlines.
- Implement data governance, security, management, lineage, and compliance controls; recommend security measures based on post-implementation analysis.
- Automate batch processing, scheduling, and monitoring; identify data quality issues and support data wrangling, variance analysis, and reporting accuracy.
- Analyze complex application and business-process issues, consult with users and technology groups, and recommend programming solutions.
- Liaise with platform teams and vendors to resolve platform issues and maintain a stable reporting environment.
- Align stakeholders on requirements and risks, define operating standards, and advise or coach analysts as a subject matter expert.
- Assess risk, follow applicable laws and policies, and escalate or report control issues.
Qualifications
- At least five years designing and implementing large-scale data processing systems for regulatory and financial reporting.
- At least five years of data engineering experience on platforms such as Cloudera, AWS, or Databricks, including independent production pipeline delivery.
- Hands-on AWS S3 and Glue experience; hands-on Databricks experience with Delta Lake, Unity Catalog, and Delta Live Tables.
- Advanced PySpark, Scala, and SQL skills for tested, high-throughput batch and streaming ETL pipelines.
- Deep Apache Spark tuning experience and working knowledge of HDFS, YARN, Hive, Impala, and HBase.
- Experience with real-time Kafka solutions; Confluent Kafka is preferred.
- Experience with data governance, security, and lineage controls, including relevant tools and encryption in transit and at rest.
- Experience with Git and CI/CD, batch scheduling and automation, and monitoring and alerting tools.
- Strong troubleshooting, data analysis, data quality, and variance analysis skills; ability to work independently under pressure and manage changing deadlines.
- Bachelor's or university degree, or equivalent experience.
- Angular or React UI development for data applications and dashboards is a plus.
Technology environment
- AWS S3 and Glue; Databricks, Delta Lake, Unity Catalog, and Delta Live Tables.
- Cloudera, Hadoop/HDFS, YARN, Hive, Impala, HBase, and Kafka.
- PySpark, Scala, SQL, and Apache Spark; batch and streaming ETL.
- Git, Jenkins or GitHub Actions; Autosys, Control-M, or Apache Airflow; Grafana, Prometheus, Splunk, or ELK Stack.
- Governance and security technologies may include Apache Ranger, Apache Atlas, Kerberos, and Unity Catalog.
Compensation
- The listed full-time salary range is $94,300–$141,500; no compensation cadence is specified.
About Citi
Citi provides financial products and services to consumers, corporations, governments, and institutions around the world.
Enterprise Operations & Technology develops and operates technology supporting secure, reliable, and efficient services.
Apply for this job in 1 click
Skip the repetitive application forms
Install the Base Career Chrome Extension and autofill job applications across major job boards with your profile.
Trusted by over 500,000 job seekers on Base Career
More from this employer
More jobs at Citi
Corporate Banking Summer Intern Analyst, Dubai, UAE (UAE Nationals Preferred)
Dubai, UAE
Citi seeks summer analysts for its UAE Corporate Banking team in a 10-week internship program. Analysts will perform financial and credit analysis, support client ideas, and assist with deal structuring; applicants shoul
Corporate Banking, Full Time Analyst, Dubai
Dubai, UAE
Citi seeks full-time analysts for its UAE Corporate Banking team, with training in corporate banking fundamentals. The role supports financial and credit analysis, client solutions, and transaction structuring. Applicant
Corporate Banking, Placement Analyst, Dubai, UAE
Dubai, UAE
Citi seeks a placement analyst to support its Middle East & Africa Corporate Banking team in client relationships, financial analysis, and transaction execution. Candidates should be university students or recent graduat
Application Support Tech Lead Analyst - Vice President
Mississauga, CAN
Applications Development Senior Analyst
, UAE
Discover your future at Citi -------------------------------- Working at Citi is far more than just a job. A career with us means joining a team of approximately 219,000 dedicated people from around the globe. At Citi, y
Applications Development Intermediate Analyst
, UAE
Discover your future at Citi -------------------------------- Working at Citi is far more than just a job. A career with us means joining a team of approximately 219,000 dedicated people from around the globe. At Citi, y
Application Support Tech Lead Analyst - Vice President
Mississauga, CAN
Corporate Banking Summer Intern Analyst, Dubai, UAE (UAE Nationals Preferred)
Dubai, UAE
Corporate Banking, Full Time Analyst, Dubai
Dubai, UAE
Corporate Banking, Placement Analyst, Dubai, UAE
Dubai, UAE
Application Support Tech Lead Analyst - Vice President
Mississauga, CAN
Applications Development Senior Analyst
, UAE
Applications Development Intermediate Analyst
, UAE
Sr Python Developer - VP
Tampa, USA
Application Support Tech Lead Analyst - Vice President
Mississauga, CAN