Senior Data Engineer

Responsibilities:

• Platform Engineering & Modernization
• Refactor and modernize existing Spark pipelines to Databricks native architectures
• Eliminate legacy Hadoop dependencies and adopt cloud native AWS patterns
• Enhance and extend existing processing logic using optimized Spark (ScalaSpark) on Databricks
• Databricks Native Development
• Build and optimize solutions using Databricks features, including Delta Lake, Databricks Workflows for orchestration and Auto scaling and job clusters
• Design & Solution Engineering
• Contribute to low and mid level architecture and design
• Translate high level architecture into detailed technical designs
• Define data models, pipeline patterns, and reusable components
• Ensure solutions are scalable, maintainable, and production ready
• Performance Optimization & Simplification
• Analyze, improve Spark job performance and simplify complex or over engineered pipelines into standardized, efficient patterns
• Engineering Standards & Best Practices
• Follow and contribute to Databricks and Spark engineering standards
• Write clean, modular, and testable code
• Contribute to shared frameworks, reusable libraries, and quality standards
• Collaboration & Stakeholder Engagement
• Work closely with senior architects, platform teams, and DevOps engineers
• Provide technical inputs, troubleshooting support, and implementation guidance
• Participate in design discussions and technical decision making
• Testing & Quality Assurance
• Develop unit, integration, and data validation tests
• Support production releases and post deployment validation
• Qualifications:
• Core Technical Skills:
• 8+ years in data engineering or distributed systems
• Strong expertise in Apache Spark (ScalaSpark/ JavaSpark / PySpark), Databricks on AWS, and Delta Lake
• Experience on SQL
• Experience with AWS services and large‐scale distributed data processing
• Modernization & Optimization Experience
• Experience modernizing or refactoring legacy data platforms into cloud‐based architectures
• Strong background in Spark performance tuning and large‐scale batch optimization
• Design Capability
• Ability to translate architecture into implementable designs
• Understanding of data modeling and pipeline orchestration patterns
• Behavioral Competencies
• Strong problem‐solving mindset for complex distributed systems
• Comfortable working in time‐bound, high‐impact environments
• Proactive, accountable, and collaborative
• Clear communication skills across global teams
• Education:
• Bachelor’s degree/University degree and equivalent experience

Job Family Group:

Job Family:

Time Type:

Most Relevant Skills

Other Relevant Skills

Qualifications

• 8+ years in data engineering or distributed systems
• Strong expertise in Apache Spark (ScalaSpark/ JavaSpark / PySpark), Databricks on AWS, and Delta Lake
• Experience on SQL
• Experience with AWS services and large‐scale distributed data processing
• Modernization & Optimization Experience
• Experience modernizing or refactoring legacy data platforms into cloud‐based architectures
• Strong background in Spark performance tuning and large‐scale batch optimization
• Design Capability
• Ability to translate architecture into implementable designs
• Understanding of data modeling and pipeline orchestration patterns
• Behavioral Competencies
• Strong problem‐solving mindset for complex distributed systems
• Comfortable working in time‐bound, high‐impact environments
• Proactive, accountable, and collaborative
• Clear communication skills across global teams
• Bachelor’s degree/University degree and equivalent experience
• Technology
• Applications Development
• Full time
• Most Relevant Skills
• Please see the requirements listed above.
• Other Relevant Skills
• For complementary skills, please see above and/or contact the recruiter.
• Citi is an equal opportunity employer, and qualified candidates will receive consideration without regard to their race, color, religion, sex, sexual orientation, gender identity, national origin, disability, status as a protected veteran, or any other characteristic protected by law.
• If you are a person with a disability and need a reasonable accommodation to use our search tools and/or apply for a career opportunity review Accessibility at Citi.
• View Citi’s EEO Policy Statement and the Know Your Rights poster.