smartrecruiters
Sr. Software Engineer
LinkedIn
Bengaluru, IND
Full-time
Senior · 5+ years experience
Hybrid
Discovered 2 weeks ago
azureazure-data-factorydatabricksdelta-lakefastapiflask
Key Skills for This Role
azureazure-data-factorydatabricksdelta-lakefastapiflask
Full Job Posting
Responsibilities
- Design, develop, and maintain scalable enterprise Data Warehouse and Lakehouse solutions using Azure Databricks, Azure Data Factory, Azure Data Lake Storage, and SQL Server.
- Build and optimize batch and incremental data pipelines using Python, PySpark, SQL, and distributed data processing frameworks.
- Develop secure and reliable data integrations with enterprise applications using REST/SOAP APIs, SFTP, and database connectivity.
- Implement data modeling, data quality, metadata management, lineage, and governance capabilities to deliver trusted, analytics-ready datasets.
- Design and maintain highly available, secure, and performant data platforms with monitoring, automation, and disaster recovery capabilities.
- Drive engineering excellence through architectural decisions, CI/CD, automated testing, performance optimization, and technical mentorship.
Basic Qualifications
- Bachelor's degree in Computer Science or a related technical discipline, or equivalent practical experience.
- 5+ years of experience designing and developing enterprise data warehouses, data pipelines, or distributed data processing solutions.
- Hands-on programming experience with Python, PySpark, and SQL.
- Experience with cloud-based data platforms, ETL/ELT frameworks, and relational databases.
- Experience designing and developing backend services, APIs, or applications using modern software engineering principles.
- Understanding of data modeling, distributed systems, software architecture, data structures, algorithms, and production engineering practices.
- Experience with Azure Data Factory, Azure Databricks, Azure Data Lake Storage Gen2, and Delta Lake.
- Experience with Microsoft SQL Server, including T-SQL, performance tuning, and high-availability architectures.
- Experience integrating enterprise applications through REST/SOAP APIs, SFTP, and secure data connectivity.
- Experience developing Python-based services and APIs using frameworks such as FastAPI or Flask.
- Familiarity with modern application architectures, microservices, asynchronous processing, and event-driven systems.
- Familiarity with LLMs, embeddings, RAG architectures, vector search, and enterprise AI application development.
- Knowledge of data governance, metadata management, data lineage, and data quality frameworks.
- Experience with Azure security, CI/CD, automated testing, monitoring, and production support.
- Familiarity with Azure Analysis Services, Power BI, and real-time data processing technologies.
- Data Engineering: Python, PySpark, SQL, Apache Spark
- Azure Data Platform: Databricks, Data Factory, ADLS Gen2, SQL Server
- Data Architecture: Data Warehousing, ETL/ELT, Data Modeling, Data Governance
Preferred Qualifications
- Experience with Azure Data Factory, Azure Databricks, Azure Data Lake Storage Gen2, and Delta Lake.
- Experience with Microsoft SQL Server, including T-SQL, performance tuning, and high-availability architectures.
- Experience integrating enterprise applications through REST/SOAP APIs, SFTP, and secure data connectivity.
- Experience developing Python-based services and APIs using frameworks such as FastAPI or Flask.
- Familiarity with modern application architectures, microservices, asynchronous processing, and event-driven systems.
- Familiarity with LLMs, embeddings, RAG architectures, vector search, and enterprise AI application development.
- Knowledge of data governance, metadata management, data lineage, and data quality frameworks.
- Experience with Azure security, CI/CD, automated testing, monitoring, and production support.
- Familiarity with Azure Analysis Services, Power BI, and real-time data processing technologies.
- Suggested Skills
- Data Engineering: Python, PySpark, SQL, Apache Spark
- Azure Data Platform: Databricks, Data Factory, ADLS Gen2, SQL Server
- Data Architecture: Data Warehousing, ETL/ELT, Data Modeling, Data Governance
- India Disability Policy
- LinkedIn is an equal employment opportunity employer offering opportunities to all job seekers, including individuals with disabilities. For more information on our equal opportunity policy, please visit https://legal.linkedin.com/content/dam/legal/Policy_India_EqualOppPWD_9-12-2023.pdf
- Global Data Privacy Notice and Compliance Posters for Job Candidates
- Please use this link to access documents that provide information about how LinkedIn handles the personal data of employees and job applicants, as well as the E-Verify Participation Notice and the Department of Justice Immigrant and Employee Rights Section Right to Work posters: https://www.linkedin.com/legal/candidate-portal.
- LinkedIn is the world's largest professional network, built to create economic opportunity for every member of the global workforce. Our products help people make powerful connections, discover exciting opportunities, build necessary skills, and gain valuable insights every day. We're also committed to providing transformational opportunities for our own employees by investing in their growth. We aspire to create a culture that's built on trust, care, inclusion, and fun – where everyone can succeed.
- Join us to transform the way the world works.
Apply for this job in 1 click
Skip the repetitive application forms
Install the Base Career Chrome Extension and autofill job applications across major job boards with your profile.
Trusted by over 500,000 job seekers on Base Career
More from this employer
More jobs at LinkedIn
Staff Software Engineer
, USA
Discovered TodayFull-time
GTM Transformation Lead
Chicago, USA
Discovered TodayFull-time
Senior Associate, Sales Analytics
San Francisco, USA
Discovered TodayFull-time
GTM Transformation Lead
New York City, USA
Discovered TodayFull-time
Software Engineering Manager, Core Storage
Bellevue, USA
Discovered 2 days agoFull-time
Sr. People Analytics Analyst
Sunnyvale, USA
Discovered 3 days agoFull-time
Director, Technical Program Management (Infra)
, USA
Discovered 3 days agoFull-time
Sr. Event Marketing Manager
Sunnyvale, USA
Discovered 3 days agoFull-time