Data Engineer
Role Overview
The Data Engineer designs and optimizes scalable pipelines, integration solutions, and cloud data platforms for analytics, reporting, and AI/ML initiatives.
The role supports reliable ingestion, high-quality data assets, efficient processing, and collaboration with architects, analysts, and business stakeholders.
Key Skills for This Role
Full Job Posting
About the employer
Blackbuck Insights is a data engineering consulting firm providing data integration, modernization, migration, architecture, platform support, and application services.
Role overview
The Data Engineer designs and optimizes scalable pipelines, integration solutions, and cloud data platforms for analytics, reporting, and AI/ML initiatives.
The role supports reliable ingestion, high-quality data assets, efficient processing, and collaboration with architects, analysts, and business stakeholders.
Data engineering and operations
- Design, develop, and maintain scalable, high-performance pipelines, reusable frameworks, and ETL/ELT processes.
- Integrate structured and unstructured sources and support cloud data platform initiatives.
- Manage batch and real-time ingestion, monitor pipeline performance, troubleshoot processing issues, automate workflows, and ensure timely data delivery.
Quality, platform support, and collaboration
- Implement validation, reconciliation, quality controls, governance, metadata management, lineage, and observability practices.
- Support Meridian implementation, integrations, testing, deployment, and operational support.
- Provide technical guidance, participate in reviews, collaborate with stakeholders, promote best practices, and document designs and procedures.
Required qualifications
- Bachelor's degree in Computer Science, Engineering, Information Systems, or a related field; 5+ years in Data Engineering or a related discipline.
- Strong enterprise pipeline and integration experience; SQL and one or more of Python, Scala, or Java.
- Experience with ETL/ELT, data warehousing, data lakes, and a cloud platform such as Azure, AWS, or Google Cloud.
- Knowledge of orchestration tools such as Airflow or Azure Data Factory and strong understanding of data quality, governance, and monitoring.
- Experience troubleshooting and optimizing large-scale data processing solutions.
Additional experience listed
- Meridian platform implementation or support; Databricks, Snowflake, Azure Synapse, Microsoft Fabric, or similar platforms.
- Kafka, Spark Streaming, Event Hubs, DevOps, CI/CD, Infrastructure as Code, automated testing, AI/ML pipelines, feature engineering, or relevant certifications are listed as additional qualifications.
Apply for this job in 1 click
Skip the repetitive application forms
Install the Base Career Chrome Extension and autofill job applications across major job boards with your profile.
Trusted by over 500,000 job seekers on Base Career
More from this employer
More jobs at BLACKBUCK iNSIGHTS
Business System Analyst
, IND
The employer seeks an experienced Healthcare Business Systems Analyst to support payer integration and member-intelligence solutions. The role covers requirements, process analysis, data flows, testing, and compliance; c
Sr Cloud Data Engineer
, IND
BLACKBUCK iNSIGHTS is seeking a Senior Cloud Data Engineer to build and optimize enterprise data pipelines and cloud data platforms for analytics and AI/ML. The role also covers data quality, governance, Meridian solutio
Data Analyst
, IND
BLACKBUCK iNSIGHTS is seeking a Senior Data Analyst for its Wealth Management practice to lead data discovery, quality profiling, and source-to-target mapping for enterprise data platforms. The role requires five or more