CLDRSRC10415 - Junior Data Engineer/Report Developer - Toronto - Onsite

Role overview

Must-have qualifications

• Degree or diploma in Computer Science, Software Engineering, Information Technology, or a related field.
• Experience developing data solutions with Azure Data Factory, Azure Data Lake Storage, Databricks, Power BI, Delta Lake, and Lakehouse architecture principles.
• Experience with Azure DevOps, Git, CI/CD pipelines, and automated deployment.
• Strong Python, SQL, and Spark-based data processing proficiency.
• Knowledge of data governance, security, privacy, and access control.
• Experience with REST APIs, application integration patterns, and automation frameworks.

Data engineering and pipelines

• Design, develop, and maintain scalable batch and near-real-time pipelines with Azure Data Factory, Databricks, and related cloud technologies.
• Build ingestion frameworks to onboard data from source systems into the Ontario Data Platform.
• Develop and optimize ETL/ELT processes for transformation, cleansing, analytics, and reporting.
• Implement data validation, monitoring, and reconciliation to ensure quality and reliability.

Databricks and data storage

• Develop and maintain Databricks notebooks, workflows, and processing jobs using Spark and Lakehouse architecture.
• Support Bronze, Silver, and Gold medallion architecture patterns.
• Monitor and troubleshoot Databricks jobs, workflows, and performance issues.
• Develop and manage Delta Lake tables and optimize workloads for performance and scalability.
• Design Azure Data Lake Storage solutions; implement integration, partitioning, optimization, and archival strategies.
• Support metadata management, data cataloguing, and governance initiatives.

Software development and collaboration

• Develop reusable frameworks, scripts, and utilities to automate data engineering processes.
• Participate in CI/CD, code reviews, testing, and deployment automation; apply version control, test automation, code quality, and secure coding practices.
• Maintain technical documentation, design specifications, and operational procedures; contribute to platform improvements.
• Work with architects, product managers, data scientists, analysts, and business stakeholders to understand requirements and deliver solutions.
• Participate in Agile sprint planning, estimation, and delivery; provide technical guidance and knowledge sharing.
• Participate in architecture reviews and contribute to technology roadmaps and design decisions.

Evaluation breakdown