Data Engineer
Key Skills for This Role
Full Job Posting
Description & Requirements
- On behalf of National Crime Agency, we are looking for a Data Engineer Inside IR35 for a 6 month contract based REMOTELY.
- As a Data Engineer , your main responsibilities will be:
- SC/DV Clearance is an essential requirement for this role, as a minimum you must be willing & eligible to undergo checks. Please note, due to the exceptional requirements of this position (short-term nature of this role and speed at which we require a postholder in situ) preference may be given to candidates who meet all of the essential criteria and hold active security clearance.
- Key Responsibilities
- Design, build and maintain scalable data pipelines within Microsoft Azure
- Migrate and transform data from legacy Oracle systems into cloud-based platforms
- Develop and support ETL/ELT processes using Azure Data Factory and Azure Databricks
- Perform data mapping, schema transformation and data modelling activities
- Manage and optimise data movement into Azure Data Lake environments
- Work with Delta Lake/Delta formats to support efficient data storage and processing
- Ensure high levels of data quality, integrity and reliability across all datasets
- Troubleshoot and resolve data-related issues and pipeline failures
- Support both batch and real-time (streaming) data processing requirements
- Produce clear technical documentation, including data dictionaries and metadata records
- Contribute to data governance activities including data lineage, classification and quality controls
- Design secure, scalable and high-performing data solutions
- Work collaboratively with project teams and stakeholders to meet programme objectives
- Deliver knowledge transfer and handover sessions to ensure long-term supportability
- Essential Skills & Experience
- Proven experience working as a Data Engineer in large-scale data migration projects
- Strong experience migrating data from Oracle databases into Microsoft Azure
- Hands-on expertise with Azure Databricks
- Experience using Azure Data Factory (ADF)
- Experience working with Azure Data Lake technologies
- Knowledge of Delta Lake / Delta formats
- Strong ETL/ELT development experience
- Data mapping and schema transformation expertise
- Excellent SQL and data integration skills
- Understanding of data governance principles , including: Data lineage Data classification Data quality management Metadata management
- Data lineage
- Data classification
- Data quality management
- Metadata management
- Experience creating and maintaining technical documentation and data dictionaries
- Ability to build secure, scalable and performant data solutions
- Strong stakeholder communication skills
- Desirable Experience
- Azure Synapse Analytics
- Microsoft Fabric
- Python and/or PySpark
- Spark-based data processing
- Experience within government, policing, defence or other highly regulated environments
- Active SC Clearance
- Active SC/DV Clearance (Please select level of clearance required. For roles not requiring clearance please delete completely and delete this note)
Disability Confident
As a Disability Confident employer, CA encourage applications from disabled and neurodivergent candidates . Applications from candidates who identify as having a disability and meet the essential criteria for the role will be prioritised for review. Where application volumes are high, additional role-specific and desirable criteria may be applied as part of the shortlisting process which may include holding active security clearance.
In applying for this role, you acknowledge the following "this role falls in scope of the Off Payroll Working in the Public Sector legislation. Any rates of payment quoted will reflect the gross rate per day for the assignment and will be subject to appropriate taxes and statutory costs. As such the payment to the intermediary and your income resulting from this contract will be different".
About National Crime Agency
Verified company details for this employer are not available yet.
Apply for this job in 1 click
Skip the repetitive application forms
Install the Base Career Chrome Extension and autofill job applications across major job boards with your profile.
Trusted by over 500,000 job seekers on Base Career
More from this employer
More jobs at National Crime Agency
Officer Development Programme (ODP) for Trainee Investigator, Intelligence Officer and Intelligence Analyst
Birmingham, GBR
The National Crime Agency is recruiting trainees for its two-year Officer Development Programme, with pathways to become an accredited investigator, intelligence officer or analyst. No prior law-enforcement experience is
Officer Development Programme (ODP) for Trainee Investigator, Intelligence Officer and Intelligence Analyst
Birmingham, GBR
The National Crime Agency is recruiting trainees for a structured, two-year programme leading to an accredited Investigator, Intelligence Officer, or Analyst pathway. No prior law-enforcement experience is required; trai
Officer Development Programme (ODP) for Trainee Investigator, Intelligence Officer and Intelligence Analyst
Birmingham, GBR
The National Crime Agency is recruiting trainees for a two-year programme leading to accreditation as an investigator, intelligence officer or analyst. No prior law enforcement experience is required; trainees complete s
Officer Development Programme (ODP) for Trainee Investigator, Intelligence Officer and Intelligence Analyst
Birmingham, GBR
The National Crime Agency is recruiting trainees for a two-year development programme with investigation, intelligence officer and analyst pathways. Participants receive structured training and may be placed in an approp
Officer Development Programme (ODP) for Trainee Investigator, Intelligence Officer and Intelligence Analyst
Birmingham, GBR
The National Crime Agency is recruiting trainees for a two-year programme leading to accreditation as an Investigator, Intelligence Officer or Intelligence Analyst. Trainees develop pathway-specific competencies through
Officer Development Programme (ODP) for Trainee Investigator, Intelligence Officer and Intelligence Analyst
Birmingham, GBR
The National Crime Agency is recruiting trainees for a two-year programme leading to an accredited investigator, intelligence officer, or analyst pathway. No prior law-enforcement experience is required; trainees complet
Officer Development Programme (ODP) for Trainee Investigator, Intelligence Officer and Intelligence Analyst
Birmingham, GBR
The National Crime Agency is recruiting trainees for a two-year development programme leading to an investigator, intelligence officer or analyst pathway. No prior law-enforcement experience is required; trainees must co
Officer Development Programme (ODP) for Trainee Investigator, Intelligence Officer and Intelligence Analyst
Birmingham, GBR
The National Crime Agency is recruiting trainees for a two-year development programme leading to an investigator, intelligence officer or analyst pathway. Trainees develop and evidence role-specific competencies, complet
Officer Development Programme (ODP) for Trainee Investigator, Intelligence Officer and Intelligence Analyst
Birmingham, GBR
Officer Development Programme (ODP) for Trainee Investigator, Intelligence Officer and Intelligence Analyst
Birmingham, GBR
Officer Development Programme (ODP) for Trainee Investigator, Intelligence Officer and Intelligence Analyst
Birmingham, GBR
Officer Development Programme (ODP) for Trainee Investigator, Intelligence Officer and Intelligence Analyst
Birmingham, GBR
Officer Development Programme (ODP) for Trainee Investigator, Intelligence Officer and Intelligence Analyst
Birmingham, GBR
Officer Development Programme (ODP) for Trainee Investigator, Intelligence Officer and Intelligence Analyst
Birmingham, GBR
Officer Development Programme (ODP) for Trainee Investigator, Intelligence Officer and Intelligence Analyst
Birmingham, GBR
Officer Development Programme (ODP) for Trainee Investigator, Intelligence Officer and Intelligence Analyst
Birmingham, GBR