Senior Data Engineer

Data Engineer

Location: Cheltenham, Manchester or Southampton
Salary: £80,000 to £100,000
Working pattern: Full time

We are looking for an experienced Data Engineer to join a growing National Security technology team delivering mission critical data solutions for UK customers.

This is an opportunity to work on complex data engineering challenges across data platforms, data pipelines, cloud technologies, analytics and distributed computing , supporting projects where security, reliability and performance are critical.

You will work closely with customers and multidisciplinary engineering teams to design, develop and maintain data services that transform large and complex datasets into valuable information for downstream analytics, intelligence and reporting.

The role

• Design, develop and maintain data pipelines and data ingestion services
• Develop data pipeline orchestration solutions and tooling
• Integrate and enrich data from multiple and diverse sources
• Build reliable and scalable data processing solutions
• Work with large datasets using distributed computing technologies
• Develop data services that support analytics, reporting and downstream applications
• Apply appropriate data security measures when working with sensitive information
• Monitor data systems and identify opportunities for performance optimisation
• Investigate and resolve issues across data pipelines and data systems
• Work as part of an Agile Scrum team
• Help decompose user requirements and key results into epics and user stories
• Write clean, secure and maintainable code
• Follow a test driven development approach
• Develop reusable code and engineering solutions
• Work closely with customers and other technical specialists to understand and solve complex problems

Essential experience

We are looking for a Data Engineer with experience in:

• Data engineering and data pipeline development
• Python or Java
• AWS and cloud based data technologies
• Apache NiFi
• Big Data technologies such as Apache Spark, Apache Hadoop or Apache Pig
• Data ingestion, integration and transformation
• Distributed data processing
• Data quality and data security
• Software development and engineering best practices
• Version control and collaborative software development
• Working within Agile or Scrum development environments

Desirable experience

Experience in any of the following would be beneficial:

• National Security or Intelligence environments
• Defence or Government projects
• Highly secure or sensitive data environments
• Cloud data platforms
• DevOps or SRE
• Platform Engineering
• AI, Machine Learning or Data Science environments
• Data analytics and intelligence applications
• Automated testing and continuous integration

Security clearance

This role requires candidates to hold, or be eligible to obtain, the highest level of UK national security clearance.

Due to the nature of the work, candidates will need to satisfy the relevant UK security and residency requirements.

Candidates who already hold DV clearance are particularly relevant and should make this clear when applying.

Why join?

You will be joining a specialist technology environment working across Data Engineering, AI, Data Science, Cyber, Cloud, DevOps and Platform Engineering .

The organisation has secured long term work within the National Security domain, providing significant opportunity to work on complex technical challenges while developing your career across a broad range of emerging technologies.

#J-18808-Ljbffr

Full job description from WhatJobs. Apply for this role →