Databricks RSA - DPP Level - SC Cleared

Role overview

A Databricks Partner Program (DPP) vetted Resident Solutions Architect with active SC clearance, embedded with a UK public sector client to design, govern and deliver a production Databricks Lakehouse.

Job title

• Databricks Resident Solutions Architect (DPP), SC Cleared
• Engagement: Contract, 6 months initial with strong extension potential
• Clearance: Active SC required from day one
• Location: London
• Hybrid vs remote: client site attendance as required
• Rate: DOE - Outside IR35
• Start: ASAP

About the engagement:

Our client is a UK Databricks partner consultancy delivering data modernisation for central government.

The programme moves sensitive, regulated datasets off legacy warehouses and siloed cloud storage onto a governed Lakehouse. Data must stay within UK sovereign boundaries, access must be tightly controlled, and cost efficiency matters under ongoing public spending pressure. You will be the senior Databricks authority on site, trusted by both the client and Databricks.

Key responsibilities:

• Lead the architecture and build of a secure Databricks Lakehouse on Azure or AWS, using medallion (bronze, silver, gold) design
• Design and implement Unity Catalog governance: catalogues, schemas, role-based access control, row and column level security, lineage and audit for sensitive government data
• Build production-grade ingestion and pipelines with Lakeflow Connect, Lakeflow Declarative Pipelines (DLT) and Workflows, moving beyond notebooks to tested, deployable code
• Establish CI/CD and platform engineering standards using Databricks Asset Bundles (DABs), Terraform and Azure DevOps or GitHub Actions
• Lead migrations from legacy platforms (SQL Server, Oracle, SAS, Hadoop, Synapse) with clear cut-over and reconciliation plans
• Monitor and optimise DBU consumption and cluster policies, giving the client clear cost visibility
• Enable MLOps and GenAI use cases where in scope, including MLflow, Model Serving and Genie
• Act as trusted advisor to client stakeholders, running workshops, design authorities and architecture reviews
• Coach and upskill client and partner engineers, leaving the client self-sufficient
• Support pre-sales and RFP responses with solution design and estimates

Essential skills and experience:

• Active UK SC clearance (please state the sponsoring organisation and expiry date)
• Current DPP vetting as a Databricks Resident Solutions Architect, or equivalent Databricks-approved delivery status
• 4+ years hands-on Databricks delivery, with at least two end-to-end production implementations as lead or architect
• Deep Unity Catalog experience: you have configured it, not just used it, including RBAC on sensitive data
• Strong PySpark, Spark SQL and Delta Lake, including performance tuning at scale
• Production pipeline experience with Lakeflow / DLT, Workflows and DABs

Desirable:

• Databricks Certified Data Engineer Professional, Solutions Architect Champion, or Databricks Champion status
• Databricks Summit, user group or community speaking
• MLflow, Model Serving, Mosaic AI or Genie delivery experience
• Lakebase, Delta Sharing or Lakehouse Federation experience
• Knowledge of GDS Service Standard, NCSC cloud security principles and government data classifications
• DV clearance or willingness to undergo DV vetting

#J-18808-Ljbffr

Full job description from WhatJobs. Apply for this role →