AI Platform Engineer

Working at CreateFuture

CreateFuture is an AI-native consulting partner where people do work that matters and are supported to do it well. We work alongside organisations such as PayPal, adidas, NatWest, FanDuel and Money Saving Expert, building digital products and services that make a difference while always putting people first.

We’re a team of creators. We write code, shape delivery, build go-to-market strategies, develop AI solutions and create the practices that support our people. We work side by side with our clients, challenging what’s not working and helping them to build the future. Our commitment to craft, quality, and culture has helped us scale to over 600 people in just a few years.

Our UK Benefits

35 days leave (including bank holidays).

• Private medical insurance.
• Enhanced parental and adoption leave.
• Financial coaching + 5% pension match.
• 40 hours of paid learning and development.

(View our full list of UK benefits.

)( is aGreat Place to Work-Certified company and has won Best Workplaces UK multiple years in a row.

Join us on our journey. Let’s create tomorrow, together, today.

About the engagement

Most organisations shipping AI features have no reliable way of knowing whether those features are any good. They have vibes, a demo that went well, and a support queue. We’re engaging two Senior AI Engineers to fix that properly on a major AI platform programme with a client in a high-traffic, heavily regulated consumer sector.

You’ll own the evaluation and observability layer of the platform. That covers the golden datasets, the scoring pipelines, the eval gates that sit in the promotion path, and the production monitoring that tells the client when quality has quietly drifted. It’s the part of AI engineering that decides whether the rest of it can be trusted, and in a regulated business it’s also the part that has to stand up to scrutiny.

This is a hands-on role embedded with the client’s engineers and product people, and a large part of the job is persuasion. An eval result that nobody acts on has cost the programme money and taught it nothing. You need to be the person who can say "this score is real, here's what we should do" and be believed by a team that didn’t ask for your help.

Key Responsibilities

Technical Delivery & Implementation

• Eval platform: Design and build the evaluation pipelines on a platform such as Braintrust, LangSmith, Arize or Weights & Biases, and own the decision about which one fits the problem.
• Datasets and scoring: Curate golden datasets that represent real user behaviour. Design LLM-as-judge and human-in-the-loop scoring pipelines with a clear view of where each one is trustworthy and where it isn’t.
• Production observability: Build token-level tracing, quality signal monitoring, hallucination and refusal detection, and per-use

#J-18808-Ljbffr

Full job description from WhatJobs. Apply for this role →