Machine Learning Quality Engineer

• The Corporate Engineering AI team is the central enablement and platform delivery function for LSEG's internal agentic AI ecosystem
• The team's mission is to scale safe, high-quality AI capabilities across the enterprise by providing shared platforms, patterns, governance, and delivery support
• CE AI owns and operates core AI platforms including LSEG AI Assist, the Question Answering Service (QAS), and the Internal MCP Gateway
• Rather than delivering individual business use cases end-to-end, the team enables product engineering groups across LSEG to expose knowledge, data, and actions to AI agents in a consistent, governed, and repeatable way
• The team operates a Central MCP Delivery model: building critical MCP tools and services \"for\" product teams where required, while simultaneously defining standards, patterns, and platform capabilities that allow teams to progressively move towards self-service contribution
• This programme delivers an LSEG-owned, production-grade agentic AI platform with MCP as its extensibility layer
• The scope of work includes:
• Building and operating LSEG AI Assist, an in-house agentic experience capable of reasoning, planning, and tool-calling
• Operating QAS, the enterprise RAG and search layer used to ground agent responses in approved data sources
• Delivering a production Internal MCP Gateway providing discovery, security, policy enforcement, observability, and lifecycle management for MCP tools and Skills
• Designing and building MCP servers and Skills that expose internal and vendor systems safely to agents
• Establishing evaluation, quality control, and governance mechanisms so MCP tools and Skills can be promoted through PTB/PTO and operated with confidence at scale
• Quality Engineers - Responsibilities & Skills:
• Define and implement evaluation frameworks for MCP tools and Skills covering correctness, safety, and regression impact
• Build and maintain automated test pipelines for agentic behaviours, including tool invocation and multi-step workflows
• Evaluate and mitigate agentic failure modes such as hallucination, tool misuse, invalid inputs, and latency amplification
• Produce testing evidence required for Permit to Build (PTB) and Permit to Operate (PTO)
• Partner with ML Engineers to embed testability and evaluation hooks into MCP servers and Skills
• Help define the long-term quality and governance model for federated MCP contributions across LSEG

• Strong Python experience for test harnesses and automation
• Experience with LLM and RAG evaluation frameworks or custom evaluation pipelines
• Test automation expertise covering unit, integration, and regression testing
• Understanding of agentic system risks and failure modes
• Ability to assess solutions against governance, security, and audit expectations
• Experience working in regulated or highly governed engineering environments

#J-18808-Ljbffr

Full job description from WhatJobs. Apply for this role →