Senior Data Architect

Principal Data & AI Engineer

Permanent

Hybrid - Winchester/Basingstoke

Must hold active SC clearance

About the Role

We're partnering with an innovative UK technology company developing a next-generation AI platform for critical defence and healthcare applications. As they continue to scale, they're looking for a Principal Data & AI Engineer to provide technical leadership across the design, architecture, and delivery of enterprise-grade AI and data platforms.

This is a rare opportunity to take ownership of the data and AI engineering function within a rapidly growing business, shaping the technical direction of products that solve complex, real-world problems. You'll work alongside AI researchers, software engineers, and product teams to build highly secure, scalable data platforms that power advanced AI capabilities in mission-critical environments.

If you're passionate about modern data engineering, large-scale AI systems, and influencing technical strategy while remaining hands-on, this role offers genuine autonomy and long-term career progression.

What You'll Be Doing

• Lead the architecture and development of scalable AI and data platforms supporting advanced machine learning and LLM workloads.
• Design and build high-performance data pipelines capable of processing structured and unstructured data from multiple secure sources.
• Architect robust data ingestion, transformation and orchestration frameworks using modern cloud-native tooling.
• Build and optimise Retrieval-Augmented Generation (RAG) pipelines and enterprise knowledge retrieval platforms.
• Develop and optimise vector search capabilities using technologies such as Weaviate, Pinecone, Milvus or Elasticsearch.
• Engineer highly available APIs and backend services using Python (FastAPI/Flask) and modern software engineering practices.
• Design secure, containerised platforms using Docker and Kubernetes for deployment into restricted environments.
• Drive best practice around data modelling, governance, observability, testing and platform reliability.
• Mentor and provide technical leadership to engineers across the wider software and AI teams.
• Work closely with stakeholders to translate complex business challenges into scalable technical solutions.

What We're Looking For

• Significant commercial experience as a Senior, Lead or Principal Data Engineer, AI Engineer or Platform Engineer.
• Proven experience designing and delivering large-scale production data platforms.
• Strong expertise in Python and SQL, alongside modern software engineering principles.
• Deep understanding of distributed data processing and orchestration technologies (e.g. Spark, Kafka, Airflow, Dagster or similar).
• Experience building AI-enabled platforms using Large Language Models (LLMs), embeddings, vector databases and RAG architectures.
• Hands-on experience with vector databases such as Pinecone, Weaviate, Milvus or Elasticsearch.
• Strong understanding of cloud-native architecture, Docker and Kubernetes.
• Experience designing resilient microservice architectures and RESTful APIs.
• Knowledge of data governance, security, metadata management and production data operations.
• Comfortable leading technical direction, mentoring engineers and influencing architecture decisions.
• Excellent communication skills with the ability to engage technical and non-technical stakeholders.
• Degree in Computer Science, Software Engineering, Data Engineering or a related discipline (or equivalent commercial experience).

Desirable Experience

• Experience deploying AI platforms into secure or highly regulated environments.
• Knowledge of MLOps tooling and model lifecycle management.
• Experience with Azure, AWS or private cloud infrastructure.
• Familiarity with Infrastructure as Code (Terraform, Helm, Ansible or similar).
• Experience within Defence, National Security, Healthcare, Government or other regulated industries.
• Exposure to knowledge graphs, semantic search or graph databases.
• Experience leading technical strategy within a fast-growing technology business.

Security Requirements

Due to the nature of the work, applicants must hold active SC Security Clearance