Purpose of the role
To build and maintain the systems that collect, store, process, and analyse data, such as data pipelines, data warehouses and data lakes to ensure that all data is accurate, accessible, and secure.
Accountabilities
- Build and maintenance of data architectures pipelines that enable the transfer and processing of durable, complete and consistent data.
- Design and implementation of data warehoused and data lakes that manage the appropriate data volumes and velocity and adhere to the required security measures.
- Development of processing and analysis algorithms fit for the intended data complexity and volumes.
- Collaboration with data scientist to build and deploy machine learning models.
Role Overview
Join us as a Data Engineer and help build the data platform that will underpin a new standalone payments business. As one of the first permanent engineers in the team, you will play a hands-on role in both shaping the platform and delivering the initial data products that run on it. Working with Databricks and AWS, you will design and build production-grade data pipelines, establish engineering standards and create the reusable patterns, automation and controls that will form the foundation of the platform.
This is not a role where platform engineering is separated from delivery. You will work closely with domain teams, building early data products, identifying common requirements and turning successful approaches into repeatable golden paths that make it easier for future teams to onboard and succeed. As the platform evolves, your focus will increasingly shift from hands-on support towards self-service capabilities, reusable tooling and platform features that enable domain teams to build and operate trusted data products independently.
Company Context
Barclays’ payments acceptance business provides critical infrastructure to the UK economy. In April 2025, we announced a long-term partnership with Brookfield Asset Management to grow and transform the payments acceptance business. For more information on our partnership with Brookfield, please visit Barclays.com.
We support a hybrid working pattern with 3 days per week office-based presence expected. The successful candidate will be based in London.
Essential Skills
- Production Databricks and data pipeline engineering: Hands-on experience building production-grade Databricks platforms and reliable batch and streaming pipelines using Python, PySpark and SQL.
- Software engineering, testing and automated delivery: The ability to create modular, maintainable and reusable components, supported by automated pipeline testing, Git-based workflows and CI/CD deployment.
- Secure and operable platform engineering: Experience engineering systems for sensitive or regulated data, including identity and access control, encryption, secrets management, least privilege, monitoring and production incident resolution.
Highly Valued Skills
- Advanced Unity Catalog expertise: Practical experience configuring lineage, permissions, metadata and governance through Unity Catalog.
- Infrastructure-as-code and Databricks deployment automation: Experience with Terraform, Databricks Asset Bundles and GitHub Actions or equivalent policy-controlled delivery tooling.
- Platform product and self-service engineering: Experience creating paved roads, golden paths, metadata-driven automation or internal developer platforms that enable domain teams to build compliant data products with less manual support.
