Principal Digital Specialist - Principal Data Engineer (Azure)

Together we are Trusted to Serve Scotland. Principal Data Engineer (Azure) We are seeking a Principal Data Engineer to join our Business Analytics team and lead the development of our cloud-based data platform. Our team’s mission is to turn data into actionable insights that support better decision‑making and deliver measurable value for the business. To enable this, we have built an Azure data platform built on lakehouse architecture. The platform leverages Fabric and Azure Databricks and provides a secure, governed and trusted foundation for our data. The platform offers a consistent and safe environment to develop, deploy and scale data products and AI/ML solutions, ensuring insights can be shared efficiently and used with confidence across the organisation. We are an ambitious team that uses data, advanced analytics and cloud technologies to design and deliver digital solutions that help colleagues drive improvements in key business areas, including asset management and water quality. We work closely with stakeholders through an inclusive and collaborative approach, ensuring we focus on the right problems and deliver outcomes that are genuinely useful. Our Business Analytics function brings together a strong pool of talent and provides excellent opportunities for learning, development and growth. The role As Principal Data Engineer, you will be the senior technical authority for data engineering in our organisation. This is a hands-on leadership role with no direct line‑management duties, where you will set the standards and technical direction for our data engineering practice. You will guide the design and implementation of robust data pipelines and architectures on Azure, ensure high‑quality engineering outcomes through code reviews and mentoring, and spearhead innovation in tools and techniques. In this role, you will work closely with data scientists, solution developers, enterprise architects and platform engineers to ensure our data ecosystem is reliable, scalable and delivers value. You will also act as the key technical liaison with our external Digital partners, ensuring alignment with our standards and providing technical assurance on joint initiatives. This is a fantastic opportunity to shape the future of data engineering in a forward‑thinking, data‑driven organisation while staying at the forefront of industry developments. What you will do You will own and evolve data engineering standards and guiding technical principles across the Azure data analytics estate, acting as the technical authority for data engineering and ensuring high‑quality, well‑assured engineering outcomes. This includes gatekeeping pull requests in Azure DevOps for changes to production data pipelines and leading the research and development of new tools, patterns and platform capabilities within the analytics environment. You will design and assure technical designs for all projects on the Analytics platform and undertake hands‑on development for proof‑of‑concept work and complex technical initiatives. Collaboration is central to the role, and you will work closely with analytics, data science, architecture and platform teams, as well as with the Platform Product Owner and Technical Support Team Lead to transition established solutions into support. What you'll bring Essential skills and experience You will bring significant hands‑on experience as a senior or principal Data Engineer on Microsoft Azure, along with strong experience of modern Azure data platforms such as Azure Data Lake, Microsoft Fabric and Databricks. You will have a proven ability to design and build scalable, secure data pipelines and lakehouse architectures, supported by strong programming skills, for example Python and SQL, with Spark experience desirable. You will have expertise working within DevOps CI/CD environments using Azure DevOps and a solid understanding of data quality, security and governance in regulated environments. Excellent communication skills are essential, along with the ability to influence through technical expertise. Desirable Experience with streaming or IoT data would be advantageous, along with familiarity with Infrastructure as Code, for example Terraform or Bicep. Azure certifications are also desirable. Looking out for you While the work we do is important, we know it’s not the only thing that matters. That’s why we make sure you have everything you need to find a good work-life balance. This is a full time (35 hours per week) role, but you’ll get plenty of time to recharge with 38 days of holiday every year (including public holidays). And you can even buy five more if you need a little extra time. You can also take paid time off to volunteer in your community. When it comes to major life events we have family-friendly policies around adoption, maternity, and parental leave. And for the everyday? We embrace flexible working. For your future, there’s our pension (the ‘know-what-you’ll-get-when-you-retire’ kind), as well as life assurance. As for some nice extras? You can expect an annual company bonus. Not to mention access to SW Splash – handy rewards that can save you money on holidays, bills, and shopping. The not-so-small print This role will be a Grade 8 level. If you’re new to Scottish Water, you’ll join us on a salary between £54,776 - £74,386. If you’re interested in this role – or know someone else who might be - we’ll be accepting applications until midnight on the 8th September 2026 We operate Balanced Hybrid working at Scottish Water and we encourage people to work half their time in a Scottish Water location. Job offers at Scottish Water are conditional, and will be confirmed on completion of our pre-employment screening, e.g. referencing, criminal record checks, license checks, etc. These checks are conducted on our behalf by a third party. Don’t worry - we’ll ask your permission before starting these checks.