Azure Databricks Data Engineer

Azure Databricks Data Engineer (Contract) | International Digital Infrastructure Organisation | London (Hybrid)

6-Month Initial Contract | Inside IR35 | 4 Days Onsite, 1 Day Remote

We're partnering with a leading international digital infrastructure organisation undergoing a significant data transformation programme. Operating across multiple markets and supporting critical communications infrastructure, the business is investing heavily in its cloud, data, and analytics capabilities to enable more informed decision-making and unlock greater value from its enterprise data estate.

This is an opportunity to join a high-performing Digital Innovations team at a pivotal stage of that journey. You'll play a key role in building and enhancing a modern Azure Databricks platform that supports business-critical reporting, analytics, and future AI initiatives. Working closely with architects, engineers, and senior stakeholders, you'll help shape the direction of the data platform while delivering scalable, secure, and high-quality data solutions.

What You'll Do

• Design, develop, and optimise Azure Databricks solutions using PySpark, Python, SQL, and Azure-native technologies.
• Build and maintain enterprise-grade ETL/ELT pipelines supporting ingestion, transformation, CDC, validation, and data delivery across multiple business functions.
• Develop and support Lakeflow pipelines, Delta Lake implementations, and modern data engineering frameworks.
• Drive best practices across performance optimisation, security, governance, monitoring, and platform reliability.
• Collaborate with business, data, and technology teams to deliver high-quality data products that support organisational growth and strategic decision-making.

What You'll Bring

• Proven experience designing and delivering solutions within Azure Databricks environments.
• Strong Python, PySpark, SQL, and T-SQL development skills.
• Expertise building and supporting large-scale ETL/ELT pipelines and cloud-based data platforms.
• Strong understanding of Delta Lake, Unity Catalog, data governance principles, and medallion architecture.
• Experience integrating REST APIs, JSON data sources, Azure Data Factory, ADLS, Key Vault, and other Azure services.
• Hands-on experience with Git, CI/CD pipelines, deployment automation, and production support.
• Strong troubleshooting, performance tuning, and optimisation experience across Databricks and SQL environments.
• Excellent stakeholder engagement and communication skills, with the ability to operate effectively across both technical and business teams.

Desirable Experience

• Exposure to Microsoft Fabric.
• Databricks certifications.
• Experience with Terraform and infrastructure-as-code practices.
• Experience working within large, international or regulated environments.

The Opportunity

This role offers the chance to join a business where data is becoming increasingly central to operational performance and strategic growth. You'll have the opportunity to influence a core enterprise platform, work on complex and meaningful data challenges, and contribute to a long-term transformation programme with real organisational impact.

We are committed to protecting the privacy of all our candidates and clients, please visit and for our privacy policy.

#J-18808-Ljbffr