Data Engineer - Azure, DataBricks, PySpark
- london, england, United Kingdom
- Job type not listed
- £68 - £83 Per Hour
Job Description
London | Hybrid
Up to £75,000 + 25% Bonus + Excellent Benefits
A leading London Market insurer is continuing to invest in its enterprise data platform and is looking for a Data Engineer to join a growing Data & Analytics function.
The Opportunity
Join a growing Data & Analytics team within a leading London Market insurer investing heavily in its cloud data platform. Working with modern technologies including Azure, Databricks, SQL and Python , you'll play a key role in designing, building and maintaining scalable data solutions that enable smarter decision making across Underwriting, Finance and Operations.
You'll collaborate with engineers, analysts and business stakeholders to develop robust data pipelines, improve data quality and optimise cloud-based data processing. This is an excellent opportunity to work on enterprise-scale data engineering projects while helping shape the future of a modern Azure and Databricks platform, with plenty of scope to develop your technical skills and progress your career.
What You'll Be Doing
- Designing and developing cloud-based data pipelines using Azure Data Factory and Databricks.
- Building scalable ETL and ELT solutions to ingest and transform data from multiple business systems.
- Developing data solutions using SQL, Python and PySpark.
- Supporting the deployment of production-ready data pipelines using CI/CD and DevOps best practices.
- Working with business stakeholders to understand data requirements and deliver technical solutions.
- Improving data quality, performance and reliability across the data platform.
- Supporting data modelling, orchestration and cloud data architecture initiatives.
- Troubleshooting production issues and continuously improving engineering standards.
- Collaborating with the wider Data & Analytics team to deliver high-value data products.
What We're Looking For
- Commercial experience as a Data Engineer.
- Experience with Python and/or PySpark.
- Experience working with Microsoft Azure data technologies.
- Exposure to Databricks or a strong desire to develop expertise in the platform.
- Experience building ETL or ELT data pipelines.
- Knowledge of Azure Data Factory or similar integration tools.
- Understanding of Git, source control and CI/CD practices.
- Experience working within Agile delivery environments.
- Strong analytical and problem-solving skills.
- Excellent communication and stakeholder engagement skills.
- A proactive mindset with a genuine enthusiasm for learning new technologies.