Consultant/Senior Consultant, Databricks Data & AI Engineer, Engineering, AI & Data, Technology & Tr

Job description

Connect to your Industry

Do you want to help deliver some of the UK's largest and most ambitious Data & AI transformation programmes?

Technology & Transformation helps organisations unlock the full value of their data through modern platforms, cloud technologies, and AI. Within Deloitte's Engineering, AI & Data practice, we work with leading organisations to transform how they manage, govern, and utilise data, creating the trusted foundations required for advanced analytics, machine learning, and emerging AI capabilities.

As a Databricks Data & AI Engineer, you'll design, build and optimise scalable data & AI solutions using the Databricks Platform, enabling advanced analytics, machine learning, and AI. Working alongside leading architects, engineers, and AI specialists, you'll help clients modernise their data platforms and create the trusted foundations that power intelligent decision‑making.

We're looking for practitioners with a passion for building scalable, high-performance data platforms, strong data engineering fundamentals, and hands‑on experience across modern cloud and data ecosystem.

Connect to your career at Deloitte

Deloitte drives progress. Using our vast range of expertise, we help our clients' become leaders wherever they choose to compete. To do this, we invest in outstanding people. We build teams of future thinkers, with diverse talents and backgrounds, and empower them all to reach for and achieve more.

What brings us all together at Deloitte? It's how we approach the thousands of decisions we make every day. How we behave, our beliefs and our attitudes. In other words: our values. Whatever we do, wherever we are in the world, we lead the way, serve with integrity, take care of each other, foster inclusion, and collaborate for measurable impact. These five shared values lead every decision we make and action we take, guiding us to deliver impact how and where it matters most.

Connect to your opportunity

We're expanding our AI & Data Engineering practice and are looking for people who are natural initiative-takers, who bring out the best in others, are brilliant listeners, and can grow our business without compromising standards, integrity, or culture. You will work with outstanding data, AI, and cloud talent across multiple disciplines to innovate and create powerful solutions for iconic brands and complex organisations.

We are seeking a skilled Databricks Data & AI Engineer who combines strong data engineering capability with practical experience in enabling advanced analytics, machine learning and AI-enabled business solutions with strategic consulting acumen and problem-solving skills. The ideal candidate should possess strong knowledge of the Databricks Data Intelligence Platform and the modern data & AI engineering practices, complemented by a keen ability to translate complex business requirements into robust, scalable Data & AI solutions that create trusted data foundations, AI‑ready products and measurable business value.

As a key member of our team, you will:

• Design, build, test and deploy scalable batch and streaming data pipelines, AI-ready data products, feature pipeline and analytics solutions on the Databricks Platform using Delta Lake, Spark SQL/PySpark, Unity Catalog and Databricks SQL.
• Deliver solutions across the full data & AI engineering lifecycle, from requirements analysis and solution design through development, testing, deployment, operational readiness and continuous improvement.
• Work directly with clients and stakeholders to understand business, data,AI and technical requirements, facilitate design workshops and recommend practical solution options.
• Communicate technical concepts clearly, explaining solution choices, trade-offs, dependencies and recommendations to both technical and non-technical audiences.
• Design and implement robust data models and ETL/ELT patterns that support enterprise reporting, analytics, machine learning and AI use cases.
• Optimise data pipelines and workloads for performance, reliability, scalability and cost, identifying engineering risks and implementing appropriate mitigations.
• Embed security, governance, lineage, access control, data quality and Responsible AI considerations into solutions, including cataloguing and policy enforcement through Unity Catalog.
• Enable AI and machine learning solutions by preparing trusted, governed and reusable datasets, supporting feature engineering, experimentation, model deployment and integration with downstream AI applications.
• Apply modern engineering practices, including version control, automated testing, CI/CD, infrastructure-as-code, monitoring and data observability.
• Produce and maintain high-quality technical artefacts, including solution designs, data models, interface specifications, deployment documentation and operational runbooks.
• Support and guide other engineers, contributing to technical reviews, reusable engineering standards and quality assurance across delivery teams.
• Enable advanced analytics, machine learning Gen AI & Agentic AI solutions by creating trusted, governed , discoverable and production-ready data foundations.

Connect to your skills and professional experience

Overall, you're a curious problem solver and hands-on engineer who enjoys working collaboratively to solve complex data challenges. You thrive in diverse teams, enjoy learning from others, and bring a pragmatic approach to delivering high-quality data solutions that create measurable business value.

You'll have strong technical skills including some or all of the following:

• Proven experience in data engineering, data modelling, and architecture, preferably on a major cloud platform (Azure, AWS, or GCP).
• Strong experience designing and building scalable data platforms, data products and analytics solutions using modern lake house or cloud data warehouse technologies
• Proficiency writing advanced, highly optimised SQL and Spark (PySpark/Scala), with a focus on performance optimisation and maintainability.
• Designing and building solutions on a cloud data warehouse or lakehouse platform (e.g., Databricks, Snowflake, BigQuery, Redshift).
• Strong understanding of data modelling principles for analytics, reporting, machine learning and AI use cases.
• Experience applying modern engineering practices, including CI/CD, automated testing, infrastructure-as-code and data observability.
• Knowledge of cloud-native security, governance and access management practices, including technologies such as Unity Catalog and cloud IAM frameworks.
• Experience working with Databricks Lakehouse Platform, including Delta Lake, Databricks Workflows, Unity Catalog , Lakeflow and Databricks SQL.
• Experience building & enabling AI workloads using Databricks AI capabilities such as Agent Bricks, MLflow, Feature Engineering, Vector Search, Mosaic AI and model serving, with awareness of agent orchestration, RAG and production grade AI agents.

Required

• Strong understanding of Databricks Lakehouse architecture, Medallion design patterns and Delta Lake principles.
• Hands-on experience delivering at least one enterprise-scale Databricks implementation from design through production deployment.
• Proven experience developing and supporting production-grade data pipelines and data products.
• Strong proficiency in Python or Scala for data engineering.
• Experience working directly with business and technical stakeholders to gather requirements and communicate solution approaches.
• Strong analytical, problem-solving and troubleshooting skills

Preferred

• Databricks Certified Data Engineer (Associate or Professional) or Databricks Certified Generative AI Engineer Associate
• Experience with orchestration technologies such as Azure Data Factory, Airflow or Databricks Workflows.
• Knowledge of containerisation and platform engineering technologies such as Docker and Kubernetes.
• Experience enabling machine learning, GenAI or advanced analytics workloads using technologies such as MLflow, Feature Engineering, Vector Search and Mosaic AI capabilities.
• Exposure to DevOps, platform automation and Infrastructure-as-Code tools such as Terraform

Connect to your business - Technology and Transformation

Distinctive thinking, deep expertise, innovation and collaborative working. That's what connects us. That's what makes us Deloitte. If you want to help solve some of the biggest tech and transformational challenges around, join us. Together, we'll make an impact that matters.

Engineering, AI and Data

We lead transformation at the heart of our clients; re-engineering the core of our clients' organisations, helping to unleash growth, and creating better futures for their customers, citizens and employees.

Personal independence

Regulation and controls are standard practice in our industry and Deloitte is no exception. These controls provide important legal protection for both you and the firm. We are subject to a number of audit regulations, one of which requires that certain colleagues abide by specific personal independence constraints (e.g., in relation to any financial interests and employment relationships). This can mean that you and your "Immediate Family Members" are not permitted to hold certain financial interests (shares, funds, bonds etc.) with audit clients of the firm, and also prohibitions on certain employment relationships (e.g., you are not permitted to hold a secondary employment role with SEC audit clients of the firm whilst being employed by the firm). The recruitment team will provide further detail as you progress through the recruitment process or you can contact the Independence team upon request.

Connect with your colleagues

"The great culture, flexibility, and range of experiences that Deloitte offers is what sets this firm apart. In Technology & Transformation, Deloitte is far ahead of any of our immediate competitors." - Sophia, Technology & Transformation

"What attracted me to Deloitte were the endless opportunities and the collective experience of other like-minded individuals. Deloitte's clients include many of the world's largest organisations; I wanted to be part of a team that made a difference that I could be proud of." - Dan, Technology & Transformation

Our hybrid working policy

You will be based in London with hybrid working.

At Deloitte we understand the importance of balancing your career alongside your home life. That's why we'll support you to work flexibly through our hybrid working policy. Depending on the requirements of your role, you'll have the opportunity to work in your local office, virtual collaboration spaces, client sites and remotely. You'll get the chance to meet face to face when needed, while you collaborate and learn from colleagues, share your experiences, and build the relationships that will fuel your career and prioritise your wellbeing. Please check with your recruiter for the specific working requirements that may apply for your role.

Our commitment to you

Making an impact is more than just what we do: it's why we're here. So we work hard to create an environment where you can experience a purpose you believe in, the freedom to be you, and the capacity to go further than ever before.

We want you. The true you. Your own strengths, perspective and personality. So we're nurturing a culture where everyone belongs, feels supported and heard, and is empowered to make a valuable, personal contribution. You can be sure we'll take your wellbeing seriously, too. Because it's only when you're comfortable and at your best that you can make the kind of impact you, and we, live for.

Your expertise is our capability, so we'll make sure it never stops growing. Whether it's from the complex work you do, or the people you collaborate with, you'll learn every day. Through world-class development, you'll gain invaluable technical and personal skills. Whatever your level, you'll learn how to lead.

Connect to your next step

A career at Deloitte is an opportunity to develop in any direction you choose. Join us and you'll experience a purpose you can believe in and an impact you can see. You'll be free to bring your true self to work every day. And you'll never stop growing, whatever your level.

Discover more reasons to connect with us, our people and purpose-driven culture at deloitte.co.uk/careers

#J-18808-Ljbffr