Senior Data Engineer - Big Data Pipelines

About the Role

Our client is seeking an experienced Senior Data Engineer to design, build, and maintain robust big data pipelines in Sheffield . This fully remote position involves working with large datasets, developing ETL/ELT processes, and ensuring the reliability and efficiency of our data infrastructure. You will collaborate with data scientists, analysts, and other engineers to provide clean, accessible data for reporting and analytical purposes. This is an excellent opportunity to contribute to a data-driven culture and work on challenging big data problems with a distributed team.

Key Responsibilities

• Design, build, and optimize scalable data pipelines using technologies like Spark , Hadoop , and cloud data services.
• Develop and maintain ETL/ELT processes to ingest, transform, and load data from various sources.
• Ensure data quality, integrity, and security across all data platforms.
• Collaborate with data scientists and analysts to understand data needs and provide required datasets.
• Monitor and troubleshoot data pipeline performance, identifying and resolving bottlenecks.
• Implement data warehousing solutions and optimize database performance.

Requirements

• Bachelor's or Master's degree in Computer Science, Engineering, or a related quantitative field.
• 5+ years of experience in data engineering, with a focus on big data technologies.
• Proficiency in programming languages such as Python , Scala , or Java.
• Strong experience with distributed data processing frameworks (e.g., Apache Spark).
• Experience with cloud platforms (AWS, Azure, GCP) and their data services (e.g., S3, Data Lake Storage, BigQuery).

Benefits

• Competitive salary and performance-based bonuses.
• Comprehensive health, dental, and vision insurance.
• Generous pension contribution and life insurance.
• Fully remote working environment, offering excellent flexibility.
• Opportunities for continuous learning and professional development in data engineering.