Big Data Engineer

dunnhumby is the global leader in Customer Data Science, partnering with the world’s most ambitious retailers and brands to put the customer at the heart of every decision. We combine deep insight, advanced technology, and close collaboration to help our clients grow, innovate, and deliver measurable value for their customers.

dunnhumby employs nearly 2,500 experts in offices throughout Europe, Asia, Africa, and the Americas working for transformative, iconic brands such as Tesco, Coca-Cola, Nestlé, Unilever and Metro.

We’re looking for a Big Data Engineer who expects more from their career. It’s a chance to extend and improve dunnhumby’s Data Engineering Team. It’s an opportunity to work with a market-leading business to explore new opportunities for us and influence global retailers.

What we expect from you

• 2–5 years of experience in Data Engineering or a related field
• Strong experience with Python, PySpark, and SQL (mandatory)
• Experience working in Linux environments and writing shell scripts
• Hands-on experience with big data technologies such as Spark, Hive, Hadoop, Airflow, Oozie, HBase, or MapReduce
• Experience with data pipeline development and data flow management
• Familiarity with Git and process automation
• Experience with relational database management systems (RDBMS) and Data Flow Development

Key Responsibilities

• Build data solutions, including data lakes, data warehouses, ETL/ELT pipelines, APIs, and analytics platforms.
• Build scalable and low-latency data pipelines using tools like Apache Kafka, Flink, or Spark Streaming to handle high-velocity data streams.
• Automate data pipelines and processes end-to-end using orchestration frameworks such as Apache Airflow to manage complex workflows and dependencies.
• Develop pipeline for real-time data processing.
• Implement data governance, metadata management, and data quality standards.
• Explore appropriate tools, platforms, and technologies aligned with organizational standards.
• Ensure security, compliance, and regulatory requirements are addressed in all data solutions.

What you can expect from us

We won’t just meet your expectations. We’ll defy them. So you’ll enjoy the comprehensive rewards package you’d expect from a leading technology company. But also, a degree of personal flexibility you might not expect. Plus, thoughtful perks, like flexible working hours and your birthday off.

You’ll also benefit from an investment in cutting-edge technology that reflects our global ambition. But with a nimble, small-business feel that gives you the freedom to play, experiment and learn.

And we don’t just talk about diversity and inclusion. We live it every day – with thriving networks including dh Gender Equality Network, dh Proud, dh Family, dh One, dh Enabled and dh Thrive as the living proof. We want everyone to have the opportunity to shine and perform at your best throughout our recruitment process. Please let us know how we can make this process work best for you.

Our approach to Flexible Working

At dunnhumby, we value and respect difference and are committed to building an inclusive culture by creating an environment where you can balance a successful career with your commitments and interests outside of work.

We believe that you will do your best at work if you have a work / life balance. Some roles lend themselves to flexible options more than others, so if this is important to you please raise this with your recruiter, as we are open to discussing agile working opportunities during the hiring process.

For further information about how we collect and use your personal information please see our Privacy Notice which can be found (here)