Big Data Lead

Last updated 13 days ago
Location:Greater London
Job Type:Full Time

As a Lead Data Engineer, you will be working on a global initiative to take our data platforms and practice to the next level.

You will be leading the data and analytic solutions for our trading businesses. The existing platform is one year old, and this is an opportunity to contribute to the evolution of the platform and the team.

You will bring to the team senior experience of working in big data environments. You will apply your knowledge around continuous improvement and integrating new services and third-party products.

Knowledge of the following would be useful:

  • Experience with open source big data technologies. (Spark, Presto, Airflow, Cloud Storage)
  • Public cloud infrastructure (networking, instances, security) and services (e.g. lambda, RDS etc.).
  • Containerization (ideally with Kubernetes).
  • Data prep, transformation, and pipe lining (ideally spark).
  • Logging and Monitoring solutions
  • Coding languages: Python/Java/Scala/Bash

If you would like to apply your skills is a fast-paced environment, building next generation data platforms for our market leading trading business, please apply now

The Corporate Operations Group (COG) brings together specialist support services including workplace, human resources, market operations and technology. COG's purpose is to drive operational excellence through business-aligned services with a focus on quality, cost and risk. COG comprises the following divisions: Business Improvement and Strategy, Business Services, Human Resources, Market Operations, and Technology.

Find out more about Macquarie at

Macquarie understands the importance of diversity and inclusion - our long history of success has come from being different. At Macquarie we value the innovation and creativity that diversity of thought brings. The one thing we all have in common is our focus on high performance. If you're capable, motivated and can deliver, we want you on our team.

Macquarie is an equal opportunities employer and does not discriminate on the grounds of age, disability, sex, sexual orientation, gender reassignment, gender identity, marriage, civil partnership, pregnancy, maternity, race (including colour and ethnic or national origins), religion or belief.