Lead Data Engineer

Lead Data Engineer 

Marcus Donald are currently looking for a Lead Data Engineer to join a large Oil and Energy company in the heart of Canary Wharf. 

You will be responsible for leading the technical development work directly reporting the Head of Data. The Lead Data Engineer has the autonomy to shape and direct data technology development for the Data Services, and therefore influence the wide data architecture across the organisation. 

Key Skills required for Lead Data Engineer: 

• Python and SQL 
• Hands on experience (including QA) in platforms such as Azure and the Hadoop ecosystems (HIVE, HDFS and Apache spark) 
• Good knowledge on BI Tools 
• Good knowledge on Linux 
• Data Modelling 

Desirable Skills: 

• Scala 
• HDInsight 

Key responsibilities for Lead Data Engineer: 

• Lead a team of engineers, supported by QA and analyst staff across London and Glasgow 
• End-to-end data architecture and handling approach in the Data Services Platform - Defining and communicating technical needs for data collection, ingestion, transformation and calculation 
• Carry out data modelling and work closely with data scientist 
• Design pipelines, create proof of concepts to production, run functional tests, end to end regression testing, quality/automation tests 
• Ensure principles of privacy and security are assessed and designed into their data solutions 

Key outputs and deliverable

• The data software engineering meets the requirements of each data product design 
• Data services data engineering work follows agile methodologies 
• The Data Service Platform functionality is continually reviews and improved 
• The data architecture of the data platform meets the users' needs and has a design that is flexible to meet future needs 

This is a fantastic opportunity for a Lead Data Engineer to be heavily involved in a 2 year old Greenfield Project! 

If you feel you are suitable for this role then you can contact Ellie or alternatively, you can apply through this job advert. 

Lead Data Engineer