Data DevOps Engineer

Data DevOps Engineer

Marcus Donald are now looking for a Data DevOps Engineer to join a client based in London.

In this role you will be responsible for the configuration and on-going management of cloud PaaS/IaaS, working with Big Data platforms such as Hortonworks based on Hadoop clusters configured on cloud data centres such as those of Azure or AWS.

You will ensure availability of the platform to services teams and their service users. You will also carry out continuous improvements to platforms, including system development, processes design and automation and diagnostics improvements. Assurance of platform security, privacy and resilience will be a main focus.

Key Skills as a Data DevOps Engineer

• Agile software engineering practices (CI/CT/CD)
• Azure
• Microservices Architecture and APIs
• Hadoop ecosystem using Hortonworks or Cloudera
• Cloudbreak, Hive, Ambari, Sqoop, Oozie, SPARK, Atlas, Ranger, HBase, HDFS, YARN and ELK
• Experience connecting cloud infrastructure to on-prem networks
• VPNs, encryption & key/certificate management, endpoint protection, virtual firewalls/ACLs/NSGs, setting up bastion nodes, etc
• VPNs, tunnelling, AD connectors, routing
• Authentication technologies (LDAP, OAuth, 2FA, SAML, and SSO via Kerberos)
• Distributed system design and dependency management
• Troubleshooting
• Bash, Python, Java or Ruby
• Monitoring tools, approach and implementation
• Linux system administration

Key Responsibilities as a Data DevOps Engineer

• Ensure principles of privacy, security and resilience are assessed and designed into solutions
• Implementation and on-going administration of Hadoop infrastructure
• Propose and deploy new hardware and software environments required for Hadoop and to expand existing environments
• Building and supporting technical stack as well as evaluating technology for use in the stack
• Setup new Hadoop users, setting up Linux users, Kerberos principles and testing HDFS and Hive access for new users
• Working with the infrastructure, network, database, application and business intelligence teams to guarantee high data quality and availability
• Monitor and maintain Hadoop ecosystem connectivity and security to guarantee confidentiality, integrity and availability
• Integrating technology to provide end to end architecture

Please note that unfortunately, my client do not offer sponsorship.

If you feel you would be suitable for this Data DevOps Engineer position, please apply via this advert or contact Joe directly on 0203 328 0400.

Data DevOps Engineer