Senior Big Data Lead Engineer

A major international banking institution has an urgent requirement for an Temporary Senior Big Data Lead Engineer.

This is an initial contract to the end of the year offering up to £700 per day (Inside IR35, Umbrella) offering hybrid working between WFH and London office.

Role

The role would be entrusted to developed solutions/design ideas, work with architects/BA and other engineers to build strategic data ingestion and distribution architecture to support near real time and Tier 1 service requirements. The person would need to be able to stand on his/her own feet, review existing infrastructure and tooling, provide feedback on improvements and design/build strategic near real time (Kafka streaming, CDC), cloud solutions and set up automated end to end data ingestion pipeline and automated test report.

You will

• Design and implement strategic near real time infrastructure
• Tier 1 Service / DR (Disaster Recovery)
• Review existing infrastructure and recommend changes - infrastructure and tooling enhancements
• GCP service set up
• Automation
• Data Distribution

Experience

• Experience with most of the following technologies (Apache Hadoop, Scala, Apache Spark, Spark streaming, YARN, Kafka, Hive, HBase, Presto, Python, ETL frameworks, MapReduce, SQL, RESTful services).
• Sound knowledge on working Unix/Linux Platform
• Experience in end to end automation
• Hands-on experience building data pipelines using Hadoop components Sqoop, Hive, Pig, Spark, Spark SQL.
• Experience with time-series/analytics db's such as Elasticsearch
• Experience with industry standard version control tools (Git, GitHub), automated deployment tools and requirement management in JIRA
• Exposure to Agile Project methodology but also with exposure to other methodologies (such as Kanban)
• Understanding of big data modelling techniques using relational and non-relational techniques
• Understanding or experience of Cloud design patterns

This is an urgent role - if interested, please apply ASAP and any questions, please email swajal.sinha@1st-executive.com