£650 - £700 per day
7 months ago
Senior Big Data Lead Engineer
A major international banking institution has an urgent requirement for an Temporary Senior Big Data Lead Engineer.
This is an initial contract to the end of the year offering up to £700 per day (Inside IR35, Umbrella) offering hybrid working between WFH and London office.
The role would be entrusted to developed solutions/design ideas, work with architects/BA and other engineers to build strategic data ingestion and distribution architecture to support near real time and Tier 1 service requirements. The person would need to be able to stand on his/her own feet, review existing infrastructure and tooling, provide feedback on improvements and design/build strategic near real time (Kafka streaming, CDC), cloud solutions and set up automated end to end data ingestion pipeline and automated test report.
- Design and implement strategic near real time infrastructure
- Tier 1 Service / DR (Disaster Recovery)
- Review existing infrastructure and recommend changes - infrastructure and tooling enhancements
- GCP service set up
- Data Distribution
- Experience with most of the following technologies (Apache Hadoop, Scala, Apache Spark, Spark streaming, YARN, Kafka, Hive, HBase, Presto, Python, ETL frameworks, MapReduce, SQL, RESTful services).
- Sound knowledge on working Unix/Linux Platform
- Experience in end to end automation
- Hands-on experience building data pipelines using Hadoop components Sqoop, Hive, Pig, Spark, Spark SQL.
- Experience with time-series/analytics db's such as Elasticsearch
- Experience with industry standard version control tools (Git, GitHub), automated deployment tools and requirement management in JIRA
- Exposure to Agile Project methodology but also with exposure to other methodologies (such as Kanban)
- Understanding of big data modelling techniques using relational and non-relational techniques
- Understanding or experience of Cloud design patterns
This is an urgent role - if interested, please apply ASAP and any questions, please email firstname.lastname@example.org