Senior Data Engineer - Big Data, AWS, NoSQL
As part of a group-wide IT transformation programme, an industry leader in the financial services sector has a requirement for a Senior Data Engineer to join their team and help lead this programme.
They are looking for someone who can work and make decisions in a cross-functional delivery team including analysts, architects, big data engineers, machine learning engineers, and testers. The individual will seek to identify improvements and efficiencies, while utilising new technologies and existing tools.
This represents an opportunity to join and make a real impact at a point where the project is still essentially at green field stage.
The core responsibilities and objectives of the role include, but are not limited to:
- Resolving complex problems and implementing high performance solutions that will run on top of their cloud based big data platform.
- Evolving common standards and best practices as the programme progresses.
- Engineering, optimising and debugging high performance data pipelines to contribute to the creation of data platforms.
- Participating in a programme that will deliver a scalable solution for how data is collected, stored, accessed and managed in the firm.
- Collaborate with multiple business and IT functions to extract, transform and load both structured and unstructured data.
- Working with multiple stakeholders, and in particular Data Architects, Scientists and Analysts.
Suitable candidates will need to have the following skills and experience:
- Experience with object oriented and functional design, coding, and testing patterns as well as experience in engineering software platforms and large-scale data infrastructures.
- Considerable experience in a data engineering role, preferably within large and complex corporate environments.
- An expert with SQL and NoSQL.
- Programming skills with Python and ideally a second or third language such as scala.
- An expert in the strengths and limitations of data structures, Big Data knowledge is a must have.
- Experience with AWS cloud storage, Business Intelligence tools and DevOps principles.
- Exposure to different testing, development and project management methodologies.