Data Pipeline Engineer Sunbury, England
Job description
Job Title: Data Pipeline Engineer
Job Location: Remote
Industry: Oil and Energy, Tech
Contract Length: 6 Months (Subject to extension)
Summary
Critical to achieving BP’s digital ambitions is the delivery of our high value data and analytics initiatives, and the enablement of the technologies and platforms that will support those objectives.
Join us in creating, growing, and delivering innovation at pace, enabling us to thrive while transitioning to a net zero world. All without compromising our operational risk management.
Key Responsibilities
As a member of our team, you will design, develop, test and maintain highly performant, scalable, robust and available data pipelines for our business teams.
Work closely with the data science and engineering team and the data platform team to implement automated data pipeline workflows in the Azure/AWS cloud platforms to enable data ingestion, processing, and distribution.
Deploy scalable solutions typical to critical bp business environments.
Build the automation and monitoring frameworks that capture metrics and operational KPIs for data pipeline quality and performance.
Build prototypes and workflows to try out new ideas in an iterative manner.
Skills/Experience
3+ years of hands-on software development, data engineering, and systems architecture
2+ years of experience in designing, developing and maintaining high performance, low latency and large-scale data pipelines.
2+ years of experience in SQL optimization and performance tuning, and development experience in programming languages like Python, PySpark, Scala etc.)
2+ years in cloud data engineering experience in Azure or/and AWS
Experience building/operating highly available, distributed systems of data extraction, ingestion, and processing of large data sets
Experience with data modelling, data warehousing, and building high-volume ETL/ELT pipelines
Experience with modern data technologies such as SQL and non-SQL databases, REST / GraphQL APIs, data event streaming technologies, and in-memory data storage.
Experience with version control systems like Github and deployment & CI/CD tools
Excellent communication skills, both verbal and written
If the offer fits you and you have the relevant experience, don't hesitate to apply!
