Back

data engineer Remote

JLCAC Ltd

Full Time Remote 80000 - 65000 GBP ANNUAL Today

Job description

Senior/Lead Data Engineer

Key responsibilities and accountability

Translates complex functional and technical requirements into detailed architecture, design, and high performing data platform
Works on multiple initiatives as a technical lead driving user story analysis, design and development of data processing flow.
Support leadership team to selects data solution software and defines hardware requirements
Builds and designs solution on top of AWS Data stack and other specialized technology such as Snowflake
Codes, tests, and documents new or modified data systems to create robust and scalable applications for data analytics
Implements security and recovery tools and techniques as required, works with developers to make sure that all data solutions are consistent
Creates data flow diagrams for all of business systems and builds automation tools.
Creates and maintains data catalog, interprets data results to business stakeholders and develops standards and processes for integration initiatives
Leads the design of the logical model and implements the physical database to support business needs.
Designs key and indexing schemes and designs partitioning, constructs and implements operational data stores and data marts.
Ensures database changes are reviewed and approved by Technology and Product teams.
Resolves conflicts between models, ensuring that data models are consistent with the ecosystem model (e.g., entity names, relationships and definitions) and conducts Level 2 and 3 support
Leads or participates in creating, refining, managing and enforcing data management policies, procedures, conventions and standards
Contributes to the establishment of business continuity & disaster recovery requirements, methods and procedures for data systems and databases
Performs technology and product research to identify opportunities that impact business strategy, business requirements and performance.
Evaluates and provides feedback on future technologies and new releases/upgrades.
Assesses and provides recommendations on business relevance, appropriate timing and deployment

Knowledge, skills, training, and experience

Essential

Bachelor's in computer science, computer engineering, or equivalent work experience
Typically requires over 7 years data engineering, data science, or software engineering experience
Requires 3 years of demonstrated experience leading teams of engineers
Capability to architect highly scalable distributed systems, using AWS
Demonstrated experience with Agile or other rapid application development methods
Demonstrated experience with object-oriented design, coding and testing patterns as well as experience in engineering (commercial or open source) software platforms and largescale data infrastructures
Extensive knowledge in SQL, Python and PySpark
Expert knowledge of data modeling and understanding of different data structures and their benefits and limitations under particular use cases
Experience using Big Data batch and streaming tools.

Desirable