data engineer

data engineer Remote

AppsIntegration.Inc
Full Time Remote 120840 - 74181 USD ANNUAL Today
Job description

Must have ADF, Azure SQL, Spark

1) Support and refine Constellation’s data and analytics technology stack with an emphasis on improving reliability, scale, and availability. ADF, Spark

2) Assist in the design and management of enterprise grade data pipelines and data stores that will used for developing sophisticated analytics programs, machine learning models, and statistical methods.

3) Experience delivering data solutions via Agile methodologies and designing CI/CD workflows.

PRIMARY DUTIES AND ACCOUNTABILITIES

Item Accountability %

1 Create and maintain optimal data pipeline architecture 20

2 Assemble large, complex data sets that meet functional / non-functional business requirements. 20

3 Identify, design, and implement internal process improvements: automating manual processes, optimizing data delivery, re-designing infrastructure for greater scalability, etc. 20

4 Build the infrastructure required for optimal extraction, transformation, and loading of data from a wide variety of data sources using SQL and Big Data technologies. 20

5 Deliver automation & lean processes to ensure high quality throughput & performance of the entire data & analytics platform.? 10

6 Work with data and analytics experts to strive for greater functionality in our analytics platforms. 10

POSITION SPECIFICATIONS

Minimum: Preferred:

  • Experience in building/operating/maintaining fault tolerant and scalable data processing integrations using Azure
  • Strong problem-solving skills with emphasis on optimization data pipelines
  • Excellent written and verbal communication skills for coordinating across teams
  • A drive to learn and master new technologies and techniques
  • Experienced in DevOps and Agile • Experience using Docker or Kubernetes is a plus
  • Demonstrated capabilities with cloud infrastructures and multi-cloud environments such as Azure, AWS, IBM cloud
  • environments and using CI/CD pipelines.
  • Experienced using Databricks & Apache Spark
  • Experienced using Azure Data Factory or Synapse Analytics

Job Type: Contract

Pay: $74,181.34 - $120,840.44 per year

Experience:

  • Informatica: 1 year (Preferred)
  • SQL: 1 year (Preferred)
  • Data warehouse: 1 year (Preferred)

Work Location: Remote

Speak with the employer
+91 9255967099

data engineer
AppsIntegration.Inc

Related Jobs

All Related Listed jobs

Customer Service Assistant Level 1
Wilko Tamworth, England 10.5 GBP HOURLY Today

Thanks to your warm friendly manner, helpful approach and great product knowledge, customers will keep coming back to us every time they need to tick those to

Retail Associate - TK Maxx - Christmas Temp - Leicester - 20 hours
TK Maxx Leicester, England 19832 - 21420 GBP ANNUAL Today

Assisting in coordinating and executing the smooth and timely movement of merchandise from the backroom to the sales floor, while maintaining floor operational

production worker
NHS Maidenhead, England 22838 - GBP HOURLY Today

We have a great opportunity for a General Assistant to join our team based in St Marks Hospital.

The salary on offer is 22,581.25 per annum....

Grocery Shopping Associate
McGrocer Romford, England 10.9 - 11 GBP HOURLY Today

We enable customers to buy their favourite groceries from one or multiple stores in a one-stop shop, including specialist products they can't find in major

Senior Care Assistant part time posts
Rosewood Lodge Residential Home Weston-super-Mare, England 10.5 - 11 GBP HOURLY Today

Applicants must have NVQ level 2/3 in health and social care. Rosewood Lodge Residential Care Home in Weston super Mare have two vacancies for two part Time