data engineer New York, NY
Job description
Hello! We’re building the future of content discovery infrastructure with AI — come join us!
Shaped (YC W22) is an API for developers to seamlessly add personalized ranking and recommendation to their products. These frictionless discovery experiences help end-users find what they want faster, and, consequently, grow conversion and engagement business metrics. Behind the scenes, we’re building state-of-the-art ML infrastructure for training and deploying large AI models to power our personalization engine.
We’re a seed stage start-up backed by top-tier investors (e.g. Y-Combinator, Susa Ventures, Tribe Capital etc…) and executives (e.g. Google, Amazon, Uber, Dropbox etc…). Our team comes from Meta, Google, Apple and Uber. We’re a remote team but have a small office in Brooklyn, New York.
We’ve been lucky to have a huge amount of success since our initial launch earlier this year and now looking to expand our real-time data engineering capabilities. We are looking for a data engineer to design, build and optimize Shaped's real-time streaming data ingestion endpoints. You will be a founding engineer working on state-of-the-art infrastructure. As one of Shaped’s early employees you will help shape our product, culture and vision.
This role is for a full-time data engineer at $120k - $200k a year and 0.1 - 0.5% equity.
Responsibilities:
- Manage the unification of our batch and real-time data processing pipelines.
- Work on our real-time and batch data ingestion pipelines using modern data engineering pipelines/streams.
- Work closely with a small team to inform the roadmap and co-develop a strong engineering culture with best practices.
- Work on our real-time and batch data ingestion pipelines using modern data engineering pipelines/streams.
- Work closely with a small team to inform the roadmap and co-develop a strong engineering culture with best practices.
Requirements:
- Bachelor's in computer science, data science or mathematics related field. Master's degree or PhD will be advantageous.
- 5+ years of data engineering experience, ideally applied to machine learning use-cases.
- Experience with AWS or common cloud technologies
- Proficient with Python, SQL.
- Extensive knowledge of both offline and online large scale data processing frameworks: Spark, Beam, Flink.
- Extensive knowledge with stream technologies like Kakfa or Kinesis.
- Excellent written and verbal communication skills.
We’re excited to work with you. Come build the future of AI and discovery with us!
