data engineer San Antonio, TX
Job description
Job Summary:
The Data Engineer will primarily perform data engineering tasks, delivering efficient and reliable data pipelines and infrastructure to support the organization’s data needs. This role requires expertise in designing, building, and maintaining scalable data solutions, as well as knowledge of database administration principles. The ideal candidate will possess strong problem-solving skills and be able to collaborate effectively with cross-functional teams. With a focus on data integrity and performance optimization, the Data Engineer will play a crucial role in ensuring the organization’s data-driven initiatives are successful. They will also actively contribute to data governance initiatives, ensuring compliance with data standards, security measures, and privacy regulations. Additionally, they will stay updated with emerging technologies to drive continuous improvement in data engineering practices.
Major Responsibilities:
- Design, build, and maintain scalable and efficient data pipelines, ensuring the timely and accurate extraction, transformation, and loading (ETL) of data from various sources.
- Collaborate with data scientists and analysts to understand their data requirements and implement data models and structures that support efficient analysis and reporting.
- Perform data quality checks and implement data validation processes to ensure data accuracy and integrity.
- Support and optimize database performance by monitoring and tuning database queries, indexes, and overall system performance.
- Troubleshoot and resolve database and data pipeline issues, ensuring minimal downtime and disruptions to data availability.
- Implement and maintain database security measures, including user access controls and data encryption.
- Collaborate with cross-functional teams to understand business requirements and contribute to the design and implementation of data solutions.
- Analyze and validate data sharing requirements within the organization and with our outside partners.
- Analyze and interpret collected data; identifying trends; writing reports and recommendations for internal or external clients.
- Use statistical practices to analyze current and historical data to make predictions, identify risks, and opportunities enabling better decisions on planned/future events.
- Work with business leadership to understand data requirements; propose and develop solutions that enable effective decision-making and drive business objectives.
- Recognize potential issues and risks during the analytics project implementation and recommend mitigation strategies.
- Document best practices for data analysis, data engineering, and evangelize their usages.
- Coach and mentor project team members in carrying out analytics project implementation activities.
- Stay up to date with emerging trends and technologies in data engineering and database administration, recommending and implementing improvements as needed.
Job Requirements and Qualifications:
- Bachelor's degree in computer science, engineering, or a related field.
- Minimum of 2 or more years experience.
- Strong proficiency in SQL script development, with experience in T-SQL (Microsoft SQL Server) being a plus.
- Knowledge of data engineering principles, including data modeling, ETL processes, and data integration techniques.
- Familiarity with database administration tasks, such as performance tuning, security management, and backup and recovery procedures.
- Experience building and optimizing data pipelines, architectures, and data sets.
- Experience with cloud platforms and technologies for data engineering, such as AWS, Azure, or Google Cloud. Azure knowledge is a plus.
- Proficiency in at least one programming language commonly used in data engineering, such as Python, Java, or Scala.
- Understanding of data governance principles and practices.
- Excellent problem-solving and analytical skills.
- Strong communication and collaboration abilities to work effectively with cross-functional teams.
- Attention to detail and a commitment to delivering high-quality data solutions.
- Minimum of 3-5 years of experience.
