Role Overview: The Data Engineer will play a crucial role in developing and maintaining scalable data pipelines, handling large and complex datasets, and collaborating with teams to enhance data models for business intelligence tools and machine learning. With strong expertise in SQL, AWS services, BI tools, and programming languages like Python, they will optimize data architectures and ensure data accessibility, while demonstrating exceptional analytical and problem-solving skills. This role requires a proactive approach to building efficient data infrastructure to support the increasing volume and complexity of data. Key Responsibilities: Develop and maintain scalable data pipelines and build new API integrations to support the increase in data volume and complexity. Handle large, complex datasets that meet functional/non-functional business requirements. Build the infrastructure required for optimal extraction, transformation, and loading of data from a wide variety of data sources. Collaborate with data scientists and business teams to improve data models that feed business intelligence tools, increasing data accessibility. Work closely with teams to develop data models and pipelines for research, reporting, and machine learning. Demonstrate excellent analytical and problem-solving skills. Write Unit and Integration tests and maintain good documentation practices. Qualifications for Data Engineer: Strong expertise in SQL and experience working with relational databases, query authoring (SQL), etc. 3-4 years of experience with data engineering. Proficiency in working with AWS Data Services like Glue, Athena, RedShift, RDS ETL, Mapping snow flex, or similar services. Experience in building and optimizing 'big data' data pipelines, architectures, and datasets. Familiarity with BI tools like Quicksight, PowerBI, Grafana, etc. Proficiency in Python and experience with data visualization tools. Strong analytical skills related to working with unstructured datasets.
