Job Title:- Data Engineer
Duration:- 8+ months
Location:- Onsite based out of Boston, MA (Only Locals)
The client is looking for a Data Warehouse Engineer(Contractor) to join the Enterprise Data Lake Team. The ideal candidate will be a well-rounded technologist with a passion for building scalable and maintainable data pipelines using SQL, Python, and some of the AWS technologies.
What you'll do:
• Design analytic solutions for business users that provide the data needed to operate the business
• Design, develop, implement, test, document, and operate large-scale, high-volume, high-performance data structures for the analytics platform
• Write scalable and maintainable ETL/ELT processes to run in batch using SQL, Airflow, and Python.
• Review code, and implementations and give meaningful feedback that helps others build better solutions
• Write documentation on design, architecture, and solutions
What you'll need:
• Experience in data warehouse design and data integration methodologies
• SQL expertise and the ability to work with many different MPPs. Vertica, Snowflake, Postgres, etc. Should be able to write highly optimized SQL.
• Hands-on Experience in writing Airflow DAGs.
• 5+ years of experience with detailed knowledge of data warehouse technical architectures, infrastructure components, ETL/ ELT, and reporting/analytic tools
• 5+ years of hands-on experience designing and developing scalable, high performing, and fault-tolerant applications for large enterprises
• 5+ years of experience writing software in native Python
• Expert level knowledge of data integration and familiarity with common data integration challenges like converting data types, handling errors, and translating between different technology stacks
• Able to work in a fast-paced environment and be comfortable being accountable for work products
• Experience performing analytics to solve complex business problems
Bonus:
• Experience with cloud-based data warehouse platforms (e.g. Snowflake, etc.)
• Ability to work with AWS technologies - S3 buckets, Kinesis, Lambda, API Gateways, ECS, EKS, MWAA, etc.
• Background in e-commerce
• Agile experience is a plus
5+ years of experience is required
Top 3 must-haves:
1. Python
2. Snowflake
3. SQL