Role: Data Engineer
Location: Hybrid in Manhattan, NY or Winter Park, FL
Exp - 8+ years minimum
MoI: F2F (2-hour interview includes an assessment too) Position Overview As a Data Engineer at Client, you will develop a deep understanding of the company's diverse data landscape and collaborate closely with cross-functional teams to gather requirements and deliver data-driven solutions. You will be responsible for providing data services through reports, systems, and interface applications to support various business functions.
The ideal candidate has a strong background in data engineering, is proficient in mainstream database systems, and possesses solid development skills, including the ability to perform in-depth debugging and troubleshooting. A passion for solving complex data challenges and enabling data-driven decision-making is essential.
Key Responsibilities - Data Analysis and Reporting : Work with business analysts and stakeholders to understand their data needs and deliver data solutions. Requires a strong understanding of data processing and hands-on experience with tools such as SQL, Pandas, and Jupiter.
- Data Service Platform Development : Identify recurring analytical and reporting needs and implement reusable solutions within the internal data service platform. Proficiency in Python, Django, and JavaScript is required, along with strong debugging skills.
- Data Processing Pipelines : Design, build, and maintain reliable and scalable daily data pipelines. Experience with Apache Airflow or other ETL orchestration tools is essential.
- External Data Interfaces : Develop and maintain data services for external systems based on internal datasets. Requires knowledge of RESTful APIs and message queue technologies such as IBM MQ or Apache Kafka.
- Data Platform Infrastructure : Work with cloud-based infrastructure components, including EC2, Docker, RDS, Redshift, and S3 to support the data platform.
Qualifications Must: - Bachelor's degree in Data Engineering, Computer Science, Information Technology, or a related field
- Minimum of 3 years' experience in a data engineering role
- Proficiency in Python and Pandas
- Strong SQL skills
Preferred: - Experience with JavaScript and web debugging tools
- Familiarity with Apache Airflow
- Experience with Django
- Experience with AWS and related cloud technologies
- Fluent in Mandarin