Job Summary:
Alpha Consulting Corp. is seeking an experienced and highly skilled Contract Data Engineer focused on building robust data pipelines. The role involves integrating data from various APIs and managing critical data infrastructure, ensuring data is clean and optimized for analytics tools like Tableau.
Responsibilities:
• Design, develop, and maintain scalable data ingestion pipelines to extract data from various third-party and internal APIs (REST, SOAP, etc.).
• Implement efficient data transformation and loading processes (ETL/ELT) within the data platform.
• Manage and optimize data storage and schemas in Snowflake and Postgres databases.
• Utilize Databricks for data processing, transformation, and orchestration tasks.
• Ensure data quality, accuracy, and integrity throughout the data pipelines.
• Collaborate with data analysts and BI developers (particularly Tableau users) to understand data requirements and optimize data models for performance.
• Monitor data pipelines and systems for performance issues, errors, and data discrepancies, implementing necessary fixes and improvements.
• Develop and maintain technical documentation for data pipelines, processes, and data models.
• Troubleshoot data-related issues and provide timely resolutions.
• Implement best practices for data security and governance within the data platform.
Qualifications:
Required:
• Proven experience as a Data Engineer, with a strong focus on API integration and database management.
• Experience building data pipelines for API data ingestion, including handling authentication, error handling, and data parsing.
• Strong proficiency in SQL and experience with database design, query optimization, and performance tuning in relational databases (e.g., Postgres).
• Hands-on experience with Snowflake as a cloud data warehouse, including data loading, querying, and performance optimization.
• Experience using Databricks for data processing, ETL/ELT, and pipeline orchestration (e.g., PySpark, notebooks).
• Proficiency in a programming language commonly used for data engineering (e.g., Python).
• Experience working with cloud platforms (e.g., AWS, Azure, GCP) where Snowflake and Databricks are deployed.
• Understanding of data warehousing concepts and best practices.
• Experience supporting BI tools (like Tableau) by providing clean, structured, and performant data sources.
• Excellent problem-solving skills and ability to work independently in a fast-paced environment.
• Strong communication and collaboration skills.
• Prior work experience at client or in client's Industry.
Preferred:
• Experience with specific ETL/orchestration tools (e.g., Airflow, Fivetran, dbt).
• Familiarity with containerization technologies (e.g., Docker).
• Experience in a contract or consulting role.
• Knowledge of data governance and compliance standards.
• Experience with BI tools like Tableau, including dashboard creation or assisting BI developers with visualization best practices related to data structure.
Company:
Alpha Consulting Corp. has been exceeding expectations in the IT, pharmaceutical, and clinical staffing business since 1994. Founded in 1994, the company is headquartered in East Brunswick, USA, with a team of 201-500 employees. The company is currently Growth Stage.