Job Summary:
Alpha Consulting Corp. is seeking an experienced Lead Data Engineer with expertise in Snowflake, DBT, and IBM DataStage. The role involves designing and developing scalable ETL/ELT pipelines and optimizing enterprise data platforms to support analytics and business intelligence.
Responsibilities:
• Design, develop, and maintain scalable ETL/ELT pipelines for structured and semi-structured data.
• Build modern ELT pipelines using DBT and Snowflake.
• Develop and maintain enterprise ETL workflows using IBM DataStage.
• Design scalable dimensional data models including Star and Snowflake Schemas.
• Develop optimized SQL transformations for large-scale datasets.
• Implement efficient data loading strategies leveraging Snowflake architecture.
• Process structured, semi-structured, and JSON/Parquet data within Snowflake.
• Develop reusable and modular DBT models, macros, and tests.
• Optimize Snowflake performance through clustering, partitioning, and warehouse tuning.
• Improve ETL/ELT pipeline performance and scalability.
• Ensure high availability, reliability, and maintainability of data pipelines.
• Implement best practices for metadata management and documentation.
• Monitor data quality, reconciliation, and validation processes.
• Work with cloud platforms including AWS, Azure, or GCP.
• Schedule and monitor workflows using Control-M, Airflow, or equivalent orchestration tools.
• Develop Python scripts for automation, data processing, and utilities.
• Utilize UNIX/Linux shell scripting for operational support and automation.
• Lead technical design discussions and mentor junior data engineers.
• Collaborate with Data Architects, Business Analysts, Data Scientists, and business stakeholders.
• Translate business requirements into scalable technical solutions.
• Participate in code reviews and establish engineering best practices.
• Drive continuous improvement initiatives across the data platform.
Qualifications:
Required:
• 10+ years of experience in designing, developing, and optimizing enterprise data platforms using Snowflake, DBT, IBM DataStage, SQL, and Python
• Expertise in both traditional ETL (IBM DataStage) and modern ELT (DBT + Snowflake)
• Strong experience in data modeling, performance optimization, orchestration, and cloud-based data engineering
• Design, develop, and maintain scalable ETL/ELT pipelines for structured and semi-structured data
• Build modern ELT pipelines using DBT and Snowflake
• Develop and maintain enterprise ETL workflows using IBM DataStage
• Design scalable dimensional data models including Star and Snowflake Schemas
• Develop optimized SQL transformations for large-scale datasets
• Implement efficient data loading strategies leveraging Snowflake architecture
• Process structured, semi-structured, and JSON/Parquet data within Snowflake
• Develop reusable and modular DBT models, macros, and tests
• Optimize Snowflake performance through clustering, partitioning, and warehouse tuning
• Improve ETL/ELT pipeline performance and scalability
• Ensure high availability, reliability, and maintainability of data pipelines
• Implement best practices for metadata management and documentation
• Monitor data quality, reconciliation, and validation processes
• Work with cloud platforms including AWS, Azure, or GCP
• Schedule and monitor workflows using Control-M, Airflow, or equivalent orchestration tools
• Develop Python scripts for automation, data processing, and utilities
• Utilize UNIX/Linux shell scripting for operational support and automation
• Lead technical design discussions and mentor junior data engineers
• Collaborate with Data Architects, Business Analysts, Data Scientists, and business stakeholders
• Translate business requirements into scalable technical solutions
• Participate in code reviews and establish engineering best practices
• Drive continuous improvement initiatives across the data platform
Company:
Alpha Consulting Corp. has been exceeding expectations in the IT, pharmaceutical, and clinical staffing business since 1994. Founded in 1994, the company is headquartered in East Brunswick, USA, with a team of 201-500 employees. The company is currently Growth Stage.