Job Summary:
New York Power Authority is leading the charge toward a carbon-free, resilient, and economically vibrant New York. They are seeking an Associate Data Engineer to join their growing Data Engineering organization, where the role will focus on building scalable data pipelines and supporting enterprise analytics and AI initiatives.
Responsibilities:
โข Build, enhance, and debug scalable data pipelines and integrations
โข Translate business and technical requirements into data engineering solutions
โข Collaborate with lead data engineers and cross functional partners, including AI Fusion teams
โข Support development of data products aligned with NYPAโs analytics and AI strategies
โข Work with cloud and distributed data processing tools to deliver reliable, high quality data
โข Develop data solutions that are flexible, extensible, elastic, secure and reliable at large scale.
โข Work with Lead Data Engineer to provide guidance and direction to project teams ensuring compliance with coding standards and best practices.
โข Collaborate with Data Governance team to capture and manage meta data, and implement data quality rules.
โข Building and managing data pipelines, data products, integrations and promoting production.
โข Develop Application Integrations, APIs and Microservices using hybrid cloud architecture.
โข Continuously learn and be at the leading edge of Data/Application Integration, Cloud, Containerization, and other industry trends.
โข Work with stakeholders including product, data and business teams to assist with data-related technical issues and support their data infrastructure needs.
โข Follow Cyber security guidelines and polices to monitor the company's data security and privacy.
โข Build and maintain batch data pipelines for structured and semi-structured data
โข Support ingestion and preprocessing of unstructured data.
โข Implement basic data quality validations (schema checks, null checks).
โข Assist in preparing AI-ready datasets for analytics, AI/ML and GenAI use-cases.
โข Support implementation of data contracts through schema validation and data checks.
Qualifications:
Required:
โข Bachelor of Science Degree in MIS or Computer Science/Engineering (or similar) is required.
โข Minimum of 2 years of Data Engineering experience.
โข Hands-on experience with at least one data integration, data pipelines or application integration platform.
โข Practical scripting or programming experience across languages applicable to data engineering workloads.
โข Practical experience in traditional and cloud data management components (MS SQL, RDS, Athena, or similar).
โข Practical experience in metadata driven ingestion framework, building data pipelines and data sets.
โข Working-level familiarity with DevOps and Agile methodologies.
โข Strong analytical skills.
โข Practical understanding of cloud security policies and concepts.
โข Exposure to data governance and quality tools.
โข Experience with data integration, ETL/ELT orchestration, and application integration using APIs, messaging, or service-based architectures.
โข Basic understanding of AI/ML data requirements (training vs inference datasets), structured, semi-structured and unstructured data processing.
โข Exposure to streaming concepts, data parsing, text processing.
โข Familiarity with data quality and observability concepts.
โข Build, enhance, and debug scalable data pipelines and integrations.
โข Translate business and technical requirements into data engineering solutions.
โข Collaborate with lead data engineers and cross functional partners, including AI Fusion teams.
โข Support development of data products aligned with NYPAโs analytics and AI strategies.
โข Work with cloud and distributed data processing tools to deliver reliable, high quality data.
โข Develop data solutions that are flexible, extensible, elastic, secure and reliable at large scale.
โข Work with Lead Data Engineer to provide guidance and direction to project teams ensuring compliance with coding standards and best practices.
โข Collaborate with Data Governance team to capture and manage meta data, and implement data quality rules.
โข Building and managing data pipelines, data products, integrations and promoting production.
โข Develop Application Integrations, APIs and Microservices using hybrid cloud architecture.
โข Continuously learn and be at the leading edge of Data/Application Integration, Cloud, Containerization, and other industry trends.
โข Work with stakeholders including product, data and business teams to assist with data-related technical issues and support their data infrastructure needs.
โข Follow Cyber security guidelines and polices to monitor the company's data security and privacy.
โข Build and maintain batch data pipelines for structured and semi-structured data.
โข Support ingestion and preprocessing of unstructured data.
โข Implement basic data quality validations (schema checks, null checks).
โข Assist in preparing AI-ready datasets for analytics, AI/ML and GenAI use-cases.
โข Support implementation of data contracts through schema validation and data checks.
Preferred:
โข Cloud platform certification is preferred.
Company:
New York Power Authority is the largest state public power organization,producing some of the cheapest electricity. Founded in 1931, the company is headquartered in White Plains, USA, with a team of 1001-5000 employees. The company is currently Late Stage.