Role: AZURE DATA ENGINEER
Location: Weehawken, NJ - 5 days onsite (Need Local Candidates from NY, NJ)
Client: Infosys/UBS
Need PP no. as well !!
Duration: 12 Months
Experience: Need 10+ Years experience
Job Details:
We are seeking a skilled Azure Data Factory Engineer/Developer with strong PostgreSQL knowledge to design, develop, and maintain data pipelines for extracting, transforming, and loading (ETL) data from various sources, including PostgreSQL databases, into Azure data platforms
- Design, Develop, and Maintain ADF Pipelines:
- Design, develop, and maintain robust and efficient ADF pipelines for data ingestion, transformation, and loading from PostgreSQL databases and other sources.
- Optimize ADF pipelines for performance, scalability, and reliability.
- Develop and implement data quality checks and validation rules within ADF pipelines.
- PostgreSQL Expertise:
- Proficiently use SQL to query, manipulate, and extract data from PostgreSQL databases.
- Understand PostgreSQL database architecture, performance tuning, and best practices.
- Troubleshoot and resolve issues related to PostgreSQL connectivity, performance, and data integrity.
- Data Integration and Transformation:
- Design and implement data transformation logic within ADF pipelines, leveraging various ADF activities and connectors.
- Develop and maintain data mappings and transformations to ensure data consistency and quality.
- Collaborate with data engineers and business stakeholders to understand data requirements and implement solutions.
- Monitoring and Optimization:
- Monitor the performance and health of ADF pipelines and PostgreSQL databases.
- Identify and resolve performance bottlenecks and data quality issues.
- Implement and maintain monitoring and alerting mechanisms for ADF pipelines and PostgreSQL databases.
- Skills:
- Azure Data Factory (ADF): Strong experience in designing, developing, and maintaining ADF pipelines.
- PostgreSQL: Extensive knowledge of PostgreSQL database architecture, SQL, and performance tuning.
- SQL: Proficient in writing and optimizing SQL queries.
- Data Modeling: Experience in designing and implementing data models.
- ETL/Data Integration: Experience with ETL processes and data integration tools.
- Cloud Computing: Familiarity with Azure cloud services and technologies.
- Problem-Solving: Strong analytical and problem-solving skills.
- Communication: Excellent communication and collaboration skills.
Feel Free to reach out to me on anjali08@kanakits.com