Data Engineer Airflow Jobs in Edison, NJ (NOW HIRING)

Job Title: Sr Data Engineer 

 

Location:  Jersey City, NJ(Hybrid). 

 

Duration: 6-12 Months

 

Rate: DOE

 

 

Job Description:
 

We are seeking a hands-on Data Engineer with strong experience in building scalable enterprise data solutions within Financial Services environments. The ideal candidate will have expertise in cloud-based data platforms, modern data engineering practices, and large-scale data integration initiatives supporting operational, analytical, and regulatory data needs.
This role requires strong technical capabilities in data pipeline development, cloud data processing, Master Data Management (MDM), and enterprise data integration. The candidate should be comfortable working across complex distributed environments and partnering with architecture, analytics, governance, and business teams to deliver reliable, secure, and scalable data solutions.
 
Key Responsibilities:
•                    Design, develop, and support scalable data pipelines and enterprise data integration solutions.
•                    Build and maintain batch and real-time data ingestion, transformation, and processing frameworks.
•                    Develop cloud-native data engineering solutions supporting enterprise data lake, warehouse, and lakehouse platforms.
•                    Implement ETL/ELT processes for structured, semi-structured, and unstructured data sources.
•                    Support Master Data Management (MDM) initiatives across security, account, client, and reference data domains.
•                    Collaborate with data architects, business analysts, governance teams, and application teams to support enterprise data initiatives.
•                    Implement data quality validation, monitoring, metadata management, and lineage processes.
•                    Support cloud migration and modernization efforts involving legacy and enterprise data platforms.
•                    Optimize data processing, storage, and pipeline performance for scalability and operational efficiency.
•                    Ensure compliance with enterprise security, governance, and regulatory standards within financial services environments.
•                    Support reporting, analytics, and downstream consumption platforms through reliable and trusted data delivery.
 
Required Skills & Experience:
•                    Strong hands-on experience in Data Engineering and enterprise-scale data integration.
•                    Proven experience developing scalable ETL/ELT pipelines and distributed data processing solutions.
•                    Experience working with modern cloud-based data platforms and data ecosystems.
•                    Hands-on expertise with Strong SQL expertise along with programming/scripting experience in Python, PySpark, or Snowpark.
•                    Experience with dbt (Data Build Tool) for:
·               Data transformation and modeling
·               ELT pipeline development within Snowflake/Databricks
·               Modular, reusable SQL-based data workflows
·               Data testing, documentation, and version control integration
•                    Experience with cloud platforms such as Azure, AWS, or GCP, including integration with Snowflake and Databricks.
•                    Solid understanding of data lake, data warehouse, and lakehouse architectures, and their implementation across platforms.
•                    Experience with orchestration and workflow tools (e.g., Airflow, Databricks Workflows, Snowflake Tasks) for pipeline scheduling and automation.
•                    Experience supporting Master Data Management (MDM) and enterprise data governance initiatives.
•                    Familiarity with metadata management, data lineage, data cataloging, and data quality processes.
•                    Experience integrating diverse data sources, including:
·               APIs and microservices
·               File-based ingestion (batch)
·               Real-time/streaming data (e.g., Kafka, Spark Streaming)
•                    Knowledge of performance tuning, cost optimization, and scalability techniques across both Spark-based and Snowflake environments.
•                    Understanding of enterprise security, compliance, and governance standards, including RBAC, data masking, and encryption.
•                    Experience working in Agile and DevOps environments, including CI/CD for data pipelines.
 
Preferred Qualifications:
•                    Financial Services or Banking industry experience preferred.
•                    Experience supporting regulatory, risk, compliance, or operational reporting data environments.
•                    Exposure to real-time data processing and streaming technologies.
•                    Familiarity with CI/CD processes and infrastructure automation.
•                    Strong analytical, troubleshooting, and problem-solving skills.
•                    Excellent communication and collaboration skills.
 
Education:
Bachelor’s degree in Computer Science, Information Systems, Engineering, or related field.
 
Job Responsibilities
•                    Design, develop, and support scalable data pipelines and enterprise data integration solutions.
•                    Build and maintain batch and real-time data ingestion, transformation, and processing frameworks.
•                    Develop cloud-native data engineering solutions supporting enterprise data lake, warehouse, and lakehouse platforms.
•                    Implement ETL/ELT processes for structured, semi-structured, and unstructured data sources.
•                    Support Master Data Management (MDM) initiatives across security, account, client, and reference data domains.
•                    Collaborate with data architects, business analysts, governance teams, and application teams to support enterprise data initiatives.
•                    Implement data quality validation, monitoring, metadata management, and lineage processes.
•                    Support cloud migration and modernization efforts involving legacy and enterprise data platforms.
•                    Optimize data processing, storage, and pipeline performance for scalability and operational efficiency.
•                    Ensure compliance with enterprise security, governance, and regulatory standards within financial services environments.
•                    Support reporting, analytics, and downstream consumption platforms through reliable and trusted data delivery.