ZipRecruiter

Log In

1

Airflow Developer Jobs in New York (NOW HIRING)

Phyton Talent Advisors

Senior Data Engineer

Jersey City, NJ · On-site

$110K - $150K/yr

Build and manage workflow orchestration using TWS/ Apache Airflow. * Ensure data quality ... Bachelor''s degree in Computer Science, Engineering, Information Systems, or a related field. * 5+ ...

Citigroup, Inc.

Lead Generative AI Developer

New York, NY · On-site

$176K - $265K/yr

Exposure to Kafka, Spark, or Airflow for data pipeline engineering * Experience working in an Agile/SAFe delivery environment * Advanced degree (M.S.) in Computer Science, AI/ML, or a related ...

Beeutech

Snowflake Developer

Manhattan, NY · On-site

$125 - $150/hr

Position: Snowflake Developer Location: New York, NY [Onsite/Hybrid] Experience: (7-10) Years ... Knowledge of ETL/ELT tools (dbt, Informatica, Talend, Matillion, Airflow, etc.) * Understanding of ...

Gain America

Data Engineer - Remote (US)

Hicksville, NY · On-site +1

$124K/yr

Python, SQL, Spark, Snowflake, dbt, Airflow, AWS/Azure data pipelines What you'll do * Design and ... data engineering with strong Python and SQL * Hands-on Spark, Snowflake, and dbt experience

Citi

Senior Generative AI Developer

New York, NY

$142K - $213K/yr

Spark, Airflow, Kafka Databases: PostgreSQL, MongoDB, Redis Education: Bachelor's degree/University ... Systems & Engineering ----- Time Type: Full time ----- Primary Location: New York New York United ...

247Hire

Senior Analytics Engineer

Manhattan, NY · On-site

$125 - $150/hr

Build reliable data pipelines using tools like dbt, Airflow, ensuring high quality and ... Senior Analytics Engineer Qualifications * Over 5 years of experience in analytics engineering ...

previous pagenext page

Showing results 21-40

All JobsAirflow Developer JobsAirflow Developer Jobs in New York

Airflow Developer information

What is an Airflow developer?

Airflow Developers are professionals who design, build, and maintain data workflows using Apache Airflow, an open-source platform for orchestrating complex computational workflows and data processing pipelines. They are responsible for writing, scheduling, and monitoring tasks (DAGs) that automate data movement and transformation across systems. Airflow Developers work closely with data engineers, analysts, and other stakeholders to ensure reliable and efficient data pipeline automation. Their expertise includes Python programming, Airflow configuration, troubleshooting, and best practices for scalable workflow management.

What are the key skills and qualifications needed to thrive as an Airflow developer, and why are they important?

To thrive as an Airflow Developer, you need strong programming skills in Python, experience with data pipelines, and a solid understanding of workflow orchestration concepts. Familiarity with Apache Airflow, cloud platforms (like AWS or GCP), and version control systems such as Git are typically required, along with knowledge of containerization tools like Docker. Analytical thinking, attention to detail, and effective communication are key soft skills for collaborating with data teams and troubleshooting complex workflows. These competencies ensure reliable, scalable, and maintainable data pipeline solutions that support organizational data needs.

What are some common challenges Airflow developers face when managing complex data pipelines, and how can these be addressed?

Airflow Developers often encounter challenges such as managing dependencies between tasks, handling large-scale workflows, and ensuring reliable pipeline execution. To address these, it's essential to design modular DAGs (Directed Acyclic Graphs), implement robust error handling, and use features like sensors and retries strategically. Collaboration with data engineers and stakeholders is also key for troubleshooting and optimizing workflows. Effective monitoring and logging practices further help in quickly identifying and resolving issues.

What is the difference between Airflow Developer vs Data Engineer?

AspectAirflow DeveloperData Engineer
Required CredentialsKnowledge of Apache Airflow, Python, SQLData modeling, SQL, Python, cloud platforms
Work EnvironmentFocus on workflow orchestration, automationData pipeline development, storage, processing
Industry UsageTech, finance, healthcare for workflow automationBroad industries for data infrastructure

While both roles involve working with data and Python, an Airflow Developer specializes in designing and maintaining workflow automation using Apache Airflow. In contrast, a Data Engineer builds and manages data pipelines and infrastructure across various tools and platforms. The roles often overlap but differ mainly in scope and focus.

Does Airflow require coding?

Airflow developers typically need to have programming skills in Python, as workflows are defined using code. Coding knowledge is essential for creating, maintaining, and troubleshooting data pipelines in Airflow.

Is Airflow part of DevOps?

An Airflow Developer works with Apache Airflow, a platform used to programmatically author, schedule, and monitor workflows. While Airflow is often employed within DevOps environments to automate data pipelines and deployment processes, it is not inherently part of DevOps but complements DevOps practices by enabling automation and orchestration. Knowledge of CI/CD tools and infrastructure management is beneficial for such roles.

What are popular job titles related to Airflow Developer jobs in New York?

For Airflow Developer jobs in New York, the most frequently searched job titles are:

What job categories do people searching Airflow Developer jobs in New York look for?

The top searched job categories for Airflow Developer jobs in New York are:

What cities in New York are hiring for Airflow Developer jobs?

Cities in New York with the most Airflow Developer job openings:

Airflow Developer jobs near you
Infographic showing various Airflow Developer job openings in New York as of August 2026, with employment types broken down into 81% Full Time, 6% Part Time, and 13% Contract. Highlights an 83% Physical, 5% Hybrid, and 12% Remote job distribution.

Engineer - Delta Lake & Unity Catalog Developer

Exchange Robotics Inc

Manhattan, NY • On-site

$150 - $200/hr

Other

Posted 19 days ago

Job description

Design and build scalable, governed data lake architectures using Delta Lake and Unity Catalog to power enterprise AI solutions for financial institutions.

Help develop the data foundation behind Scalata's generative AI platform by enabling secure, compliant, and high-performance analytics.

What you'll do
  • Architect, implement, and optimize Delta Lake tables for analytics and machine learning workloads
  • Build and maintain scalable batch and streaming ETL pipelines using Apache Spark
  • Implement data governance, permissions, lineage, and auditing using Unity Catalog and related technologies
  • Design enterprise data catalogs, schemas, and storage architectures for secure data management
  • Integrate cloud storage platforms such as AWS S3, Azure Data Lake Storage, or Google Cloud Storage
  • Develop data pipelines using Python, Scala, Rust, or Go
  • Automate workflows using Airflow, dbt, and CI/CD best practices
  • Collaborate with AI, Data, and Engineering teams to support enterprise-scale financial AI applications
What we’re looking for
  • Experience with Delta Lake, Apache Spark, and large-scale data engineering
  • Hands-on experience with Unity Catalog, Apache Iceberg, Hive Metastore, or similar catalog technologies
  • Strong SQL skills for data modeling, governance, and permission management
  • Experience building ETL pipelines and orchestrating workflows with Airflow, dbt, or similar tools
  • Knowledge of cloud data lake architectures on AWS, Azure, or Google Cloud
  • Experience with Python, Scala, Rust, or Go
  • Strong understanding of data governance, lineage, auditing, and enterprise security
What we offer
  • Competitive salary and meaningful equity
  • Work with cutting-edge AI and data infrastructure technologies
  • Professional development and technical growth opportunities
  • Collaborative engineering team building the future of financial AI
#J-18808-Ljbffr

Apply