1

Contractual Azure Databricks Jobs in New York (NOW HIRING)

Contractual Azure Databricks information

What is the difference between Contractual Azure Databricks vs Data Engineer?

AspectContractual Azure DatabricksData Engineer
CredentialsAzure certifications, Spark, SQLData engineering certifications, cloud platform knowledge
Work EnvironmentCloud-based, project-specific contractsVaried, including cloud and on-premises environments
Employer & Industry UsageConsulting firms, tech companies, project-based rolesTech companies, finance, healthcare, often permanent or contract roles

Contractual Azure Databricks professionals focus on implementing and managing data solutions on the Azure platform for specific projects, often on a contractual basis. Data Engineers build and maintain data pipelines across various environments. While both roles require cloud and data processing skills, Contractual Azure Databricks specialists are more platform-specific, whereas Data Engineers have broader responsibilities across data architecture and engineering tasks.

What are the most commonly searched types of Azure Databricks jobs in New York?

The most popular types of Azure Databricks jobs in New York are:

What cities in New York are hiring for Contractual Azure Databricks jobs?

Cities in New York with the most Contractual Azure Databricks job openings:

Databricks Engineer / Architect

Celer Soft LLC

Jersey City, NJ • On-site

Other

Posted 4 days ago


Job description

Position: Databricks Engineer / Architect
Employment Type: Contractual
Work Arrangement: Remote
Experience: 10+ years overall, with 4+ years of hands-on Databricks experience
Location: Remote
About the Role
We are seeking an experienced Databricks Engineer / Architect to design, develop, and optimize modern data platforms and large-scale data engineering solutions using Databricks and cloud technologies.
The ideal candidate will have strong hands-on expertise in Databricks, Apache Spark, Python/SQL, Delta Lake, data architecture, ETL/ELT pipelines, and cloud platforms. This role requires someone who can operate at both the engineering and architecture levels translating business requirements into scalable, secure, and high-performance data solutions.
Key Responsibilities
Design and architect scalable, reliable, and high-performance data platforms using Databricks.
Develop and optimize data pipelines using PySpark, Python, SQL, Spark, and Delta Lake.
Design modern Lakehouse architectures and implement enterprise-grade data solutions.
Develop batch and streaming data pipelines and integrate data from multiple sources.
Implement Delta Lake capabilities including schema evolution, partitioning, optimization, and data lifecycle management.
Work with Databricks Workflows, Jobs, notebooks, clusters, and related platform capabilities.
Design and implement data ingestion, transformation, and orchestration frameworks.
Optimize Spark workloads, Databricks clusters, SQL queries, and data pipelines for performance and cost.
Establish and implement data engineering best practices, coding standards, CI/CD, and deployment processes.
Implement appropriate security, governance, access controls, and data quality mechanisms.
Collaborate with data scientists, analysts, application teams, cloud engineers, and business stakeholders.
Provide technical leadership and mentorship to data engineering teams.
Evaluate existing data architectures and recommend improvements, modernization strategies, and technology adoption.
Troubleshoot complex production issues and provide root-cause analysis and long-term solutions.
Participate in architecture reviews, technical design sessions, and documentation of enterprise data solutions.
Required Technical Skills
Databricks & Data Engineering
Strong hands-on experience with Databricks
Advanced knowledge of Apache Spark / PySpark
Strong proficiency in Python and SQL
Extensive experience with Delta Lake
Experience designing and implementing scalable ETL/ELT pipelines
Strong understanding of Lakehouse architecture and modern data platforms
Experience with batch and real-time/streaming data processing
Cloud Technologies
Strong experience with at least one major cloud platform:
Microsoft Azure - Azure Data Lake Storage, Azure Data Factory, Azure Synapse, Azure Key Vault, Azure DevOps
AWS - S3, Glue, Lambda, IAM, Redshift, CloudWatch
Google Cloud Platform - Cloud Storage, BigQuery, Dataflow, Pub/Sub, IAM
Preferred Qualifications
Databricks certifications such as Databricks Certified Data Engineer or Databricks Certified Data Engineer Professional
Experience with Unity Catalog and Databricks governance capabilities
Experience implementing enterprise Medallion Architecture (Bronze/Silver/Gold)
Experience with structured streaming and event-driven architectures
Knowledge of data modeling and dimensional modeling
Experience with Kafka or other messaging/streaming platforms
Experience with data quality frameworks and observability
Familiarity with MLOps and integration with machine learning platforms
Strong communication, documentation, and stakeholder-management skills
Education
Bachelor's or Master's degree in Computer Science, Information Technology, Engineering, or a related field.
Equivalent professional experience may be considered.
What We re Looking For
The successful candidate should be:
A strong hands-on Databricks Engineer who can also think at the architecture level.
Comfortable working independently in a fully remote environment.
Capable of translating complex business requirements into scalable technical solutions.
Strong in problem-solving, troubleshooting, and performance optimization.
An effective communicator who can work with both technical and non-technical stakeholders.
Comfortable providing technical leadership and driving architecture decisions.