ZipRecruiter

Log In

1

Databricks Architect Jobs in California (NOW HIRING)

Tror AI for everyone

Databricks Architect

Los Angeles, CA · On-site

$69.75 - $91.50/hr

Databricks Architect Location : Los Angeles CA (Hybrid) The Databricks Architect is responsible for designing, implementing, and optimizing scalable data analytics and data engineering solutions on ...

Data Piper

Databricks Architect

Pleasanton, CA · Remote

$72 - $94.50/hr

Title: Resident Solutions Architect Job Type: Fulltime Location: Remote 10+ years experience in ... databricks Working knowledge of two or more common Cloud ecosystems (AWS, Azure, GCP) with deep ...

Data Piper

Databricks Architect

Pleasanton, CA · On-site +1

$72 - $94.50/hr

Resident Solutions Architect Job Type: Fulltime Location: Remote • 10+ years experience in data ... experience in development on databricks • Working knowledge of two or more common Cloud ...

Cloudious

Databrick Architect

Los Angeles, CA · On-site

$69.75 - $91.50/hr

C2C The Databricks Architect is responsible for designing, implementing, and optimizing scalable data analytics and data engineering solutions on the Databricks Lakehouse Platform. This role requires ...

next page

Showing results 1-20

All JobsDatabricks Architect JobsDatabricks Architect Jobs in California

Databricks Architect information

What is the difference between Databricks Architect vs Data Engineer?

AspectDatabricks ArchitectData Engineer
Primary FocusDesigning and implementing data solutions on Databricks platformBuilding, maintaining, and optimizing data pipelines and infrastructure
Skills & CertificationsDatabricks certifications, Spark, cloud platforms (AWS, Azure), SQLSQL, ETL tools, cloud platforms, programming (Python, Scala)
Work EnvironmentData platforms, cloud environments, collaboration with data teamsData pipelines, databases, cloud infrastructure, scripting

While both roles work with data and cloud platforms, a Databricks Architect primarily focuses on designing and implementing data solutions using Databricks, whereas a Data Engineer builds and maintains the data pipelines and infrastructure that support these solutions. The Architect often oversees the technical design, while the Engineer handles the day-to-day pipeline development.

What are the key skills and qualifications needed to thrive as a Databricks Architect, and why are they important?

To thrive as a Databricks Architect, you need strong expertise in big data engineering, cloud platforms (such as Azure or AWS), distributed computing, and proficiency in languages like Python or Scala, typically supported by a relevant degree and cloud certifications. Familiarity with Databricks Workspace, Apache Spark, Delta Lake, and CI/CD tools is crucial for designing and implementing scalable data solutions. Excellent problem-solving, communication, and project management skills set top performers apart by enabling effective collaboration and solution delivery. These competencies are essential for architecting reliable, high-performance data platforms that drive business insights and innovation.

What are some common challenges Databricks Architects face when designing large-scale data solutions?

Databricks Architects often encounter challenges such as optimizing cluster performance for cost and efficiency, ensuring data security and compliance across distributed environments, and integrating Databricks with legacy systems or diverse data sources. They must carefully design data pipelines and workflows to handle large volumes of data without bottlenecks, and also collaborate closely with data engineers, data scientists, and IT teams to align on best practices. Staying updated with evolving Databricks features and cloud platform updates is also essential for success in this dynamic role.

What is a Databricks Architect?

A Databricks Architect is an IT professional who designs, implements, and manages data solutions using the Databricks platform, which is built on Apache Spark. They are responsible for creating scalable data pipelines, optimizing data workflows, and ensuring security and compliance within the cloud environment. Databricks Architects often work closely with data engineers, data scientists, and business stakeholders to deliver robust analytics solutions that drive business insights. Their expertise helps organizations leverage big data technologies efficiently and effectively.
What are the most commonly searched types of Databricks Architect jobs in California? The most popular types of Databricks Architect jobs in California are:
What are popular job titles related to Databricks Architect jobs in California? For Databricks Architect jobs in California, the most frequently searched job titles are:
What job categories do people searching Databricks Architect jobs in California look for? The top searched job categories for Databricks Architect jobs in California are:
What cities in California are hiring for Databricks Architect jobs? Cities in California with the most Databricks Architect job openings:
Databricks Architect jobs near you

Databricks Architect

Tror AI for everyone

Los Angeles, CA • On-site

$69.75 - $91.50/hr

Contractor

Re-posted 2 days ago

Job description

Job Title: Databricks Architect

Location : Los Angeles CA  (Hybrid)

The Databricks Architect is responsible for designing, implementing, and optimizing scalable data analytics and data engineering solutions on the Databricks Lakehouse Platform. This role requires deep expertise in cloud platforms (Azure/AWS/GCP), distributed data processing, Delta Lake architectures, and modern data engineering practices. The architect will collaborate with cross-functional teams to define data strategies, ensure platform reliability, and enable advanced analytics, ML, and BI use cases.

 

Key Responsibilities

  • Architecture & Design
  • Design end-to-end Databricks Lakehouse architectures for data ingestion, processing, storage, and consumption.
  • Define and implement Delta Lake patterns, including medallion architecture (Bronze/Silver/Gold).
  • Develop scalable data pipelines using PySpark, Spark SQL, and Databricks workflows.
  • Architect solutions for structured, semi-structured, and unstructured data.
  • Engineering & Implementation
  • Build robust ETL/ELT pipelines with Databricks notebooks, jobs, and workflows.
  • Design and implement high-performance streaming solutions using Structured Streaming.
  • Optimize Spark jobs for cost, performance, and scalability.
  • Implement CI/CD and automation using Databricks Repos, Git, and DevOps pipelines.
  • Cloud & Platform Expertise
  • Architect solutions across Azure/AWS/GCP leveraging native cloud services (e.g., Azure Data Factory, AWS Glue, GCP Dataflow).
  • Ensure security, governance, and compliance through Unity Catalog, RBAC, and encryption.
  • Monitor workloads and optimize cluster configurations for performance and cost.
  • Collaboration & Leadership
  • Work closely with data engineers, data scientists, BI teams, and business stakeholders.
  • Act as a subject matter expert (SME) for Databricks best practices, standards, and patterns.
  • Conduct architectural reviews and guide teams on design decisions.
  • Lead PoCs, evaluate new features, and drive platform adoption.
  • Quality, Governance & Observability
  • Define standards for data quality, lineage, observability, and governance.
  • Implement automated testing frameworks for pipelines and notebooks.
  • Establish performance baselines and monitoring dashboards.

Required Skills & Experience

Technical Skills

  • 7+ years of experience in data engineering/architecture.
  • 3+ years of hands-on experience with Databricks.
  • Strong expertise in Spark, PySpark, SQL, and distributed data processing.
  • Deep understanding of Delta Lake features: ACID transactions, OPTIMIZE, ZORDER, Auto Loader.
  • Experience with workflow orchestration, jobs, and Databricks REST APIs.
  • Hands-on expertise with at least one cloud platform:
  • Azure (preferred): ADF, ADLS, Key Vault, Event Hub, Azure DevOps
  • AWS: S3, Glue, Lambda, Kinesis
  • GCP: GCS, Dataflow, Pub/Sub
  • Familiarity with CI/CD, Git, DevOps, and Infrastructure-as-Code (Terraform preferred).
  • Soft Skills
  • Strong analytical and problem-solving skills.
  • Excellent communication and stakeholder management.
  • Ability to lead design discussions and guide technical teams.
  • Strong documentation and architectural blueprinting skills.

Preferred Qualifications

  • Databricks certifications, such as:
  • Databricks Certified Data Engineer Professional
  • Databricks Certified Machine Learning Professional
  • Databricks Lakehouse Fundamentals
  • Experience with MLflow, Feature Store, or MLOps workflows.
  • Experience working in regulated industries (BFSI, healthcare, etc.).