Senior Databricks Developer Jobs in Colorado (NOW HIRING)

THE COMPANY:

STACK INFRASTRUCTURE (STACK) provides digital infrastructure to scale the world's most innovative companies. We are an award-winning industry leader in building, owning, and operating highly efficient, cost-effective wholesale, colocation, and cloud data centers. Each of our national facilities meets or exceeds the highest industry standards in all operational categories of availability, security, connectivity, and physical resilience.

STACK offers the scale and geographic reach that rapidly growing hyperscale and enterprise companies need. The world runs on data. Data runs on STACK.

The Position:

The Senior Data and Platform Integration Engineer owns both the enterprise data architecture and the integration fabric that connects business applications into a unified, governed ecosystem. This dual-domain role builds the modern data platform (Azure Databricks, Delta Lake, Unity Catalog, Cosmos DB, Azure AI Search, ADLS Gen2) and architects the API integrations, event-driven pipelines, and identity provisioning that connect enterprise systems (NetSuite, Workday, Procore, Freshworks, SharePoint, M365) to each other and to the data platform. The primary platform is Azure, but the ideal candidate brings multi-cloud awareness across AWS and GCP. The scope includes data lakehouse architecture, NoSQL and vector database design, real-time streaming pipelines, RESTful/GraphQL API development, enterprise SSO/SCIM provisioning, and data governance and observability frameworks. This engineer ensures data flows seamlessly from source systems through transformation layers into consumption-ready serving layers for analytics, automation, and AI agents. Responsibilities include but are not limited to:

Modern Data Architecture & Platform Design

• Design the enterprise data architecture as a unified foundation for BI, analytics, and AI - defining data domains, ownership models, and consumption layers. Apply data mesh and data fabric principles for federated governance and metadata-driven discovery.

• Architect the data lakehouse on Azure Databricks with Delta Lake, Unity Catalog, and medallion architecture (Bronze/Silver/Gold) across dual EA subscriptions.

• Evaluate and recommend optimal platforms per workload: Databricks (lakehouse/ML), Cosmos DB or DynamoDB (operational NoSQL), Azure AI Search or purpose-built vector databases (semantic retrieval), and AWS/GCP services where cross-cloud strategies add value.

• Design NoSQL data models on Cosmos DB and vector/semantic search infrastructure on Azure AI Search with hybrid retrieval strategies optimized for RAG and AI workloads.

Enterprise Application Integration & API Development

• Design and build API integrations connecting NetSuite (ERP/Finance), Workday (HR), Procore (Construction PM), Freshworks ITSM, SharePoint, and M365 to the data platform and to each other.

• Develop RESTful and GraphQL APIs enabling automation workflows and AI agents to query, retrieve, and act on live business data across connected systems.

• Implement event-driven integration architectures using Azure Event Grid, Service Bus, Event Hubs, and Logic Apps for real-time data synchronization and trigger-based workflows.

• Create and maintain an API gateway layer (Azure API Management) for centralized governance, rate limiting, usage analytics, and developer self-service.

Identity, SSO & SCIM Provisioning

• Own the enterprise identity integration strategy - defining standards and configuring SSO and SCIM provisioning across all business applications using Entra ID, SAML 2.0, and OIDC protocols.

• Configure and maintain SCIM provisioning for automated user lifecycle management across NetSuite, Workday, Procore, Freshworks, and other SaaS applications.

• Manage Entra ID application registrations, service principals, and OAuth 2.0 flows for secure system-to-system communication.

• Provide SSO/SCIM configuration guidance and standards to the Digital Workplace team for FreshService and SharePoint implementations.

Data Engineering, Streaming & AI-Ready Pipelines

• Build modern ELT pipelines using Azure Data Factory, Databricks, dbt, and Azure Functions to ingest data from enterprise applications into consumption-ready serving layers.

• Design real-time pipelines using Event Hubs, Kafka/Confluent, and Spark Structured Streaming for operational intelligence and near-real-time data availability.

• Build AI-ready data pipelines: document ingestion (text extraction, OCR, chunking, metadata tagging), embedding generation, and vector index population for RAG and document intelligence.

• Implement incremental refresh strategies (CDC, event-driven triggers, delta loads) and automated data reconciliation across source systems.

Data Governance, Observability & Quality

• Establish data quality frameworks including automated validation, profiling, anomaly detection, and quality scoring for data feeding AI models and agents.

• Implement data observability (Monte Carlo, Great Expectations, or equivalent) for proactive detection of freshness violations, schema drift, and pipeline failures.

• Manage data lineage and cataloging via Unity Catalog and Microsoft Purview. Coordinate with the SharePoint Information Architecture and Governance Lead on content-level Purview policies.

• Define governance policies (classification, sensitivity labeling, retention, access controls) aligned with SOX compliance. Establish data product standards with SLAs per domain.

The Details:

• Location: Denver, CO or Dallas, TX

• Travel: <10%

• Benefits: Healthcare, Dental Care, Vision Insurance, Life Insurance, Paid Time Off, and Paid Leave Programs

• Must be eligible to work in the United States

• Must pass comprehensive background and drug screening

Must Have Qualifications:

• 6+ years of data engineering, data architecture, or platform integration experience with at least 2 years supporting AI/ML or advanced analytics workloads.

• Deep experience with Azure Databricks (Delta Lake, Unity Catalog, Spark) - Azure is the primary platform and is a must-have.

• Strong API integration experience connecting enterprise SaaS platforms (NetSuite, Workday, Procore, Freshworks, or similar) via REST APIs, GraphQL, webhooks, and event-driven patterns.

• Deep expertise in Entra ID (Azure AD): SSO configuration (SAML 2.0, OIDC), SCIM provisioning, application registrations, and OAuth 2.0 flows.

• Hands-on NoSQL experience: Cosmos DB required; familiarity with DynamoDB or MongoDB Atlas is a plus.

• Experience with vector and semantic search: Azure AI Search required; exposure to Pinecone, Weaviate, or Qdrant is a plus.

• Experience building ELT pipelines using Azure Data Factory, Databricks, dbt, or similar tools at enterprise scale.

• Hands-on experience with streaming architectures: Event Hubs, Kafka/Confluent, Spark Structured Streaming, or equivalent.

• Proficiency in Python and SQL; familiarity with JavaScript/TypeScript for API development is a plus.

• Solid understanding of data governance, observability, lineage tracking, and master data principles.

• Multi-cloud awareness: familiarity with AWS and/or GCP data services to evaluate cross-cloud strategies. Azure remains primary.

• Experience building AI-ready data infrastructure (embedding pipelines, vector indexes, RAG data preparation) is a strong plus.

Compensation Range:

$140,980.00 - $163,240.00

THIS MIGHT BE RIGHT FOR YOU IF:

• You are a strong communicator, you are persuasive and clear, blending analytics with experience in decision-making.

• You do not get flustered easily. You can juggle multiple priorities while balancing urgent requests with shifting timelines and deliverables.

• You are a team builder. You take the time to understand and develop the strengths of your resources while formulating long-term plans for the growth and success of the team.

• You are naturally curious and driven toward continual improvement. While you celebrate your successes, you take time to review and analyze campaigns for future learning.

WHY STACK?

• We offer a competitive compensation package with strong benefits, including medical, dental, and vision insurance, a 401K program, flexible spending accounts - even a cell phone subsidy.

• We foster a culture of appreciation, including peer-to-peer recognition and rewards programs.

• Fun is part of our DNA, with events, game nights, happy hours, and barbecues.

• We're growing - this is a great time to join and make an impact!

STACK is an Equal Opportunity Employer. All qualified applicants will receive consideration for employment without regard to race, color, religion, sex, sexual orientation, gender identity and expression, age, national origin, mental or physical disability, genetic information, veteran status, or any other status protected by federal, state, or local law

Note to external agencies: We are not accepting any blind submissions or resumes/cvs from recruitment agencies. Any candidates sent to STACK Infrastructure, Inc. will not be accepted or considered as a submission without a signed agreement in place. Fees will not be paid in the event a candidate submitted by a recruiter without an agreement in place is hired; such resumes will be deemed the sole property of STACK Infrastructure, Inc.