Job Summary:
Infinitive is a data & AI consultancy that enables global brands to deliver results through insights, innovation, and efficiency. The Director, Databricks Platform Architect will define and shape a unified platform service that enables scalable, governed, and cost-efficient data access across the bank.
Responsibilities:
• Define and champion the end-to-end architecture for the bank’s Databricks-based data platform, ensuring scalability, security, cost efficiency, and ease of adoption.
• Design a self-service platform layer that leverages Databricks Unity Catalog to deliver seamless data discovery, access, and observability across all environments.
• Establish architectural patterns and reference implementations that encourage enterprise-wide reuse and standardization.
• Lead the design and implementation of Databricks Unity Catalog as the central governance plane—defining catalog hierarchies, fine-grained access controls, and cross-environment lineage.
• Evaluate and implement metadata, RBAC/ABAC, and data masking capabilities to meet regulatory and compliance requirements (e.g., GLBA, GDPR, HIPAA).
• Define the template architecture that allows Unity Catalog to operate as a scalable and cost-effective shared service across lines of business.
• Engineer platform capabilities that provide deep visibility into compute, storage, and catalog operations through integrated observability, monitoring, and FinOps practices.
• Develop resource optimization strategies to balance performance and cost while maintaining compliance and SLAs.
• Establish metrics, dashboards, and alerts to ensure the platform scales predictably under enterprise workloads.
• Architect streamlined RESTful/GraphQL APIs for secure, governed data access and metadata integration.
• Ensure interoperability with enterprise systems, APIs, and external data consumers using modern, consistent, and documented integration patterns.
• Guide teams in building Lakehouse-aligned data models that maximize reuse and governance.
• Oversee design of ETL/ELT architectures (Spark, PySpark, SQL) that integrate seamlessly with Unity Catalog for lineage and access tracking.
• Partner with engineering, data science, and risk teams to align platform design with business outcomes and regulatory expectations.
• Influence architecture steering committees and platform engineering groups to adopt the Databricks foundation as a managed, enterprise-wide service.
• Promote a culture of easy adoption through clear design patterns, documentation, and working sessions.
• Mentor engineers and architects on Databricks, Unity Catalog, and best practices for cost, scale, and observability.
• Contribute to internal architecture communities and upskill teams across multiple domains.
Qualifications:
Required:
• Education: Bachelor’s or Master’s degree in Computer Science, Engineering, or related field.
• Experience: 8+ years in data architecture or platform engineering, including experience designing enterprise-scale, distributed data environments.
• Databricks Expertise: Deep hands-on knowledge of Databricks, Delta Lake, Apache Spark, and Lakehouse principles.
• Unity Catalog Mastery: Demonstrated success architecting and operationalizing Databricks Unity Catalog for enterprise governance, metadata management, and access control.
• Programming & Data: Advanced proficiency in Python (PySpark) and SQL; experience with cloud data platforms (AWS, Azure, or GCP).
• API Engineering: Strong background in API architecture (REST, GraphQL, OpenAPI) and applying best-in-class security and observability.
• Governance Knowledge: Expert-level understanding of data governance frameworks, data quality management, and regulatory compliance.
• Soft Skills: Outstanding communication and influence skills, with ability to advocate for design principles across executive, technical, and risk audiences.
Preferred:
• Experience deploying Databricks and cloud infrastructure using Terraform or IaC frameworks.
• Familiarity with MLflow and model governance integration.
• Relevant certifications (Databricks Certified Data Engineer, AWS/Azure/GCP Architect).
• Experience with real-time data streaming technologies (Kafka, Structured Streaming).
Company:
Infinitive helps companies master the digital world with capabilities in marketing, digital advertising, business transformation and more. Founded in 2003, the company is headquartered in Washington, USA, with a team of 51-200 employees. The company is currently Growth Stage.
