Catalog Model Jobs in Raleigh, NC (NOW HIRING)

Job Summary:
TowneBank is seeking a Data Engineer to design, build, and maintain batch ETL pipelines on a modern Databricks Lakehouse platform. The role involves taking ownership of data pipelines from ingestion through delivery, ensuring data quality, and collaborating with various stakeholders to translate business needs into effective data solutions.
Responsibilities:
• Design, build, and maintain batch ETL pipelines that ingest data from diverse source systems into the Databricks environment.
• Own the full pipeline lifecycle including ingestion, transformation, serving, monitoring, and incident resolution.
• Implement automated validation, reconciliation checks, and data quality gates across pipelines.
• Ensure data meets standards for accuracy, completeness, timeliness, and consistency.
• Maintain historical data for auditability and compliance.
• Optimize data processing performance on Databricks through efficient Spark SQL, partitioning strategies, and Delta Lake table maintenance.
• Design dimensional models (star schema, aggregation tables) that serve Power BI dashboards and self-service analytics effectively.
• Prepare the semantic layer for AI-powered analytics capabilities, including Databricks Genie Rooms, through clean business logic, well-documented table relationships, and intuitive naming conventions.
• Establish and maintain runbooks, deployment procedures, coding standards, and operational documentation.
• Contribute to code review practices, automated quality checks, and repeatable processes that enable the team to scale.
• Adhere to enterprise data governance policies and implement security best practices for sensitive financial data.
• Enforce access controls, encryption, and data lineage tracking across pipelines in accordance with banking regulations.
• Work with data architects, analysts, and business stakeholders to gather requirements and translate business needs into scalable data solutions.
• Communicate technical constraints and timelines clearly to non-technical partners.
• Support the development of junior and mid-level engineers through code review, pairing, and in-context coaching.
• Contribute to a culture of continuous learning and shared technical ownership.
• Identify and implement improvements to enhance pipeline stability, efficiency, and scalability.
• Evaluate and adopt emerging Databricks features and industry best practices as appropriate.
• Adhere to applicable federal laws, rules, and regulations including those related to Anti-Money Laundering (AML) and the Bank Secrecy Act (BSA).
• Other duties as assigned.
Qualifications:
Required:
• Bachelor’s degree in Computer Science or related field (or equivalent practical experience).
• 5+ years of experience as a data engineer in complex, large-scale data environments, preferably in the cloud.
• Strong hands-on expertise with Databricks and the Apache Spark ecosystem (PySpark, Spark SQL) for building and optimizing large-scale data pipelines.
• Production experience with Delta Lake tables and Lakehouse architectural patterns.
• Working experience with OPTIMIZE, VACUUM, Z-ordering, MERGE INTO for upserts, and time travel for debugging and auditing.
• Ability to articulate practical differences between Delta Lake and raw Parquet.
• Proficient in Python (including PySpark) for data processing.
• Strong SQL skills for complex querying and transformation.
• Emphasis on writing clean, well-structured, maintainable code.
• Strong understanding of dimensional modeling and data warehousing concepts, including star schemas, slowly changing dimensions, and designing tables optimized for BI tool consumption (import vs. DirectQuery, pre-aggregation strategies).
• Experience designing end-to-end data pipeline architectures including orchestration, monitoring, alerting, and error handling.
• Familiarity with Databricks Jobs, Apache Airflow, or equivalent workflow tools.
• Hands-on experience implementing automated data validation, reconciliation checks, and quality gates in production ETL pipelines.
• Knowledge of data governance standards and security best practices for managing sensitive data.
• Understanding of compliance requirements in banking (encryption, PII handling, auditing) and ability to enforce access controls and data lineage documentation.
• Experience with Git-based workflows and CI/CD pipelines for deploying data pipeline code in a controlled, repeatable manner.
• Ability to translate technical concepts for non-technical audiences, document processes clearly, and collaborate effectively across engineering and business teams.
Preferred:
• Experience in banking, financial services, or other regulated industries.
• Familiarity with credit risk data, financial reconciliation, or regulatory reporting requirements.
• Experience building pipelines for compliance use cases with audit trail, data lineage, and accuracy requirements (e.g., CCAR, Basel, AML reporting).
• Experience with Databricks Unity Catalog for data governance, access control, and catalog management.
• Familiarity with Power BI development, DAX, or semantic modeling for analytics consumption.
• Exposure to Databricks AI/BI features, including Genie Rooms for natural-language analytics, or experience preparing data layers for AI-driven consumption.
• Exposure to real-time data streaming and event-driven architectures.
• Knowledge of Spark Structured Streaming or Kafka alongside batch workflows.
• Understanding of DataOps techniques including automated testing, monitoring, and CI/CD for data pipelines.
• Relevant certifications such as Databricks Certified Data Engineer or cloud platform data engineering certifications.
Company:
TowneBank is a bank that offers various financial services to individuals and businesses. Founded in 1999, the company is headquartered in Portsmouth, USA, with a team of 1001-5000 employees. The company is currently Late Stage.