Artificial Intelligence Machine Learning Engineer Jobs in Birmingham, AL

Job Title: Enterprise Data Architect - Customer Data Platform

*********This position follows a hybrid work schedule and requires working on-site three (3) days per week and remotely two (2) days per week. **************

About the Role

We are seeking a Senior Data/Solution Architect to lead the design, governance, and evolution of our Azure Databricks-based Lakehouse platform. This platform serves as the system of record for replicated Customer Information System (CIS) data and integrates numerous secondary applications supporting billing, rates, credit management, front office, payments, service orders, and metering & usage reporting, analytics, data science, and artificial intelligence for a customer base of approximately 4.8 million electric utility customers.

This is a highly visible, cross-functional role at the intersection of enterprise architecture, data engineering, and utility business operations. You will define the technical vision for how CIS data is replicated, transformed, secured, and consumed across the enterprise, while ensuring the platform meets the reliability, auditability, and scalability demands of critical utility infrastructure. 

Job Summary

We are seeking an experienced and strategic architect to lead the vision, design, governance, and evolution of our Azure Databricks Lakehouse platform. This role serves as the technical authority for enterprise data architecture, providing leadership across customer business operational data domains that support critical utility business processes. The architect will be responsible for establishing scalable, secure, and resilient architectural patterns that enable trusted data delivery for operational reporting, business intelligence, advanced analytics, regulatory compliance, and data-driven decision-making.

The successful candidate will own the end-to-end architecture of the Lakehouse ecosystem, including data ingestion, replication, storage, transformation, governance, integration, and consumption. Working closely with enterprise architects, application teams, cybersecurity, and business stakeholders, the architect will design solutions that ensure data quality, lineage, availability, performance, and regulatory compliance while supporting both batch and near-real-time business needs. This role will establish standards for data modeling, metadata management, platform operations, and engineering practices, ensuring consistency and interoperability across the enterprise.

In addition to providing architectural leadership and mentoring technical teams, the architect will guide technology strategy, platform roadmaps, cost optimization, and continuous improvement initiatives. Through strong collaboration and governance, this role will enable a trusted, highly available, and cost-effective data platform that accelerates business value, supports customer and revenue operations, and positions the organization for long-term success in data management, reporting, analytics, data science, and artificial intelligence.

Key Responsibilities

• Own the end-to-end architecture and strategic direction of the Azure Databricks Lakehouse platform, including data ingestion, storage, processing, governance, and consumption frameworks.
• Define and maintain enterprise architecture standards for the Lakehouse, including medallion (Bronze/Silver/Gold; Landing/Raw/Curated/Aggregated) design patterns, Delta Lake implementation, data modeling, metadata management, and platform scalability.
• Architect and oversee near-real-time and batch replication of Customer Information System (CIS) data into the Lakehouse, ensuring data accuracy, integrity, availability, and compliance with defined service level objectives.
• Design and govern integration patterns supporting downstream operational, analytical, and regulatory applications, enabling secure and reliable consumption of customer, billing, metering, and premise data.
• Serve as the final architectural approval authority for platform standards, design patterns, data integration approaches, and technology selection decisions within the Customer Data Platform ecosystem.
• Partner with data science, analytics, and business teams to establish architectural patterns that enable machine learning, generative AI, and advanced analytics use cases while maintaining governance and security standards.
• Provide architectural oversight and technical governance for vendor, consulting, and system integration partners, fostering collaborative and constructive relationships that promote shared accountability, open communication, and alignment with enterprise standards, architectural principles, and strategic business objectives.
• Establish standards and best practices for data quality, lineage, cataloging, observability, and governance through tools such as Unity Catalog and enterprise metadata management platforms.
• Lead architecture decisions for Change Data Capture (CDC), streaming, and data integration technologies to support timely and reliable movement of enterprise data.
• Define logical and physical data models that support operational reporting, business intelligence, advanced analytics, data products, and regulatory compliance requirements.
• Partner with cybersecurity, privacy, and compliance teams to implement architecture controls for data protection, access management, encryption, masking, and regulatory compliance.
• Design and maintain architecture patterns that support high availability, disaster recovery, business continuity, and operational resilience for critical customer and revenue-related processes.
• Provide technical leadership for Data Engineering teams through architecture reviews, design guidance, technology standards, and mentorship.
• Develop and maintain architecture artifacts including reference architectures, solution designs, data flow diagrams, architecture decision records, standards, and multi-year platform roadmaps.
• Collaborate with enterprise architects, application owners, vendors, and business stakeholders to translate strategic objectives and business requirements into scalable technology solutions.
• Lead platform capacity planning and FinOps activities to optimize Databricks consumption, Azure infrastructure utilization, and overall platform cost efficiency.
• Evaluate emerging technologies, architectural patterns, and industry best practices to continuously improve the Lakehouse platform and accelerate delivery of business value.
• Support major incident investigations, root cause analyses, and platform improvement initiatives to enhance reliability, performance, and operational excellence.
• Serve as the enterprise authority for customer, billing, metering, and related utility data architecture, ensuring consistency, interoperability, and alignment across data, reporting, analytics, and operational initiatives.

Expected Outcomes

• Trusted, governed, and highly available CIS data ecosystem.
• Scalable Lakehouse architecture supporting enterprise reporting, analytics, and operational workloads.
• Consistent data standards, quality controls, and architectural governance across all data domains.
• Secure and compliant management of customer and operational data.
• Reduced integration complexity through reusable architecture patterns and shared services.
• Optimized platform performance and cost management across Azure and Databricks environments.
• Clear architectural roadmaps that align technology investments with business priorities.

Qualifications

• Bachelor's degree in computer science, Information Systems, Data Engineering, Information Technology, or a related discipline. Master's degree preferred.
• Minimum of 8 years of progressive experience in data architecture, data engineering, analytics platform engineering, or related disciplines, including at least 3 years serving in a senior architecture or technical leadership role.
• Demonstrated experience designing and governing enterprise-scale cloud data platforms, preferably Azure-based Lakehouse, data warehouse, or modern data ecosystem architectures.
• Deep expertise with Azure Databricks, Delta Lake, Azure Data Lake Storage Gen2 (ADLS), Unity Catalog, and related Azure data services.
• Strong experience designing and implementing large-scale data ingestion, replication, ETL/ELT, and streaming architectures supporting both batch and near-real-time workloads.
• Proven experience architecting solutions utilizing Apache Spark, Databricks Workflows, Delta Live Tables, Structured Streaming, Event Hubs, Kafka, or similar modern data processing technologies.
• Extensive knowledge of data architecture principles, including dimensional modeling, Data Vault, canonical data models, and Lakehouse architectural patterns.
• Experience designing and governing enterprise data quality, metadata management, lineage, reference data, master data management, and data governance capabilities.
• Strong understanding of Change Data Capture (CDC) technologies and replication frameworks including tools such as Qlik Replicate, HVR, Fivetran, Debezium, GoldenGate, or equivalent technologies.
• Experience defining secure data architectures, including role-based access control, data classification, encryption, masking, privacy controls, and regulatory compliance requirements.
• Knowledge of platform resiliency practices including high availability, disaster recovery, business continuity, observability, monitoring, and operational support models.
• Demonstrated ability to establish architecture standards, conduct architecture reviews, and drive adoption of engineering best practices across multiple delivery teams.
• Experience developing architecture artifacts including reference architectures, data flow diagrams, architecture decision records (ADRs), technical standards, and strategic roadmaps.
• Proven ability to lead cross-functional initiatives and collaborate effectively with business stakeholders, enterprise architects, application teams, cybersecurity, infrastructure teams, vendors, and implementation partners.
• Strong analytical, problem-solving, and decision-making skills with the ability to balance business priorities, technical requirements, risk, and cost considerations.
• Excellent communication, presentation, and influencing skills, with the ability to communicate architectural concepts to both technical and non-technical audiences.

Preferred Qualifications

• Experience supporting or implementing utility Customer Information Systems (CIS) such as Oracle Customer to Meter (C2M), Oracle CC&B, SAP IS-U, Cayenta, or similar platforms.
• Knowledge of utility business processes including customer service, billing, collections, meter-to-cash, revenue assurance, rates, regulatory reporting, and customer analytics.
• Experience architecting data solutions that support operational reporting, self-service analytics, business intelligence, and enterprise data products.
• Familiarity with utility regulatory, privacy, and security requirements, including NERC CIP, state public utility commission (PUC) reporting requirements, and customer data protection standards.
• Experience with Agile, Scrum, SAFe, and product-oriented delivery models.
• Azure, Databricks, or other relevant cloud, data, or architecture certifications.

Knowledge, Skills, and Abilities

• Enterprise Data Architecture
• Azure Databricks and Lakehouse Architecture
• Data Modeling and Information Design
• Utility CIS and Customer Data Domains
• Data Governance and Metadata Management
• Cloud Security and Compliance
• Data Integration and CDC Technologies
• Performance Optimization and Scalability
• FinOps and Cloud Cost Management
• Strategic Technology Planning
• Architecture Governance and Standards Management
• Stakeholder Engagement and Executive Communication
• Technical Leadership and Team Mentoring
• Vendor and Partner Management

Behavioral Attributes

• Business Value Mindset - Demonstrates a deep commitment to delivering business outcomes, not just technical solutions. Actively seeks opportunities to leverage data, analytics, and technology to improve operational performance, increase customer value, reduce risk, and advance Southern Company's strategic objectives.
• Ownership and Accountability - Takes personal responsibility for commitments, outcomes, and quality of work. Embraces challenges with a solutions-oriented mindset, remains effective amid ambiguity and changing priorities, and follows through to resolution without requiring excessive oversight.
• Initiative and Self-Leadership - A proactive self-starter who identifies needs, anticipates risks, and acts decisively. Independently drives work forward, removes obstacles, and continuously seeks opportunities to improve processes, platforms, and team effectiveness.
• Execution Excellence - Delivers results with urgency, discipline, and professionalism. Balances strategic thinking with practical execution, makes sound decisions in a timely manner, and consistently meets commitments while maintaining high standards of quality and reliability.
• Collaborative Leadership - Builds trust-based relationships across business and technology organizations. Communicates clearly, listens actively, influences constructively, and works effectively with stakeholders at all levels to achieve shared objectives and drive organizational success.
• Learning Agility and Innovation - Demonstrates intellectual curiosity and a passion for continuous improvement. Actively stays current on industry trends, emerging technologies, and best practices, applying new ideas to solve business problems and enhance organizational capabilities.
• Operational and Safety Commitment - Promotes a culture of safety, reliability, and operational excellence. Exercises sound judgment, adheres to established policies and standards, and takes responsibility for protecting the well-being of colleagues, customers, and the organization's critical assets.
• Stewardship and Integrity - Acts as a trusted steward of enterprise data, technology, and information assets. Demonstrates professionalism, transparency, and ethical decision-making while maintaining the highest standards of security, compliance, and accountability.
• Enterpri...