Python Automation Testing Jobs in Philadelphia, PA

At Customers Bank, we believe in working hard, working smart, working together to deliver memorable customer experiences and having fun. Our vision, mission, and values guide us along our path to achieve excellence. Passion, attitude, creativity, integrity, alignment, and execution are cornerstones of our behaviors. They define who we are as an organization and as individuals. Everyone is encouraged to have personal development plans. By doing so, our team members are on their way to achieve their highest potential and be successful in their personal and professional lives.

Must be legally eligible to work in the United States without sponsorship, now or in the future, to be considered.

Who is Customers Bank?

Founded in 2009, Customers Bank is a super-community bank with over $25 billion in assets. We believe in dedicated personal service for the businesses, professionals, individuals, and families we work with.

We get you further, faster.

Focused on you: We provide every customer with a single point of contact. A dedicated team member who's committed to meeting your needs today and tomorrow.

On the leading edge: We're innovating with the latest tools and technology so we can react to market conditions quicker and help you get ahead.

Proven reliability: We always ground our innovation in our deep experience and strong financial foundation, so we're a partner you can trust.

What You'll Do:

The AI Automation Engineer is a builder and owner of the Bank's intelligent automation layer - designing agentic pipelines, multi-step AI workflows, and the prompt and evaluation frameworks that keep model outputs production-grade. This is a high-impact, execution-focused role for a forward-thinking engineer who thrives at the intersection of AI innovation and real-world delivery.

You will own conversation design and live monitoring for voice agent buildout, drive the model ops half of the Bank's DevOps model, and bring production-grade Python engineering depth alongside AI-specific expertise. Operating across three lifecycle domains - Loans, Deposits, and Payments - you will be a key force in shaping how AI is built and scaled at Customers Bank.

Agentic Workflow Development

• Build and scale agentic workflows across Loans, Deposits, and Payments domains.
• Design multi-step, autonomous AI pipelines with dynamic decision-making and tool use.
• Implement human-in-the-loop (HITL) review gates and confidence scoring thresholds.
• Automate structured and semi-structured workflows including exception handling, reconciliations, and regulatory reporting.

Prompt Engineering & LLM Development

• Develop, version, and evaluate prompts for OpenAI and Anthropic (Claude) models.
• Apply LLM-based development patterns: tool/function calling, agent frameworks (LangChain, LangGraph, AutoGen), and retrieval-augmented generation (RAG).
• Build NLP-based intelligent document processing (IDP) pipelines - extraction, classification, and analysis for loan origination and deposit onboarding.
• Conduct prompt regression testing and maintain evaluation frameworks to detect output quality degradation.

Model Ops & DevOps (Shared Ownership)

• Monitor live workflow outputs, detect drift or quality degradation, and trigger rollbacks when performance thresholds break.
• Manage model and prompt version rollouts in coordination with Cloud Engineering release gates.
• Monitor automation performance, troubleshoot production incidents, and continuously optimize deployed solutions.

Voice Agent

• Design conversation flows, voice persona prompts, and escalation routing rules for the call center voice agent.
• Monitor call containment rates, resolution quality, and escalation patterns in production.
• Apply NLP-based transcript analysis to continuously improve voice agent performance.

Partner & Tooling Strategy

• Evaluate AI and automation vendors and frameworks, making informed build-vs-buy recommendations across the partner portfolio.
• Partner directly with business SMEs across all three domains to translate operational workflows into AI-executable rules.

Engineering Standards & Growth

• Write production-grade Python code with automated testing, error handling, logging, and full documentation.
• Design and promote reusable code patterns and architectural standards for enterprise-scale adoption.
• Ensure all solutions meet governance, audit trail, model risk, and SOC compliance requirements.
• Mentor junior engineers and help establish engineering culture and delivery standards as the team scales.

What Do You Need?

• 5-8 years of software engineering or automation development experience with a strong foundation in Python and demonstrated ability to build and ship production-quality code.
• Hands-on experience designing and delivering agentic or automation solutions via custom code, API orchestration, workflow engines, or AI frameworks.
• Demonstrated experience with LLM-based development patterns: prompt engineering, tool/function calling, agent frameworks (LangChain, LangGraph, AutoGen), RAG, and NLP-based document or transcript analysis.
• Hands-on experience with Microsoft Azure AI workloads - Azure Functions, Logic Apps, Service Bus, Microsoft Foundry, Azure Data Factory, and Azure DevOps.
• Strong command of software engineering fundamentals: version control, automated testing, CI/CD, error handling, logging, and documentation.
• Experience in financial services or a regulated environment with practical familiarity with compliance, audit trail, model risk, and data governance requirements.

Preferred Qualifications (Nice to Haves)

• Experience building or integrating voice AI / conversational AI solutions.
• Experience evaluating AI and automation vendors and contributing to build-vs-buy decisions.
• Microsoft Azure certification (Azure Developer Associate, Azure AI Engineer Associate).

Technology Skills:

• Production-grade Python engineering with full SDLC proficiency.
• Agent frameworks: LangChain, LangGraph, AutoGen; RAG architecture and NLP pipelines.
• Microsoft Azure AI workloads: Functions, Logic Apps, Service Bus, Foundry, Data Factory, DevOps.
• OpenAI and Anthropic (Claude) API integration and prompt management.
• Voice AI / conversational AI platforms (preferred).
• Ability to work with the Microsoft Suite and learn/work with other Customers Bank's applications.

Customers Bank is an equal opportunity employer. We do not discriminate based upon race, religion, color, national origin, gender (including pregnancy, childbirth, or related medical conditions), sexual orientation, gender identity, gender expression, age, status as a protected veteran, status as an individual with a disability, or other applicable legally protected characteristics.

We also provide "reasonable accommodations", upon request, to qualified individuals with disabilities, in accordance with the Americans with Disabilities Act and applicable state and local laws.

Diversity Statement:

At Customers Bank, we believe in working smart, working together, and having fun while delivering innovative solutions and memorable experiences for our customers. We are committed to the continual advancement of a culture which reflects the value we place on diversity, equity, and inclusion. We honor the diverse experiences, perspectives, and identities of our team members, and we recognize that it is their passion, creativity, and integrity that drives our success. Step into your future with us! Let's take on tomorrow.