Director Llm Engineer Jobs (NOW HIRING)

Job Summary:
Bank of America is committed to helping make financial lives better through every connection. The Principal Engineer - Context Engineering & LLM Optimization is responsible for leading the engineering approach for solutions at the program level, focusing on context management and optimization for LLM applications.
Responsibilities:
• Develops the engineering approach for the entire program/portfolio solution and works with Architecture, to develop/analyze/deliver the implementation of technical enablers
• Leads the planning, definition, and design of the complex features which span multiple teams and explore solution alternatives
• Creates ideas on designing complex technology and solution development approaches
• Leads the technical oversight for teams in solution development including design reviews and code within own domain
• Defines the technology tool stack for the solution within ranged of internally approved and supported technologies
• Explores state-of-the-art technologies to improve development efficiencies, quality of test/QA coverage, and release management
• Leads and is responsible for the end-to-end test strategy/creation/adherence, and the integration between teams for a program/portfolio solution
• Design context engineering strategies for enterprise LLM and RAG applications.
• Define prompt architectures for system prompts, developer instructions, user prompts, retrieved context, tool outputs, conversation history, and structured constraints.
• Optimize context window usage through summarization, compression, ranking, filtering, deduplication, and context prioritization.
• Design retrieval orchestration patterns that determine what data is retrieved, when it is retrieved, and how it is injected into the LLM prompt.
• Partner with RAG database engineers to tune retrieval outputs for downstream reasoning quality.
• Partner with data ingestion engineers to improve source formatting, metadata, and chunk structures for better contextual use.
• Develop patterns for multi-turn conversation memory, session state, user intent preservation, and context refresh.
• Define strategies for grounding, citation handling, source attribution, conflicting evidence resolution, and hallucination reduction.
• Improve the experience for our developers, making it easier to deliver industry-leading solutions, while managing work efficiently and with the right controls
• Advance our technology platforms through innovation
• Reduce risk and improve quality across our technology portfolio by aligning to a single enterprise architecture strategy and delivering governance that enables consistency, integration and automation
• Design LLM evaluation frameworks for answer quality, factuality, instruction adherence, relevance, safety, and token efficiency.
• Establish prompt engineering and context engineering standards across product and platform teams.
• Evaluate LLM model behavior across different context sizes, retrieval strategies, and prompt structures.
• Define reusable patterns for agents, tool calling, function calling, dynamic prompt generation, and workflow-based reasoning.
• Lead technical reviews for LLM application design, prompt safety, and context efficiency.
• Serve as a senior technical authority for enterprise AI platform engineering.
• Own architecture decisions that impact multiple teams, systems, or domains.
• Create reusable patterns, reference architectures, standards, and engineering guardrails.
• Mentor senior engineers and influence technical direction without requiring direct reporting authority.
• Balance innovation with operational reliability, security, compliance, scalability, and cost management.
• Communicate complex AI and data engineering concepts clearly to engineering, product, risk, security, and executive stakeholders.
Qualifications:
Required:
• 10+ years of software engineering, data engineering, platform engineering, or AI engineering experience.
• 5+ years designing large-scale enterprise systems.
• 2+ years working with LLM, RAG, vector search, semantic search, or AI platform capabilities.
• Experience operating systems in regulated, security-conscious, or enterprise-scale environments.
• Extensive experience building or architecting production LLM, RAG, or AI assistant systems.
• Deep understanding of how LLMs use prompts, retrieved context, conversation history, system instructions, and tool outputs.
• Strong knowledge of context window management, token budgeting, prompt construction, grounding, and response evaluation.
• Experience with OpenAI, Azure OpenAI, Anthropic, Google Gemini, Meta Llama, or similar LLM ecosystems.
• Experience designing prompt templates, retrieval-augmented prompts, agent workflows, and tool-use orchestration.
• Familiarity with vector search, embeddings, reranking, semantic retrieval, and document chunking.
• Experience with automated LLM evaluation, prompt regression testing, and quality measurement.
• Ability to define enterprise standards for reliable, explainable, and secure LLM behavior.
• Proven ability to lead architecture across multiple engineering teams.
• Strong written and verbal communication skills.
• Bachelor’s degree in Computer Science, Engineering, Information Systems, Applied Mathematics, or a related technical field
Preferred:
• Experience with agentic workflows, multi-agent orchestration, function calling, or tool-augmented reasoning.
• Experience with prompt injection mitigation, jailbreak resistance, and secure context handling.
• Experience with token optimization, long-context models, summarization pipelines, and contextual compression.
• Experience with user personalization, enterprise memory patterns, or domain-specific copilots.
• Higher-quality LLM responses with better grounding and reduced hallucination.
• Lower token usage and improved response latency through efficient context construction.
• Standardized prompt and context patterns reused across teams.
• Improved evaluation coverage for LLM behavior, factuality, and instruction adherence.
• Better alignment between retrieved enterprise knowledge and generated responses.
• Enterprise architecture
• Distributed systems design
• AI platform engineering
• Data governance and security
• Cloud-native engineering
• Observability and operational excellence
• Technical strategy and roadmap development
• Cross-functional influence
• Vendor and platform evaluation
• Production support and continuous improvement
Company:
Bank of America is a financial institution that offers credit cards, home loans, and auto loan services. Founded in 1998, the company is headquartered in Charlotte, USA, with a team of 10001+ employees. The company is currently Late Stage.