Software Development Engineer In Test Manager Jobs in Holliston, MA

Ascensus is the leading independent technology and service platform powering savings plans across America, providing products and expertise that help nearly 16 million people save for a better today and tomorrow.

Section 1: Position Summary

We are hiring for multiple positions on our AI Technology team:

• Associate Level (0-2 years experience /New Grads)
• Mid Level (3-5 years experience)
• Senior Level (5-15 years experience)
• SDET (3+ years experience)

Video Interview Process:

As part of our Software Engineer hiring efforts, we're using a video screening process and coding assessment. To learn more and to help prepare for this click here!

Section 2: Job Functions, Essential Duties and Responsibilities

Technical Support & Triage

• Investigate and resolve issues reported by users of Agent Assist
• Reproduce issues and trace failures across:
• LLM responses and prompts
• Agent workflows and orchestration
• Tool/MCP calls and backend integrations (e.g., Salesforce and internal enterprise applications)
• Analyze logs, traces, and telemetry (e.g., Langfuse, NewRelic logs)
• Clearly document findings and partner with engineering on resolution leveraging Azure DevOps

AI System Debugging & Improvement

• Identify whether issues stem from prompting, retrieval (RAG), tool selection, or backend systems
• Propose improvements to agents, skills, prompts, and workflows
• Use AI-powered tools (e.g., Cursor) to accelerate debugging, analysis, and solutioning
• Contribute to continuous improvement of system reliability and response quality

User Enablement & Evangelism

• Train internal users on how to effectively use Agent Assist
• Run targeted demos, onboarding sessions, and office hours
• Act as a trusted advisor to client-facing associates
• Drive adoption by helping users understand both capabilities and limitations
• Cross-Functional Collaboration
• Partner closely with engineering, product, and AI teams
• Translate user issues into clear, actionable tickets and improvements
• Identify recurring patterns and escalate systemic issues

General

• Responsible for protecting, securing, and proper handling of all confidential data held by Ascensus to ensure against unauthorized access, improper transmission, and/or unapproved disclosure of information that could result in harm to Ascensus or our clients.
• At Ascensus we are guided by our Core Values of People Matter, Quality First and Integrity Always. They inspire us every day to prioritize an environment of respect for those we serve and one another and should be visible in your actions on a day-to-day.

Section 3: Experience, Skills, Knowledge Requirements

• Up to 3 years' experience in a Software Engineering, Software Support, QA, SDET, or engineering-adjacent role (including internships, co-ops, or academic projects)
• Bachelor's degree in Computer Science, Computer Information Systems, Business Information Systems, or a related technical field
• Demonstrated ability to troubleshoot and debug technical issues, focusing on root cause analysis
• Foundational experience or exposure to:
• API testing (Postman or similar)
• Reading logs and debugging application behavior
• Basic SQL or data inspection
• Exposure to or strong interest in AI/LLM-based systems (prompting, chatbots, RAG, or agents) through coursework, projects, or hands-on experimentation
• Experience using or willingness to learn AI-powered development tools (e.g., Cursor, Claude Code)
• Strong communication skills with the ability to translate technical issues into clear explanations for non-technical users
• Demonstrated curiosity and ability to learn quickly in a fast-evolving technical environment

• Qualified candidates must complete a video interview assessment after applying as the next step, to be completed within 7 days of receiving the link. How to prepare: Set aside 40-45 minutes for the self-guided assessment that includes games and questions. Find a quiet place to record and be camera-ready. You'll need a smartphone, tablet, or desktop computer with your camera and microphone enabled. You'll answer questions to share your skills and experience and bring your personality to the interview. This step accelerates the interview process, moving qualified candidates to hiring manager interview fast

• Experience supporting or building AI/agent-based applications is preferred
• Familiarity with the following is preferred:
• Retrieval-Augmented Generation (RAG)
• Tool/agent orchestration
• Prompt engineering

The national average salary range for this role is $65-150k in base pay, exclusive of any bonuses and benefits.This base salary range represents the low and high end of the salary range for this position. Actual salary offered will vary and may be above or below the range based on various factors including but not limited to location, experience, performance, and internal pay alignment. We do not anticipate that candidates hired will begin at the top of the range however, from time to time, it may occur on a case-by-case basis. Other rewards and benefits may include: 401(k) match, Medical, Dental, Vision, Paid-Time-Off, etc. For more information, please visit careers.ascensus.com/#Benefits.

For all virtual remote positions, in order to ensure associates can effectively perform their job duties with no distractions, we require an uninterrupted virtual workspace and there is also an expectation of family care being in place during business hours. Additionally, there is an internet work speed requirement of 25 Mbps or better for individual use. If more than one person is utilizing the same internet connection in the same household or building, then a stronger connection is required.If you are unsure of your internet speed, please check with your service provider. Note: For call center roles specifically, it is a requirement to either hardwire your equipment directly to the internet router or ensure your workstation is in close proximity to the router. Please ensure that you are able to meet these expectations before applying.

We are proud to be an Equal Opportunity Employer

Be aware of employment fraud. All email communications from Ascensus or its hiring managers originate from @ascensus.com or @futureplan.com email addresses. We will never ask you for payment or require you to purchase any equipment. If you are suspicious or unsure about validity of a job posting, we strongly encourage you to apply directly through our website.