Senior Software Engineer In Test Jobs in Amherst, MA

Job description

 =

We are seeking a Mid to Senior SDET to own and evolve our test infrastructure, enabling our engineering team to ship quality features at velocity. You will design test harnesses, establish testing best practices, and ensure our platform—including AI Agent capabilities—is thoroughly validated across API, backend, and full-stack scenarios.

This is not a QA role. You are an engineer with a testing focus, embedded with developers, contributing to sprint work, test strategy, and platform reliability.

Key Responsibilities

- Understand how all elements of the system software ecosystem work together and develop QA approaches and tools that fit the overall strategy

- Collaborate with Product, Architecture, and Engineering teams to define test strategy, coverage goals, and acceptance criteria

- Develop test strategies and create appropriate test harnesses for Conversational AI systems across Voice and Chat channels

- Identify testing gaps and recommend solutions

- Design, build, and maintain reusable test harnesses and frameworks that eliminate decision paralysis for engineers

- Provide clear patterns for: test framework selection, test data setup, and test implementation

- Create comprehensive examples and documentation so developers can write tests without reinventing solutions

- Provide test infrastructure to enable engineering teams to test and own quality of the services

- Support testing across Java/Node.js backends, APIs, and Conversational AI platforms

- Develop specialized test strategies for conversational AI workflows and agent interactions

- Build test harnesses for API-level, backend service, and end-to-end scenarios

- Validate complex multi-turn conversations, intent recognition, and orchestration logic

- Develop and automate Voice and Chat test plans

- Be a stakeholder of the release to ensure defects are fixed per SLA and end customer experience is protected and improved

- Monitor and report on test execution and quality metrics

- Validate that platform changes maintain or improve customer experience

- Educate and train engineers on test harnesses, frameworks, and best practices

- Mentor team members on test-driven development and automation patterns

- Make testing accessible—remove friction from the testing process

- When harness work is complete, contribute to automated test implementation and general development items

- Pair with developers on complex testing scenarios

- Help close any testing gaps across the team

Required Qualifications

- 5+ years total software engineering experience with 3+ years focused on test automation/SDET work

- A deep understanding of automation testing and ability to lead engineers in writing automation scripts/programs that aid automated testing

- Strong proficiency in at least one backend language: **Java, Node.js with object-oriented design experience

- API testing expertise: REST, request/response validation, mocking, contract testing.

- Test framework experience: JUnit, Jest, or equivalent; experience building custom frameworks a plus

- Tools and Test Platforms - SOAPUI, Postman, Bruno, Playwright, Selenium, Cucumber, Hammer and Cyara.

- Voice and Chat test automation expertise and familiarity with test methodologies for conversational systems.

- CI/CD pipeline knowledge: GitHub Actions, comfortable with automation in pipelines

- Test data management: Experience designing and maintaining test data strategies

- Full-stack testing: Comfortable testing across APIs, backends, and frontend layers

- Strong communication: Ability to document patterns, mentor others, and articulate testing strategy

- Familiarity with software technologies, methodologies, and applied software engineering practices such as Object-Oriented Design (OOD) and cloud software development

- Experience with testing LLM/AI responses, using such approaches as LLM-as-a-judge, etc

- On-site availability (required in Hartford CT or Wellesley MA)

Preferred Qualifications

- Deep hands-on technical expertise in automation, developing test frameworks, tools, and APIs

- Experience with conversational AI platforms

- Experience with contract testing or API mocking tools

- Knowledge of performance/load testing frameworks

- Understanding or experience testing AI agents