Contract Telemetry Engineer Jobs in Washington (NOW HIRING)

Everforth ECS is seeking a Monitoring & Telemetry Lead SME to work in the National Capital Region covering the Pentagon, Falls Church, and Fairfax. Please Note: This position is contingent upon contract award.
The War Data Platform (WDP) is a key initiative within the U.S. Department of War's (DoW) AI-First strategy introduced in early 2026. The WDP focuses on operational warfighting data and aims to accelerate the deployment of artificial intelligence (AI) on the battlefield. The WDP extends to Unclassified, Secret, and Top Secret environments, and supports collaboration between Combatant Commands, Joint Staff directorates, Senior Executive Service leaders, and operational analysts.
• This role defines, architects, and governs telemetry, observability, and service-level indicator frameworks supporting AI and machine learning model-serving operations across all WDP classification enclaves, ensuring enterprise-wide operational visibility, mission assurance alignment, and resilient monitoring of AI/ML-serving infrastructure and API ecosystems.
• Defines, architects, and governs telemetry, observability, and service-level indicator frameworks supporting AI and machine learning model-serving operations across Unclassified, Secret, and Top Secret enclaves within the War Data Platform (WDP) Core Integration enterprise.
• Establishes monitoring standards, performance indicators, traceability conventions, and health signal baselines for serving application programming interfaces, reverse proxies, model zoo interfaces, and external provider integrations supporting Combatant Commands, Joint Staff elements, and Senior Executive Service decision makers.
• Designs and integrates instrumentation patterns within deployment pipelines, runtime environments, service meshes, and logging and auditing frameworks to provide immediate operational visibility following production releases.
• Implements structured metrics pipelines using platforms such as Prometheus, Grafana, OpenTelemetry, Elastic, Splunk, and DoW-approved monitoring suites to capture latency, throughput, error rates, dependency bottlenecks, cross-domain access behavior, and cyber-relevant anomalies.
• Conducts telemetry readiness reviews, evaluates instrumentation completeness, and validates monitoring coverage for emerging serving endpoints, model artifacts, and external model provider interfaces.
• Coordinates with model-serving engineers, API engineers, DevSecOps teams, pipeline operators, cybersecurity personnel, and platform architects to integrate observability with test and evaluation gates, reliability assessments, and mission assurance workflows.
• Produces operational dashboards, service-level definitions, instrumentation standards, alerting policies, runbooks, and observability assessment reports that strengthen reliability, accelerate incident triage, and elevate mission readiness across all enclaves.
• Advances War Data Platform (WDP) Core Integration program value by delivering resilient, measurable, and domain-compliant monitoring capabilities for enterprise AI/ML model access.
• Performs other duties as assigned.
• Current Secret security clearance with the ability to obtain and maintain a Top Secret (TS) security clearance with Sensitive Compartmented Information (SCI).
• CompTIA A+ certification.
• Minimum 12 years of experience designing, implementing, and governing enterprise monitoring, telemetry, and observability frameworks across multi-domain or classified environments.
• Demonstrated hands-on expertise with monitoring and observability platforms, including Prometheus, Grafana, OpenTelemetry, Elastic, and Splunk, with proven ability to architect structured metrics pipelines and operational dashboards in production environments.
• Experience integrating observability frameworks with DevSecOps pipelines, service meshes, and AI/ML model-serving infrastructure to ensure real-time operational visibility and mission-assurance alignment.
• Strong problem-solving and decision-making capabilities, with a proven ability to weigh the relative costs and benefits of potential actions and identify the most appropriate solution.
• Highly developed interpersonal and oral/written communication skills, with the ability to effectively and professionally interact with a diverse set of stakeholders (from peers to end-users to executive management).