Job Alert Jobs in California (NOW HIRING)

Job Title: Senior Enterprise Observability & Incident Response Strategist
Role Overview
We are seeking a Senior Observability & Incident Response Strategist to define the standards for how our enterprise systems communicate their health. You will act as the "quality gatekeeper" for our alerting ecosystem, ensuring that every alert is actionable, prioritized, and backed by clear response instructions.
This is not a traditional NOC role; it is a high-impact engineering leadership position. You will sit at the intersection of application development, platform engineering, and service ownership, driving a cultural shift away from "alert fatigue" toward "precision observability." If you are a proponent of SRE principles and have a proven track record of reducing noise while improving system reliability, this role is for you.
Key Responsibilities
Alert Rationalization & Signal Optimization
• Governance Framework: Establish and maintain an enterprise-wide alert rationalization framework. Define the "Definition of Done" for alert onboarding, ensuring signal-to-noise ratios are optimized before any alert reaches an engineer.
• Prioritization: Evaluate alerts based on business criticality, actionability, and service impact. Lead the suppression, aggregation, or correlation of low-value signals.
• Continuous Improvement: Facilitate governance forums with engineering leads to review alert quality, eliminate duplicate signals, and identify systemic reliability gaps.
Observability Standards & Policy
• Standardization: Define the enterprise taxonomy for alerts, including naming conventions, metadata requirements (Service/CI/Owner), and severity thresholds.
• Tooling Integration: Partner with monitoring tool owners (e.g., Datadog, Dynatrace, Splunk, etc.) to embed these standards directly into our monitoring infrastructure.
• Routing Logic: Define the rules of engagement for incident response-determining exactly when an issue requires immediate human intervention versus when it can be handled via asynchronous ticketing or automated dashboards.
Knowledge Management & Response Guidance
• Playbook Lifecycle: Own the runbook/playbook catalog. Ensure every actionable alert is paired with a versioned, tested, and clear set of triage instructions, escalation triggers, and diagnostic links.
• Reliability Coaching: Mentor service owners on observability patterns-such as the "Golden Signals," SLO-based alerting, and dependency mapping-to ensure our response guidance keeps pace with system changes.
Qualifications & Requirements
Minimum Qualifications
• Experience Baseline: Minimum of five (5) years of experience in SRE, Observability, Monitoring Engineering, or large-scale Incident Management.
• Tooling Proficiency: Deep familiarity with modern observability stacks (e.g., Datadog, AppDynamics, Prometheus/Grafana, Splunk, or cloud-native monitoring).
• Operational Literacy: Strong understanding of incident response workflows, service ownership models, and CMDB-based dependency mapping.
• Governance Excellence: Proven success in defining and enforcing enterprise standards across fragmented, cross-functional teams.
Preferred Attributes
• Experience with event correlation and automation (e.g., deduplication/noise suppression engines).
• Background in SRE (Site Reliability Engineering) principles or ITIL Event Management frameworks.
• A "consultative" leadership style-the ability to influence engineering teams to prioritize reliability and observability in their service development lifecycle.
Equal Opportunity Employer / Disabled / Protected Veterans
The Know Your Rights poster is available here:
https://www.eeoc.gov/sites/default/files/2023-06/22-088_EEOC_KnowYourRights6.12.pdf
The pay transparency policy is available here:
https://www.dol.gov/sites/dolgov/files/ofccp/pdf/pay-transp_%20English_formattedESQA508c.pdf
For temporary assignments lasting 13 weeks or longer, AllSTEM Connections is pleased to offer major medical, dental, vision, 401k and any statutory sick pay where required.
We are committed to working with and providing reasonable accommodations to individuals with disabilities. If you need a reasonable accommodation for any part of the employment process, please contact your staffing representative who will reach out to our HR team.
AllSTEM Connections participates in the E-Verify program in certain locations as required by law. Learn more about the E-Verify program.
https://e-verify.uscis.gov/web/media/resourcesContents/E-Verify_Participation_Poster_ES.pdf
We also consider for employment qualified applicants regardless of criminal histories, consistent with legal requirements, including, if applicable, the City of Los Angeles' Fair Chance Initiative for Hiring Ordinance. Pursuant to applicable state and municipal Fair Chance Laws and Ordinances, we will consider for employment-qualified applicants with arrest and conviction records, including, if applicable, the San Francisco Fair Chance Ordinance. For Los Angeles, CA applicants: Qualified applications with arrest or conviction records will be considered for employment in accordance with the Los Angeles County Fair Chance Ordinance for Employers and the California Fair Chance Act.
Additional Skills
(none specified)
AllSTEM Representative Contact Info
Account Executive:
Nichols
Branch Phone:
(909) 244-1777
Location:
Ontario, CA