Splunk Software Engineer Jobs in California (NOW HIRING)

Responsible for deploying, managing and optimizing. Onboard machine data, build queries using Search Processing Language (SPL) and design dashboards to power IT Enterprise Applications, Security Operations and observability practices. Proactively monitor and report on the environmental health of applications, as well as engage in critical system outages (triage) resolution efficiently to identify, review, analyze, debug and resolve issues with a team of Site Reliability Engineers. Serve as a subject matter expert for the Splunk Platform. .

• Ensure enterprise applications and supporting platforms are fully functional, reliable, available, performant, and secure through frequent monitoring, operational checks, and timely restoration of service in the event of outages, ingestion issues, or system failures.

• Ingest and onboard new operational data sources, including server, application, infrastructure, network, security, API, and system logs; ensure data is reliable, appropriately normalized, accurately timestamped, correctly categorized, and aligned to the Splunk Common Information Model where applicable.

• Monitor and maintain Splunk platform health, including indexers, search heads, data flows, storage utilization, clustered or distributed components, internal logs, capacity trends, and service availability; resolve ingestion bottlenecks, performance issues, and platform outages.

• Create, maintain, and improve real-time Splunk dashboards, visualizations, alerts, reports, metrics, logs, traces, and event views to provide actionable insight into system health, service levels, user experience, operational risk, network activity, security events, and threat indicators.

• Craft, tune, and optimize Splunk Processing Language (SPL) searches, scheduled searches, alert logic, reports, dashboards, and resource-intensive queries to improve performance, reduce noise, support accurate KPI/SLA reporting, and enable analysis across large datasets.

• Configure automated alerts and triggers for anomaly detection, system downtime, performance degradation, ingestion issues, and cyber or operational threats so critical issues are flagged immediately and routed for response.

• Alert and accurately report KPIs on systems status with tuning recommendations at regular intervals; provide detailed analysis using APM, Splunk, and other monitoring or observability solutions.

• Support a 24x7 production environment with a team of experienced engineers, including on-call rotation, deployment support, and timely response to critical incidents as needed.

• Respond quickly to incidents, investigate triggered alerts, isolate performance bottlenecks, fix issues, and work with other engineers to ensure enterprise applications and platforms remain fully functional with a Member-first focus.

• Serve as an escalation point for complex application, infrastructure, observability, security, and platform issues; coordinate resolution across infrastructure engineering, software engineering, software quality assurance, cybersecurity, operations, vendors, and other organizational teams.

• Support incident triage, root cause analysis, post-incident reviews, postmortems, and corrective action planning; identify recurring issues and recommend preventive improvements.

• Develop solutions to meet technical and business requirements; create detailed design documents and associated solutions built around current and new technologies.

• Demonstrate application environment tuning abilities and provide solutions to capacity requirements, including JVM, web containers, database connections, HTTP servers, and related enterprise application components.

• Contribute to automation and efficiency improvements for application and infrastructure buildout components, operational tasks, monitoring coverage, and deployment readiness.

• Ensure effective release management, change management, quality assurance, rollback planning, peer review, and deployment readiness in project lifecycles using an ITIL-centric approach.

• Support production and non-production environments, account for differences in each environment for deployments and testing, and peer review changes within the Enterprise Applications team.

• Maintain and improve configuration documentation, diagrams, SOPs, runbooks, work/incident ticketing systems, knowledge articles, training materials, and other supporting documentation.

• Ensure adherence to configuration, operating, security, compliance, and governance best practices and standards, including approved capacity, licensing, access control, and retention requirements.

• Deploy and troubleshoot applications on both Java and .NET platforms as an expert for infrastructure and development teams.

• Ensure the highest levels of service for Members and Team Members.