Senior Data Pipeline Engineer Jobs (NOW HIRING)

Anticipated End Date:
2026-08-21
Position Title:
Senior Data Engineer
Job Description:
Senior Data Engineer
Hybrid 1: This role requires associates to be in-office 1 - 2 days per week, fostering collaboration and connectivity, while providing flexibility to support productivity and work-life balance. This approach combines structured office engagement with the autonomy of virtual work, promoting a dynamic and adaptable workplace. Alternate locations may be considered if candidates reside within a commuting distance from an office.
Please note that per our policy on hybrid/virtual work, candidates not within a reasonable commuting distance from the posting location(s) will not be considered for employment, unless accommodation is granted as required by law.
The Senior Data Engineer serves as the technical lead and platform owner for the security data lake supporting Application Security, Vulnerability Management, and related cybersecurity functions. This role defines the platform architecture, engineering standards, data models, integration patterns, governance controls, and technical roadmap necessary to deliver reliable and actionable security data. The role is also responsible for the design and delivery of secure, scalable, and governed data ingestion, transformation, enrichment, storage, and consumption capabilities across Microsoft Fabric and supporting Azure services.
How you will make an impact

• Leads the architecture, engineering, and ongoing operation of the security data lake and associated data products supporting multiple security technologies and data sources. Establishes the technical roadmap, engineering standards, reusable design patterns, and platform operating model.
• Acts as a subject matter expert among peers, with manager and senior management on all matters pertaining to data platforms and the supporting architecture.
• Develop and maintain the end-to-end architecture of the security data lake, including ingestion, storage, transformation, enrichment, semantic, and consumption layers, based on business, security, compliance, scalability, and reliability requirements.
• Ensure data pipelines comply with security policies and regulatory requirements (e.g., HIPAA, NIST, SOC 2)
• Develop testing plans to ensure quality of implementation, including deployment validation, data quality checks, reconciliation, and repeatable testing of data pipeline changes.
• Determine and perform complex configuration changes to meet business and information security requirements, including Microsoft Fabric workspaces, data pipelines, connections, service principals, secrets, and environment-specific configuration.
• Provide trouble resolution on complex problems and lead implementations for data pipelines, Microsoft Fabric, Azure DevOps, REST API, and source-system integration issues.
• Provide technical support to business and technology associates, in risk assessments and implementation of appropriate information security procedures, standards and technologies.
• Represent major upgrades and reconfigurations in change control; design and analyze vendor and platform services meeting business and information security requirements, including governed promotion across development, test, and production environments.
• Represent infrastructure security support in significant projects and perform the most complex operations and administration tasks.
• Respond to level 3 & 4 change and problem requests without supervision;
• Lead level 1 & 2 incident recoveries and root cause analysis.
• Coordinate and prepare the reporting of data security events and incidents.
• Perform capacity analysis; recommend and implement capacity increases for data platform performance, scalability, and reliability.
• Maintains relationship with key vendors;
• Serve as the technical escalation for results of preventative maintenance routines;
• Supervise preventative maintenance;
• Creates presentations and seeks IT management approval and acceptance of significant replacements or reconfigurations of the data pipeline.

Minimum Requirements:
Requires BS/BA in information Technology or related field of study and a minimum of 8 years experience in systems administration and security aspects of information systems, access management and network security technologies, network communications, computer networking, telecommunications, systems development and management, hardware, software, data, and people; experience with multiple technical and business disciplines required; or any combination of education and experience, which would provide an equivalent background.
Preferred Skills, Capabilities and Experiences:

• Experience with architecting and operating Microsoft Fabric, Apache Spark, or comparable enterprise lakehouse platforms
• Experience in the design, development, testing, deployment and maintenance of scalable data pipelines for structured and unstructured data.
• Experience with SQL, Python, PySpark, Spark, Delta Lake, Parquet, REST APIs, JSON, Git, Azure DevOps, automated testing, and CI/CD is strongly preferred.
• Relevant Microsoft Fabric, Azure Data, Databricks, cloud architecture, or data-engineering certifications are preferred. Security certifications such as CISSP, ISSAP, ISSEP, or equivalent certifications are also preferred.
• Experience planning and designing highly complex systems is strongly preferred.

For candidates working in person or virtually in the below locations, the salary* range for this specific position is $127,200 to $228,960
Location(s): California, Illinois, Massachusetts, New Jersey, New York, Virginia, Washington State
In addition to your salary, Elevance Health offers benefits such as a comprehensive benefits package, incentive and recognition programs, equity stock purchase and 401k contribution (all benefits are subject to eligibility requirements). The salary offered for this specific position is based on a number of legitimate, non-discriminatory factors set by the Company. The Company is fully committed to ensuring equal pay opportunities for equal work regardless of gender, race, or any other category protected by federal, state, and local pay equity laws.
* The salary range is the range Elevance Health in good faith believes is the range of possible compensation for this role at the time of this posting. This range may be modified in the future and actual compensation may vary from posting based on geographic location, work experience, education, and/or skill level. Even within the range, the actual compensation will vary depending on the above factors as well as market/business considerations. No amount is wages or compensation until such amount is earned, vested, and determinable under the terms and conditions of the applicable policies and plans. The amount and availability of any bonus, commission, benefits, or any other form of compensation and benefits that are allocable to a particular employee remains in the Company's sole discretion unless and until paid and may be modified at the Company's sole discretion, consistent with the law.
Job Level:
Non-Management Exempt
Workshift:
1st Shift (United States of America)
Job Family:
IFT > IT Security & Compliance
Please be advised that Elevance Health only accepts resumes for compensation from agencies that have a signed agreement with Elevance Health. Any unsolicited resumes, including those submitted to hiring managers, are deemed to be the property of Elevance Health.
Who We Are
Elevance Health is a health company dedicated to improving lives and communities - and making healthcare simpler. We are a Fortune 25 company with a longstanding history in the healthcare industry, looking for leaders at all levels of the organization who are passionate about making an impact on our members and the communities we serve.
How We Work
At Elevance Health, we are creating a culture that is designed to advance our strategy but will also lead to personal and professional growth for our associates. Our values and behaviors are the root of our culture. They are how we achieve our strategy, power our business outcomes and drive our shared success - for our consumers, our associates, our communities and our business.
We offer a range of market-competitive total rewards that include merit increases, paid holidays, Paid Time Off, and incentive bonus programs (unless covered by a collective bargaining agreement), medical, dental, vision, short and long term disability benefits, 401(k) +match, stock purchase plan, life insurance, wellness programs and financial education resources, to name a few.
Elevance Health operates in a Hybrid Workforce Strategy. Unless specified as primarily virtual by the hiring manager, associates are required to work at an Elevance Health location at least once per week, and potentially several times per week. Specific requirements and expectations for time onsite will be discussed as part of the hiring process.
The health of our associates and communities is a top priority for Elevance Health. We require all new candidates in certain patient/member-facing roles to become vaccinated against COVID-19 and Influenza. If you are not vaccinated, your offer will be rescinded unless you provide an acceptable explanation. Elevance Health will also follow all relevant federal, state and local laws.
Elevance Health is an Equal Employment Opportunity employer, and all qualified applicants will receive consideration for employment without regard to age, citizenship status, color, creed, disability, ethnicity, genetic information, gender (including gender identity and gender expression), marital status, national origin, race, religion, sex, sexual orientation, veteran status or any other status or condition protected by applicable federal, state, or local laws. Applicants who require accommodation to participate in the job application process should submit the following form: Accessibility Accommodation Request Form and a member of the team will be in contact. Qualified applicants with arrest or conviction records will be considered for employment in accordance with all federal, state, and local laws, including, but not limited to, the Los Angeles County Fair Chance Ordinance and the California Fair Chance Act.
Prospective employees required to be screened under Florida law should review the education and awareness resources at HB531 | Florida Agency for Health Care Administration.
NOTE: Workday keeps job postings active through 11:59:59 PM on the day before the listed end date. Example: If the end date is 3/13, the posting will automatically come down on 3/12 at 11:59:59 PM. In other words - the job is posted until 3/13, not through 3/13.