Internship Healthcare Data Engineer Jobs in Indiana

Purpose:

TheDirectorRWD Data Engineering is ahand-ontechnicalrole thatleadsthe end-to-end engineering vision for Lilly's real-world data (RWD) infrastructure.This individualleadsdesignand executesthe scalable, cloud-native pipelines and data products that allow HEOR, SDIA, Statisticians, Medical, and Clinical teams to generate evidence faster, more reproducibly, and at greater scientific depth than is possible through traditional vendor engagements.

This job involves a depth of understanding ofthe multi-modal RWD ecosystemacross Lilly's therapeutic areasto contextualize and drive RWD to the necessary end-user data productsbyleadingthe creation of sophisticated data engineering products, creating processes for automation of data ingestion and product creation, and leading special projects forGlobal Medical Affairs - Health Economics and Outcomes functions and the broader enterprise.Further, this position willbe responsible foridentifyingand advocating standard processes across the data asset lifecycle, working closely with other datadomainand analytics leaders. Collaborating closely with multi-functional teams, you willleadthe technical implementation of data products, ensuring scalability, reliability, and performance. The ideal candidatepossessesdeepexpertisein data engineering, strong problem-solving skills, and a passion forleveragingdata to drive business outcomes.

This positionreportsto HEOR Central andis embedded within the BIA organization and works in close partnership with HEOR, SDIA, Statisticians, Medical, and Clinical teams.

Responsibilities:This job description is intended to provide a general overview of the job requirements at the time it was prepared. The job requirements of any role/position can change over time and can includeadditionalresponsibilities not specifically described in the job description. Consult with your supervisorregardingyour actual job responsibilities and any related duties that might berequiredfor the role/position.

• Lead the design, development, and implementation of cloud-native data products and high-throughput data pipelines that transform raw real-world data into scalable, reliable, analysis-ready assets supporting analytics, reporting, and evidence generation.
• Lead the Analytic Data Products Strategy to deliver key data assets that enable streamlined, compliant execution and analytics.
• Own the end-to-end lifecycle of RWD data products, from requirements gathering and prototyping through production deployment and optimization, ensuring scalability, reliability, performance, and reproducibility across cloud environments (e.g., Databricks, AWS S3, Azure Data Lake).
• Build,optimize, and maintain ETL/ELT ingestion and transformation pipelines for large-scale, multi-modal RWD - including claims, complex EHR data, and other linked healthcare datasets - handling data volumes ranging from tens of millions to billions of records.
• Implement and managelakehouse-style data architectures (e.g., medallion bronze/silver/gold patterns) using Databricks and cloud object storage (AWS S3, ADLS) to produce versioned, partitioned, and audit-ready data assets.
• Write andmaintainreusable, version-controlled transformation logic incorporating healthcare coding and terminology standards (e.g., ICD-10/ICD-9, NDC,RxNorm, SNOMED, CPT/HCPCS, LOINC) to produce domain-level datasets such as demographics, diagnoses, treatments, procedures, encounters, and labs.
• OptimizeSQL and distributed processing workloads (e.g., Spark-based jobs) for performance acrossvery largedatasets, applying partitioning, indexing, predicate pushdown, denormalization, and other optimization strategiesappropriate toanalytical workloads.
• Translate analytic, business, and research requirements into reproducible data extraction and transformation logic, supporting cohort construction, temporal logic, and consistent reuse of RWD across teams.
• Apply deep understanding of healthcare data structures and standards when engineering data products, ensuring datasets are fit for purpose for downstream analytics and compliant with scientific, regulatory, and audit expectations.
• Establish and implement standard engineering practices andmethodologyacross the data asset lifecycle, including automated data ingestion, data quality checks, integrity testing, validation, monitoring, alerting, and documentation from source table to analysis-ready output.
• LeadCI/CD pipeline setup, code review, andtestingstandards, ensuring all transformation code is version-controlled, tested, and deployable in a reproducible manner.
• Collaborate closely with multi-functional partners - data scientists, statisticians, analytics leaders, and other technical teams - to understand business and technical requirements and develop documentation of RWD engineering standards, transformation templates, code list repositories, and pipeline performance guidelines.
• Provide technical consultation to collaborators onappropriate useof data products and underlying RWD assets, including structural limitations of specific data sources, join strategies, and performance considerations; develop source-specific training materials for HEOR scientists, SDIA, and statisticians.
• Develop and implement KPIs to measure system performance,efficiency andpull through to program impact.
• Createan inclusive culturewhereproducing andmaintaininghigh-quality data is a core discipline.

Technical Skills:

Core Engineering Skills

• Highproficiencyin SQL optimization across cloud platforms - complex joins, window functions, query tuning, workload management - on AWS Redshift, Databricks SQL, Snowflake, orBigQuery.
• Python fluency: pandas,PySpark, Polars for large-scale data manipulation; workflow orchestration with Apache Airflow, Prefect, orDagsterfor production pipeline scheduling and monitoring - including Databricks Workflows for orchestrating multi-task jobs within the Lakehouse.
• Distributed computing: Apache Spark (PySpark),Dask, or Ray - ability to write, tune, and debug distributed jobs processing multi-terabyte datasets across partitioned cloud storage, including Databricks clusters with auto-scaling and spot instance optimization.
• Cloud data engineering: hands-on pipeline development on AWS (S3, Glue, Redshift, EMR), Azure (ADLS, Synapse, ADF), or GCP (BigQuery,Dataflow) -as well as Databricks on any major cloud (AWS, Azure, or GCP) using Unity Catalog for cross-workspace governance - not just configuration.
• Delta Lake / Apache Iceberg: time travel, schema evolution,upsert/merge operations, partition optimization - building versioned, ACID-compliant data assets at scale; Delta Lake experience ideally hands-on within the Databricks Lakehouse platform using Delta Live Tables (DLT) for declarative pipeline authoring.
• ETL/ELT tooling: AWS Glue,dbt,dbt-databricks, or equivalent for building tested, documented transformation pipelines with automated data quality checks; familiarity with Databricks Asset Bundles (DABs) for packaging and deploying notebooks, jobs, and DLT pipelines as code.
• DevOps fluency: Git, CI/CD pipelines (GitHub Actions, Azure DevOps), Docker -maintainingall pipeline code as version-controlled, testable, and deployable artifacts; experience deploying to Databricks via CLI, REST API, or Terraformprovidera plus.
• Strong fluency with data modeling: designing star/snowflake schemas, OMOP-compliant structures, and flat domain long filesoptimizedfor analytical workloads in a research context - including materializing these structures as managed Delta tables within Databricks Unity Catalog withappropriate accesscontrols and lineage tracking.
• R fluencya plusfor collaboration with statistical and HEOR teams on dataset validation and specification.

Healthcare RWD -reviews and EHR

• Medical and pharmacy claims: deep working knowledge of CCAE, OptumClinformatics, IQVIAPharMetrics, Truveta, and Komodo data structures - enrollment/eligibility tables, revenue codes, place-of-service codes, inpatient vs. outpatient claim splitting, drug identification at NDC and GPI level, and known structural quirks of each source.
• Complex EHR data: HL7 FHIR resources, Epic/Cerner/Truveta data models, clinical note schemas, problem list hierarchies, medication order and administration tables, lab result normalization, and vital sign time series - including semi-structured and nested JSON/XML from EHR exports and FHIR APIs.
• Healthcare terminology and ontology mapping: ICD-10-CM/PCS, ICD-9-CM, NDC,RxNorm, SNOMED CT, CPT-4, HCPCS, LOINC, ATC -building, versioning, and governing code list repositories joined to raw tables toproduce concept-labeledanalysis-ready datasets.
• OMOP CDM: transforming source claims and EHR data to OMOP v5.x including vocabulary loading (Athena), ETL specification documentation, and Achilles/DQD data quality checks.
• Phenotyping and cohort construction: translating clinical study protocols into reproducible extraction logic - index date derivation, washout periods, time-varying covariates, censoring - suitable for pharmacoepidemiology and HEOR studies.
• Regulatory and privacy frameworks: HIPAA-compliant data handling, de-identification standards (Safe Harbor, Expert Determination), DUA compliance, and audit trail requirements for FDA-grade RWE submissions.

AI Fluency

• Deploy NLP pipelines for structured extraction from unstructured clinical notes - operationalizing pre-trained biomedical language models (BioBERT,ClinicalBERT) for outcome ascertainment and phenotyping within the data pipeline.
• Implement AI-powered data profiling and anomaly detection to automate quality checks across large ingestion runs and surface issues before they reach analytical teams.
• Use LLM-assisted SQL generation and code review tools to accelerate pipeline development and reduce query errors at scale.
• Apply intelligent caching, query result reuse, and automated feature engineering tooptimizecompute costs and prepare multi-modal variables for downstream ML inputs.
• Familiarity withMLOpstooling (MLflow, SageMaker, Azure ML) for versioning and monitoring AI models integrated with data pipelines.

Minimum Qualification Requirements:

• Bachelor's degree in Computer Science, Engineering, Statistics, Information Technology, Bioinformatics or Technical Field.
• Minimum of 5 years of hands-on data engineering experience.
• Minimum of 5 years of applied expertise across Python,SQLJava, Spring, Spring Boot, Prefect, and/or other business intelligence tools, ,ETL/ELT pipelines, and cloud platforms (AWS Glue/EMR, Snowflake, or Databricks),-applied directly to real-world healthcare dataatscale.

• Minimum of 3 years of direct people management experience, including leading, coaching, and developing team members.

Additional Preferences:

• Master's degree in Computer Science, Engineering, Statistics, Information Technology, Bioinformatics.
• Minimum of 5 years of hands-on data engineering experience. with a focus on healthcare or life sciences RWD Preferred.
• Experience with distributed computing frameworks (Spark,Dask) for large-scale RWD processing.
• Advanced SQL optimization skills for AWS Redshift and/or S3/Databricks-based architectures, including query tuning and workload management.
• Deep understanding of healthcare coding standards (ICD-10, NDC,RxNorm, SNOMED CT, CPT, LOINC), real-world data structures, and major RWD vendors and platforms (Truveta, Optum, IQVIA, Komodo,HealthVerity).
• Familiarity with DevOps and CI/CD practices relevant to data pipeline development and deployment.
• Strong problem-solving skills, attention to detail, and ability to work effectively in matrixed, cross-functional teams with both technical and scientific partners.
• Demonstrated ability to build production-grade ingestion pipelines for multi-terabyte, multi-modal datasets: claims, complexher.
• Experience with OMOP CDM transformation, vocabulary management (Athena), and federated analytics networks (OHDSI,PCORnet, Sentinel).
• Familiarity with bioinformatics workflow managers (Snakemake,Nextflow, WDL) and genomics cloud platforms (DNAnexus, Terra, AWS Genomics CLI).
• Experience with data pipeline observability and quality tooling (dbt, Great Expectations, Monte Carlo).
• Familiarity with FDA RWE Framework guidance, EMA RWD guidance, and audit-readiness requirements for observational studies.
• Knowledge of Agile/Scrum methodologies and project management tools such as JIRA
• Knowledge and experience with -omics RWD:Genomics(e.g., ingesting and processing VCF/GVCF files, PLINK binary formats (bed/bim/fam),Transcriptomics(e.g.,bulk RNA-seq count matrices (featureCounts, STAR/RSEM), single-cell and single-nucleus RNA-seq (AnnData/h5ad, 10x GenomicsCellRanger) - ingestion, normalization, and linkage to clinical phenotype data;Proteomics(e.g.,DIA/DDA mass spectrometry output (MaxQuant, DIA-NN) and OLINK).
• Recruitment open across sectors: pharma/biotech data teams, RWD vendors (Truveta, Komodo, Optum, IQVIA,HealthVerity), cloud providers (AWS, Google, Microsoft), genomics platforms (Illumina,DNAnexus, Broad Institute), or academic health systems with large-scale RWD programs.

Other Information

• Location: Indianapolis, IN is strongly preferred. Relocation assistance is available for qualified candidates. Remote work may be considered based on business needs and candidate qualifications.