Data Engineer Developer Jobs in Jacksonville, FL

-
MSP Owner: Rob Finton
Location: Boston, MA - Hybrid (as needed) - The customer is seeking local candidates only (in and around Boston) to ensure the consultant can visit the office when required.
Duration: 6 months
skill id: 10746966
Competencies: 10+ (Ideally 15+) years experience required
Digital : Snowflake
The customer is seeking local candidates only (in and around Boston) to ensure the consultant can visit the office when required.
Job Title: Senior Data Engineer / Data Architect (Databricks & Snowflake)
Job Summary
We are seeking a highly experienced Senior Data Engineer / Data Architect with deep expertise in Databricks, Snowflake, and Azure cloud data platforms. The ideal candidate will have extensive experience designing and implementing scalable data pipelines, Lakehouse architectures, and real-time data processing solutions, particularly in regulated domains such as Life Sciences or Healthcare.
This role requires strong proficiency in Spark (PySpark), Delta Lake, Medallion architecture, and cloud-native data engineering practices, along with a solid background in data warehouse modernization and performance optimization.
Key Responsibilities
• Design and implement end-to-end data engineering pipelines using Azure Databricks, ADLS Gen2, and Snowflake.
• Develop scalable ETL/ELT pipelines using PySpark, Spark SQL, Python, and Talend.
• Build and maintain Lakehouse architecture using Delta Lake and Medallion (Bronze, Silver, Gold) layers.
• Implement real-time and batch data ingestion pipelines, including streaming using Spark Structured Streaming.
• Design and enforce data governance, access control, and lineage using Unity Catalog.
• Optimize Spark workloads through partitioning, caching, broadcast joins, and cluster tuning to improve performance and reduce cloud costs.
• Architect and manage CI/CD pipelines using Azure DevOps, Jenkins, and Git for automated deployments.
• Integrate multiple data sources and systems, ensuring high-quality, reliable, and scalable data delivery.
• Collaborate with cross-functional teams including data analysts, scientists, and business stakeholders to support analytics and reporting needs.
• Support data warehouse modernization initiatives, including migration from legacy systems to cloud platforms.
Required Qualifications
• 10+ years (ideally 15-20+) of experience in data engineering or data architecture.
• Strong expertise in:
o Databricks & Delta Lake
o Snowflake Data Warehouse
o Apache Spark (PySpark, Spark SQL)
• Hands-on experience with Azure Cloud (ADLS Gen2, Azure Databricks, ADF).
• Proficiency in Python and SQL for data engineering.
• Experience with ETL/ELT tools such as Talend or Informatica.
• Strong knowledge of data modeling, CDC (Change Data Capture), and incremental loading techniques.
• Experience working in Linux/Unix environments with shell scripting.
Preferred Qualifications
• Knowledge of data governance, compliance, and regulatory standards (e.g., IDMP).
• Exposure to real-time data streaming technologies (Kafka, Kinesis).
• Experience with multi-cloud environments (AWS, GCP).
• Familiarity with workflow orchestration tools such as Airflow or Databricks Workflows.
Key Skills
• Data Engineering & Architecture
• Lakehouse & Data Warehousing
• Spark Performance Optimization
• Cloud Data Platforms (Azure)
• ETL/ELT Pipeline Development
• Data Governance & Security
• CI/CD & DevOps Practices
Education
• Bachelor's degree in Information Technology, Computer Science, or a related field.
Nice-to-Have Traits
• Strong analytical and problem-solving skills
• Ability to work in enterprise-scale, complex environments
• Experience working with global stakeholders and cross-functional teams
• Leadership capability with mentoring experience, Project Code :