Cortex Jobs in Florida (NOW HIRING)

Job Summary:
WatchGuard Technologies is a global leader in network security and intelligence. They are seeking a Senior Data Engineer to join their data platform team, where you will design, build, and maintain a cloud-native data lakehouse, ensuring data quality and reliability while collaborating with various stakeholders.
Responsibilities:
• Design, build, and maintain scalable ETL/ELT pipelines using Azure Data Factory (ADF) and Apache Airflow, processing structured and semi-structured data across the Medallion architecture (Bronze → Silver → Gold).
• Implement incremental load patterns, change data capture (CDC), and event-driven ingestion to ensure data freshness across the platform.
• Build and optimise Snowflake data warehouse objects — tables, views, dynamic tables, streams, tasks, and stored procedures — for performance and cost efficiency.
• Develop modular, tested dbt models aligned to each Medallion layer, enforcing consistent naming conventions, documentation, and lineage across all transformations.
• Embed automated data validation at every Medallion layer using Elementary (dbt's observability layer), ensuring anomaly detection, freshness checks, and schema drift alerts are in place before data reaches consumers.
• Define and enforce data contracts between producers and consumers — row count checks, null rate thresholds, referential integrity, and value domain validation.
• Build and maintain data quality dashboards to give engineering and business stakeholders real-time confidence in platform health.
• Manage and optimise Azure Data Lake Storage Gen2 (ADLS) — folder structures, lifecycle policies, access tiers, and partition strategies.
• Build and maintain Azure Functions and Azure Logic Apps for lightweight event-driven processing, orchestration triggers, and operational automation.
• Manage secrets, credentials, and environment-specific configuration securely using Azure Key Vault — no hardcoded credentials in pipelines or code.
• Contribute to infrastructure-as-code practices for provisioning Azure data services (Terraform or Bicep preferred).
• Translate ambiguous business requirements into well-defined data models and pipeline designs, working with analysts and stakeholders to validate assumptions before build.
• Participate in code reviews, enforce standards, and mentor junior engineers on data engineering best practices.
• Support CI/CD adoption for pipeline and dbt model deployment across Dev / Test / Prod environments.
Qualifications:
Required:
• 4+ years of professional data engineering experience with at least 2 years on Azure cloud data platforms.
• Snowflake: Advanced SQL — window functions, CTEs, recursive queries, query profiling.
• Snowflake-native features: streams, tasks, snowpipe, dynamic tables, row-level security.
• Virtual warehouse tuning and credit cost optimisation.
• dbt + Elementary: Writing, testing, and documenting production dbt models.
• Elementary integration for data observability and anomaly detection.
• dbt incremental strategies, snapshots, and semantic layer.
• Azure Cloud: Azure Data Factory — pipeline authoring, triggers, parameterisation, linked services.
• ADLS Gen2 — zone/folder design, lifecycle management, Parquet/Delta partitioning.
• Azure Key Vault — secret management, managed identities.
• Azure Functions / Logic Apps — event-driven triggers and lightweight automation.
• Airflow: DAG authoring, task dependencies, XCom, sensors, and connection management.
• Airflow deployment and monitoring in cloud-hosted environments.
• Python: Data pipeline scripting, PySpark basics, REST API integration.
• Unit testing pipeline logic and transformation functions.
• Data Quality & Medallion Architecture: Hands-on experience implementing Bronze / Silver / Gold Medallion architecture.
• Data validation checks at each layer — not just at the final Gold layer.
• Schema evolution handling and SCD Type 2 dimension management.
Preferred:
• Exposure to Snowflake Cortex, dbt Semantic Layer, or Boomi Data Hub for AI-assisted data enrichment within pipeline layers.
• Experience integrating LLM-based quality checks or AI-assisted anomaly detection into data workflows.
• Familiarity with Microsoft Fabric and OneLake as a complementary or future-state platform.
• Knowledge of data mesh or data product thinking and how it maps to Medallion layer ownership.
• Experience with Terraform or Bicep for Azure infrastructure provisioning.
Company:
WatchGuard® Technologies, Inc. is a global leader in unified cybersecurity. Founded in 1996, the company is headquartered in Seattle, USA, with a team of 1001-5000 employees. The company is currently Late Stage.