Commission Databricks Data Engineer Jobs in Georgia

Job Overview: Pay Range: $50.00hr - $55.00hr Requirement/Must Have: 5+ years building batch and/or streaming pipelines with Spark. Deep understanding of Spark fundamentals including lazy evaluation, transformations vs. actions, partitions, shuffles, joins, and caching. Experience with Databricks platform on Azure including Unity Catalog (catalogs/schemas/tables, permissions, external locations, storage credentials, lineage). Proficiency with Databricks Jobs/Workflows for task orchestration, cluster policies, retries, and parameters. Experience with Git integration (Repos), CI/CD, and environment promotion. Knowledge of cloud storage and access on Azure, specifically secret management via Databricks secret scopes and Key Vault integration. Experience with Delta Lake including ACID transactions, time travel, MERGE/UPDATE/DELETE, and schema enforcement/evolution. Strong SQL skills including complex SQL with CTEs and window functions. Experience with data modeling and architecture using Medallion (bronze/silver/gold) patterns for data lakes. Ability to design tables for CDC and incremental processing. Experience with large scale data processing including cluster sizing, autoscaling, and cost optimization. Familiarity with observability tools such as Spark UI, metrics, event logs, and Delta logs. Knowledge of software engineering practices including Python packaging, code quality, and testing (pytest, chispa). Strong verbal and written communication skills. Responsibilities: Build and maintain batch and streaming data pipelines using Spark and PySpark. Manage Databricks platform components including Unity Catalog and Workflows. Implement data modeling strategies such as partitioning and file layout to match read patterns. Ensure data governance, security, and compliance within the Databricks environment. Execute CI/CD processes for Databricks using Repos and asset bundles. Collaborate with stakeholders and provide technical documentation. Nice to Have: Performance tuning: broadcast joins, AQE, skew mitigation, and shuffle optimization. Structured Streaming and Auto Loader for incremental ingestion. Declarative pipelines with Delta Live Tables (DLT). ADLS Gen2: ABFS access, ACLs vs. RBAC, and managed identity. OPTIMIZE, Z ORDER, VACUUM and retention policies. Compaction and small files mitigation strategies. Query optimization and execution plan analysis in Databricks SQL. Lakehouse design trade-offs and table constraints. Handling TB scale datasets and throughput vs. latency trade-offs. Privilege model in Unity Catalog; table, row, and column level security patterns. PII handling, masking/anonymization, and auditability. Scala, dbt, Kafka/Event Hubs, ADF/Airflow, Terraform, Power BI, data mesh concepts, and SCD patterns. Skills: Spark. PySpark. Azure Databricks. Unity Catalog. Delta Lake. SQL. Python. Azure Data Lake Storage (ADLS). CI/CD. Data Modeling. Java. Qualification And Education: 10-12 years of professional experience in data engineering or related fields. Extensive experience with the Azure cloud ecosystem. Benefits: Our Benefits Include: Medical, Dental, and Vision Insurance 401(k) Retirement Plan Health Savings Account (HSA) Disability Insurance (Short-Term and Long-Term) Life and AD&D Insurance Paid Sick Leave (where required by applicable state or local law) Supplemental Insurance Plans Identity Theft Protection Pet Insurance Employee Wellness Programs Employee Assistance Program (EAP) Career Growth and Professional Development Opportunities Disclaimer: Benefits eligibility, accrual rates, and usage limits may vary based on employment status, length of service, and work location. Paid Sick Leave is provided in strict accordance with applicable state and municipal mandates. Cynet Systems Inc. reserves the right to modify, amend, or terminate any benefit plans at any time in accordance with applicable laws. About Cynet Systems Founded in 2010 and headquartered in the Washington, DC metro area, Cynet Systems Inc. is a leading technology staffing and workforce solutions company serving Fortune 500 companies, government agencies, and enterprise organizations across the United States and Canada. We deliver agile, scalable talent solutions across IT, engineering, life sciences, clinical, and professional staffing, powered by a high-performing recruitment engine operating across North America and Asia. As a nationally and locally certified Minority Business Enterprise (MBE), Cynet Systems is committed to helping organizations build high-performing teams while empowering professionals to grow rewarding careers. Our organization is certified to ISO 9001, ISO 14001, ISO 27001, and SOC 2 Type II standards, reflecting our commitment to quality, security, operational excellence, and customer success.