Role: Databricks Engineer (with Validation/Testing experience)
Location:ย 100% remote (east coast hours)
Duration:ย 3-year project
ย
Notes/Job Description:
The team is seeing many resumes thatย mention Databricks superficially, but they are specifically looking for aย true Databricks expertโsomeone who knows the platformย endโtoโend, not just an analyst or light user.
This is aย handsโon engineering role, not an analytics position. The ideal candidate should be comfortableย building custom utilities, writingย Python and PySpark code, and troubleshooting complex data movement and validation issues independently.
Key Requirements
- Deep Databricks expertiseย (developerโlevel), not just exposure
- Strong handsโon experience withย Databricks Medallion Architecture
- Bronze / Silver / Gold (also referred to as Raw / Silver / Mark layers internally)
- Experience validating and testing data as it moves across layers
- Experience ingesting data fromย 10+ source systemsย into data lakes
- Very strong withย Databricks notebooks,ย Unity Catalog,ย Python, andย PySpark
- Ability toย test and report on curated data layersย and clearly explain QA/testing approaches
- Experience withย CI/CD in Databricks, including promotion of code vs. developer workflows
- Familiarity withย Playwright and TypeScriptย for testing is nice to have
- Certifications are aย plus, but not mandatory
Team & Process
- Role is embedded within theย Enterprise Data & Analytics Platform team
- Must align with establishedย QA and governance processes
Overall Takeawayย They want aย Databricksโfirst engineerย who hasย designed, built, tested, and operatedDatabricks pipelines in complex, multiโsource enterprise environmentsโnot someone with Databricks listed as a secondary skill.