Position : Databrick ArchitectLocation : RemoteTerm : C2C/W2 roleDuration : 12+ monthsJob Description:We are seeking an experienced and highly skilled Databricks Architect to design, develop, and optimize scalable data platforms and cloud-based analytics solutions. The ideal candidate will have strong expertise in Databricks, Spark, cloud technologies, data engineering, and modern data architecture practices. This role involves collaborating with business stakeholders, data engineers, analysts, and architects to deliver robust big data and AI/ML solutions.
Key Responsibilities - Design and implement scalable data architectures using Databricks and Apache Spark.
- Develop and optimize ETL/ELT pipelines for structured and unstructured data.
- Architect cloud-based data solutions on AWS, Azure, or GCP.
- Build and manage Lakehouse architecture using Databricks.
- Integrate multiple data sources including APIs, databases, streaming, and cloud storage.
- Optimize Spark jobs and Databricks workloads for performance and cost efficiency.
- Implement data governance, security, data quality, and compliance standards.
- Collaborate with data scientists and analytics teams for AI/ML workloads.
- Design batch and real-time data processing solutions.
- Lead technical discussions, architecture reviews, and best practice implementations.
- Mentor junior engineers and provide technical leadership.
- Create architecture documentation, technical diagrams, and solution designs.
- Work closely with DevOps teams for CI/CD and deployment automation.
- Troubleshoot production issues and provide long-term scalable solutions.
Required SkillsMust Have Skills - Strong hands-on experience with Databricks
- Expertise in Apache Spark (PySpark / Scala Spark)
- Experience with Delta Lake and Lakehouse architecture
- Strong SQL and data modeling skills
- Experience with cloud platforms:
- Knowledge of data warehousing concepts
- Experience building ETL/ELT pipelines
- Hands-on experience with:
- Experience with data orchestration tools:
- Airflow
- Azure Data Factory
- AWS Glue
- Knowledge of performance tuning and optimization
- Understanding of CI/CD and DevOps practices
- Experience with Git/version control systems
Preferred Skills - Experience with streaming technologies:
- Knowledge of AI/ML integration on Databricks
- Experience with Snowflake or Redshift
- Infrastructure as Code experience:
- Familiarity with containerization:
- Experience with Unity Catalog and data governance
- Certification in Databricks or Cloud platforms
Qualifications - Bachelor's or Master's degree in Computer Science, Information Technology, or related field.
- 10+ years of overall IT experience.
- 5+ years of experience in Big Data/Data Engineering.
- 3+ years of hands-on Databricks architecture experience.
Nice to Have - Databricks Certified Professional certification
- Experience in Healthcare, Banking, Retail, or Telecom domains
- Exposure to Generative AI/Data AI solutions
- Experience with Medallion Architecture