Title –Databricks Lead
Location – Chester, Newtown Square PA
Duration – Contract/FTE
The Role offers
An outstanding opportunity to imagine, redesign, and apply technology to add value to the business and operations.
An end-to-end project exposure across multiple technical stack and cloud platform
An individual who has passion to learn and adapt to new technologies quickly and scale to next level easily.
High visibility, opportunity to interact with multiple groups within the organization, technology vendors and implementation partners.
Ability to work in the emerging cloud platform like AWS, Azure or GCP.
Essential Skills
• 12+ years' experience in any of the cloud data integration tools like Azure ADF,AWS Glue, Cloud agnostic: Databricks, Cloud agnostic: Talend, Data Flow.
• 12+ years strong experience in ETL tools like Informatica, SSIS, SQL, Azure ADF,AWS Glue, Cloud agnostic: Databricks, Cloud agnostic: Snowflake, IBM, DataStage, BODS.
• Minimum of 5+ years in performance fine tuning and optimization of Cloud data integration process.
• Experience cloud/on-prem devops tools for orchestration, scheduling, logging, required for data integration development.
• Strong understanding of data modeling, data warehousing, and ETL processes.
• Strong analytical and problem-solving skills.
• Excellent communication and collaboration skills.
• Handling various types of data sources and it best practice solutions.
• Solving complex business problems at large scale and Understanding business needs and fulfilling them with the right solution.
• Participation in a cross-functional initiatives and collaboration across various domains.
• Proven ability to manage multiple complex projects and deliver high-quality results.
• Strong attention to detail and a passion for creating high-quality data solutions.
• Demonstrated experience in mentoring and developing technical talent.
• Understanding the importance of governance and security rules when handling data.
• Excellent presentation skills.
Responsibilities
• Develop and execute a strategic vision for cloud-based data integration, aligning with the company’s data and business objectives.
• Architect, design, and implement scalable data integration frameworks and solutions using tools such as Azure ADF,AWS Glue, Cloud agnostic: Databricks, Cloud agnostic: Talend, Data Flow, etc.
• Lead the evaluation, selection, and integration of data integration tools and technologies to meet organizational needs.
• Collaborate with data scientists, analysts, and business stakeholders to gather requirements and deliver high-impact data solutions.
• Establish and enforce best practices for data integration, ensuring clarity, accuracy, and re-usability.
• Optimize performance and scalability of data integration solutions by leveraging cloud resources effectively.
• Mentor and provide technical leadership to data engineers and analysts.
• Develop and maintain comprehensive documentation for data integration architecture, processes, standards, and best practices.
• Stay ahead of the latest trends and advancements in data integration and cloud technologies and evaluate their potential application within the organization.
• Drive continuous improvement initiatives to enhance the quality and efficiency of data integration processes.
• Ensure seamless integration of data integration solutions into various business applications and platforms.
• Participate in code reviews, providing constructive feedback to enhance the quality of the team’s work.
• Work with cross-functional teams to integrate data integration seamlessly into various applications and platforms.
• Lead projects and initiatives, ensuring timely delivery of high-quality data solutions
Essential Qualification
• BE/MCA/equivalent master’s in computer Science, Information Technology or related filed is a must.
• Must be Databricks certified.