Senior Databricks Data Engineer Jobs in Toronto, ON

Location

Technology Services

Technology Services (TS) is responsible for delivering all enterprise infrastructure, applications and related end user technology services across all Brookfield business groups.

Brookfield Culture

Brookfield has a unique and dynamic culture. We seek team members who have a long-term focus and whose values align with our Attributes of a Brookfield Leader: Entrepreneurial, Collaborative and Disciplined. Brookfield is committed to the development of our people through challenging work assignments and exposure to diverse businesses.

Job Description

Job Title:Manager, Data Engineer

Location:Toronto

Reports To:Director of Data Analytics

We are looking for a motivatedManager, Data Engineer, to ensure the delivery of innovative data solutions that align seamlessly with our business objectives. In this pivotal role, you will harness your extensive expertise in data engineering, particularly with platforms like Databricks and Snowflake, to spearhead the design, implementation, and optimization of our data models. The ideal candidate will possess profound technical acumen, exceptional communication skills to engage with senior stakeholders, and an unwavering passion for tackling complex data challenges.

What You Will Do:

• Design and implement scalable data models and architectures within cloud data platforms like Databricks or Snowflake to support analytics and business intelligence initiatives.
• Design and implement ETL/ELT processes between various sources into Snowflake and/or Databricks platforms, ensuring high performance and reliability.
• Implement data validation and cleansing processes to ensure data accuracy and integrity.
• Migrate existing data infrastructure to Snowflake, ensuring optimal performance and scalability.
• Assess risks associated with data pipelines and identify potential issues proactively to minimize business impact.
• Identify opportunities using data-driven approach and lead data engineering projects that deliver value to the business.
• Identify areas for optimization within existing data workflows and processes, and devise plan to execute improvements for operational efficiency
• Work closely with business stakeholders to understand requirements and formulate data-driven solutions that address business needs.
• Work with end users or data scientists to develop automated workflows for key reporting processes, and develop governance and controls for underlying data and processes
• Communicate data engineering processes, architecture and technical solutions to business stakeholders and management
• Collaborate with cross-functional teams to ensure alignment between data engineering initiatives and broader technological and business objectives.
• Optimizing database schemas and performance for reading and writing of data and upstream processing in the reporting system
• Working with business end users on maintaining systems to enable ad-hoc or periodic analytics and cashflow runs in production
• Provide technical guidance and support to junior data engineers and business power users.

Qualifications and Skills

• A Bachelor's degree in Computer Science, Information Management, Data Analytics or related discipline.
• 5+ years of hands-on experience in data engineering and optimization with significant exposure to data platforms such as Snowflake, Databricks and/or other data platforms.
• Hands-on experience with data orchestration and workflow management using tools such as Workato or other similar platforms.
• Proven expertise in ETL/ELT, data modeling, data warehousing, data Lakehouse concepts with extensive experience with SQL and Python programming.
• Using best practice data frameworks like Data Centric Development Frameworks or Medalion Architecture when onboarding data, ensuring structures and efficient data integrations processes.
• Strong understanding of RESTful APIs and experience in integrations of various data sources and services.
• Familiarity or experience in source code management and continuous integration practice using Azure DevOps.
• Strong analytical skills, including conceptual, requirements interpretation, solution creation and problem-solving abilities
• Ability to assess and manage risks within data workflows and implement effective mitigation strategies.
• Strong communication skills, with the ability to collaborate effectively with cross-functional teams
• Relevant certification in Snowflake and or Databricks is a plus.
• Familiarity with data governance frameworks and ensuring compliance with data protection regulations
• Familiarity with integrating learning models into data pipelines and understanding their infrastructure requirements
• Familiarity or experience with Investment management business domain, specifically in terms of data modelling across Public and Private asset classes (nice to have)
• Experience in designing and developing interactive data applications using Streamlit and Databricks Apps to improve data accessibility and user experience.
• Implement and support Databricks Genie and platform capabilities, including automation, governance, security, and production support.

Salary Range: $120,000 - $140,000 CAD

Position Opening Reason:

Brookfield is committed to maintaining a Positive Work Environment that is safe and respectful; our shared success depends on it. Accordingly, we do not tolerate workplace discrimination, violence or harassment.