Location: North York, Ontario, Canada
Work Model: Hybrid, 3 days per week onsite
Job Type: Subcontractor, Initial Short-Term Contract with Potential Extension
Rate: Up to CAD $60-70/hour, All-Inclusive
Experience: 8–10+ Years
Industry: Insurance
Position Overview
We are looking for a highly experienced Senior Databricks Data Engineer with strong hands-on expertise in Databricks, AWS and PySpark to support an enterprise data modernization initiative within the insurance domain.
The ideal candidate will have 8–10+ years of overall Data Engineering/Data & Analytics experience, including significant hands-on experience building and delivering scalable data engineering solutions using Databricks on AWS.
This is a hands-on engineering position rather than a purely architectural role. The successful candidate should be comfortable designing solutions as well as developing, troubleshooting and delivering production-grade data pipelines.
Candidates must be located within commuting distance of North York, Ontario and available to work onsite 3 days per week from Day 1.
Key Responsibilities
- Design, develop and maintain scalable data engineering solutions using Databricks on AWS.
- Build and manage complex data transformation and ETL/ELT pipelines within Databricks.
- Develop large-scale data processing solutions using Apache Spark and PySpark.
- Work with AWS data services including Amazon S3, Redshift and Lake Formation.
- Design and implement solutions supporting Data Lake, Data Warehouse and Lakehouse architectures.
- Integrate structured, semi-structured and unstructured data from multiple enterprise sources.
- Develop reliable, reusable and performance-optimized data pipelines.
- Work with Delta Lake for scalable and reliable data management.
- Support data governance, security, metadata management and access controls using tools such as Databricks Unity Catalog.
- Troubleshoot data pipeline, performance and integration issues across Databricks and AWS environments.
- Implement appropriate data quality, monitoring and operational controls.
- Participate in solution design and technical discussions with architects, engineering teams and business stakeholders.
- Translate business and technical requirements into scalable data engineering solutions.
- Support deployment automation and CI/CD processes for Databricks workloads.
- Contribute to technical standards, development best practices and engineering documentation.
- Work collaboratively with cross-functional teams to deliver solutions from requirements through production implementation.
Mandatory Skills & Experience
- 8–10+ years of overall experience in Data Engineering, Data & Analytics or related technologies.
- Strong hands-on Databricks Data Engineering experience.
- Experience delivering at least one end-to-end Databricks implementation.
- Strong experience developing data transformation and ETL/ELT pipelines in Databricks.
- Strong hands-on experience with Apache Spark and PySpark.
- Strong understanding of the AWS cloud ecosystem.
- Hands-on experience with Amazon S3 and other AWS data services.
- Experience working with enterprise-scale Data Lakes, Data Warehouses and/or Lakehouse architectures.
- Strong SQL and data transformation skills.
- Strong understanding of data modeling and data engineering best practices.
- Demonstrated hands-on implementation and production delivery experience.
- Strong troubleshooting and problem-solving capabilities.
- Ability to work independently and take ownership of technical deliverables.
- Strong communication skills and ability to collaborate with technical and business stakeholders.
Preferred Skills
Experience with any of the following will be considered an asset:
- Databricks Unity Catalog
- Delta Lake
- AWS Redshift
- AWS Lake Formation
- Databricks Workflows
- CI/CD and DevOps for Databricks
- Data governance and data quality frameworks
- AWS cloud security and governance
- AI/ML or GenAI data pipelines
- Databricks Mosaic AI
- Databricks Genie / AI/BI
- Databricks Agent Bricks
- Data modernization initiatives
- Advanced Analytics
- AWS certifications
Industry Experience
Previous experience working within the Insurance domain is preferred but not mandatory.
Candidates with strong Databricks and AWS engineering experience from banking, financial services, healthcare, telecommunications or other large enterprise environments will also be considered.
Work Location & Availability
- Candidates must be based in the Greater Toronto Area or within reasonable commuting distance of North York.
- This is a hybrid position requiring 3 days per week onsite.
- Candidates must be prepared to work from the client's North York office from Day 1.
- This is not a fully remote position.
- No relocation, travel or accommodation expenses will be provided.
Contract & Rate
- Subcontractor engagement
- Initial short-term assignment with the possibility of extension based on client requirements and performance.
- Maximum rate: CAD $60-70/hour, all-inclusive.
- The agreed rate must include all applicable expenses and other costs associated with the engagement.
Ideal Candidate
The ideal candidate is a hands-on Senior Data Engineer who can independently build and deliver enterprise Databricks solutions rather than someone with primarily theoretical or presentation-level experience.
We are particularly interested in candidates who combine:
Databricks + AWS + PySpark + Data Engineering + End-to-End Implementation Experience
and who are comfortable working onsite in North York 3 days per week from Day 1.
Pay: $60.00-$70.00 per hour
Experience:
- databricks: 7 years (required)
Location:
- North York, ON (required)
Work Location: Hybrid remote in North York, ON (York District)
