Data Engineer (Databricks)

Accountabilities:

• • Build, maintain, and optimize data tables, views, jobs, pipelines, and orchestration workflows within the Databricks environment.
  • Develop and operate pipelines that produce and deliver syndicated and custom audience segments based on survey-derived and machine-learning methodologies.
  • Apply data modeling and architecture best practices while helping improve standards for testing, repositories, CI/CD, naming conventions, and code quality.
  • Support the delivery and performance reporting of audience segments distributed through data marketplaces and partner platforms.
  • Investigate and implement emerging Databricks capabilities for real-time processing and operational workloads, including Spark Structured Streaming and Lakeflow Declarative Pipelines.
  • Maintain a high-quality engineering environment with strong data observability, metadata standards, documentation, and data-quality practices.
  • Ensure consumer data is handled responsibly and in accordance with applicable data privacy and internal data governance standards.
  • Use AI-assisted development tools such as Claude Code to accelerate development, testing, documentation, and engineering workflows while maintaining rigorous quality standards.
  • Collaborate with Data Engineering, Data Science, and commercial stakeholders to understand business needs and translate them into functional and technical requirements.
  • Contribute to architecture discussions, engineering solution design, execution planning, releases, and internal training.
  • Monitor emerging technologies, trends, and capabilities across the Databricks ecosystem and evaluate opportunities for practical adoption.
  • Share technical knowledge through brown-bag sessions, tech talks, documentation, and advocacy of effective engineering practices.

  Requirements:

  • Bachelor's or Master's degree in Computer Science or a related technical discipline.
  • 4+ years of professional Data Engineering experience, with strong expertise in Python, SQL, Spark, and distributed data processing.
  • 2+ years of hands-on experience with Databricks, including Data Engineering and Data Warehousing capabilities such as Jobs & Workflows, Notebooks, Unity Catalog, and Delta Lake.
  • Strong understanding of database design, data structures, algorithms, data modeling, and scalable data architecture.
  • Demonstrated ability to translate business requirements into practical and maintainable engineering solutions.
  • Experience using AI coding assistants such as Claude Code, GitHub Copilot, or comparable tools within a modern software engineering workflow.
  • Strong individual-contributor mindset, with the ability to work independently while collaborating effectively with Data Engineering, Data Science, and business teams.
  • Strong focus on data quality, testing, observability, documentation, privacy, and engineering best practices.
  • Curiosity and willingness to stay current with new technologies and developments in the Databricks ecosystem.
  • Familiarity with emerging Databricks technologies such as Genie, Lakebase, AI/BI Dashboards, or Databricks Apps is an asset.
  • Previous experience in digital marketing, ad-tech, media, advertising, or a related industry is a plus.
  • Familiarity with audience, identity, or consumer segmentation data and data marketplaces such as LiveRamp, TransUnion, or Lotame is an advantage.

  Benefits:

  • Competitive compensation: CAD $80,000–$90,000 annually, depending on factors such as location, experience, and performance.
  • Fully remote work opportunity for candidates based in Canada.
  • Occasional travel opportunities to New York City.
  • Career and professional development opportunities.
  • Health, dental, and vision insurance.
  • Pre-tax savings plans and transit/parking programs.
  • 401(k) plan with competitive employer matching, where applicable.
  • Educational, social, and team-building events.
  • Volunteer and philanthropic initiatives.
  • Collaborative culture with opportunities to learn from experienced technical and business professionals.
  • Exposure to innovative technologies across data engineering, AI, advertising, and audience solutions.