Devsu
Devsu

Senior Data Engineer (Databricks on Azure)

datafull timePeru
SALARY
Not listed
WORK TYPE
remote
JOB TYPE
full time
INDUSTRY
general
Apply for this position
✦ AutoApply Sick of applying? We apply to roles like this for you, up to 20 a month.
Learn more

About the role

We are looking for a Senior Data Engineer with advanced, hands-on Databricks on Azure experience to design, build, modernise and operate scalable data pipelines supporting Tax technology platforms. You will migrate and enhance existing Spark and Python workloads on Azure Databricks and build scalable ingestion and document-processing capabilities. We need someone who has owned production Databricks on Azure pipelines end to end, understands the wider system, surfaces assumptions and trade-offs, and proposes better approaches.

Responsibilities

  • Design, build, and support scalable data pipelines using Python, Apache Spark, PySpark, and Databricks on Azure.
  • Modernise and migrate legacy data processing workloads to secure, cloud-native platforms.
  • Build and maintain batch data ingestion pipelines from structured and unstructured sources.
  • Integrate data from REST APIs, SharePoint, document repositories, enterprise applications, and cloud platforms.
  • Implement data quality, monitoring, observability, and operational controls.
  • Optimise data workloads for performance, scalability, reliability, and cost efficiency.
  • Develop document extraction, classification, metadata enrichment, and automation pipelines.
  • Apply software engineering practices, including Git, CI/CD, automated testing, and code reviews.
  • Build, deploy, troubleshoot, and maintain production ETL/ELT pipelines.
  • Collaborate with architects, developers, tax subject matter experts, and platform teams.
  • Analyse existing codebases and identify opportunities to improve maintainability, security, and reliability.
  • Use AI-assisted development tools to build, review, test, and maintain data pipeline code.
✦ Sick of applying to 40 jobs a month?
I rewrite your resume for ATS by hand first. Once you sign off on it, AutoApply applies to up to 20 roles like this a month, cover letter in your own voice each time. From $99/mo, cancel anytime.
Get AutoApply
Apply now