Senior Data Engineer (GCP • Python • Iceberg • Delta Lake • Kafka • Snowflake • Databricks)

Accountabilities

• • Design and implement the Delta Lake write layer using Iceberg UniForm to maintain dual Delta and Iceberg metadata and enable data consumption across multiple platforms without data conversion.
  • Build scalable GCS-to-BigQuery ingestion pipelines for structured operational datasets.
  • Develop robust data engineering solutions using Python and Apache Spark.
  • Implement Kafka-based change data capture (CDC) patterns for real-time and near-real-time data ingestion.
  • Design reusable, modular data-sharing adapters and components that support scalable lakehouse integrations.
  • Develop data lineage, dependency tracking, and metadata capabilities to improve data governance and operational visibility.
  • Configure Snowflake Horizon external tables to enable secure, zero-copy access to shared data.
  • Implement and certify Delta Sharing endpoints to support data consumption by Databricks users and workloads.
  • Build governed data access components, including role-based access control (RBAC), connector registry entries, and tenant-scoped authorization.
  • Align semantic-layer models with LookML definitions and established KPI catalogs.
  • Collaborate with cross-functional engineering and business teams to deliver complete, production-ready data features.
  • Troubleshoot complex issues across ingestion, storage, metadata, sharing, integration, and application layers while contributing to code quality and engineering best practices.

  Requirements

  • 6+ years of proven enterprise-level data engineering experience with strong hands-on expertise in Python and Apache Spark.
  • Advanced experience working with Google Cloud Platform (GCP), particularly BigQuery.
  • Strong practical knowledge of Apache Iceberg, Delta Lake, and Iceberg UniForm.
  • Experience implementing Delta Sharing and building Kafka-based CDC and event-driven data pipelines.
  • Hands-on experience with Snowflake Horizon Catalog and Databricks Unity Catalog.
  • Solid understanding of modern data lake and lakehouse architectures, including ingestion pipeline design and distributed data processing.
  • Experience designing scalable, reusable, and maintainable data integration and data-sharing solutions.
  • Strong understanding of data lineage, metadata, governance, access controls, and dependency management.
  • Ability to troubleshoot complex technical issues across multiple layers of a modern data platform.
  • Excellent written and verbal communication skills, with the ability to collaborate effectively across multiple technical and business teams.
  • Strong problem-solving skills and the ability to work independently while contributing effectively to a collaborative engineering environment.
  • Experience in enterprise SaaS, media, or advertising technology environments is a plus.
  • Active experience using AI-assisted software development tools is preferred, with Claude Code experience considered an advantage.

  Benefits

  • Competitive compensation package.
  • 100% remote work within the United States.
  • Full-time employment opportunity.
  • Medical, dental, and vision insurance.
  • Pet insurance.
  • Paid Time Off (PTO).
  • 401(k) retirement plan.
  • Opportunity to work on modern cloud data technologies including GCP, BigQuery, Snowflake, Databricks, Iceberg, Delta Lake, and Kafka.
  • Opportunity to contribute to sophisticated lakehouse and data-sharing architectures with significant technical ownership.
  • Collaborative environment focused on high-quality software engineering and modern development practices.