Engenheiro de Dados Sênior / Especialista - AWS / Snowflake / Iceberg

Accountabilities:

• • Design, implement, and evolve data ingestion, transformation, and delivery pipelines across AWS, including the creation and maintenance of Apache Iceberg tables on Amazon S3.
  • Develop distributed data processing jobs using Apache Spark and PySpark through platforms such as Amazon EMR and AWS Glue, including incremental loads, upserts, MERGE operations, and historical reprocessing.
  • Design and maintain interoperability between Snowflake and the Iceberg-based lakehouse, including external volumes, Glue Data Catalog integrations, metadata synchronization, and managed versus externally managed tables.
  • Automate and optimize Iceberg table maintenance, including file compaction, snapshot expiration, orphan-file removal, schema evolution, partition evolution, and related housekeeping activities.
  • Design and optimize Snowflake data models and queries, ensuring high performance and efficient utilization of computing resources.
  • Monitor and optimize AWS and Snowflake performance and costs, including warehouse sizing, clustering, caching strategies, and data lake read costs.
  • Establish and promote data engineering best practices around Git version control, CI/CD, data testing, documentation, lineage, and code review.
  • Implement data governance and security controls, including Snowflake RBAC, masking policies, row-access policies, and AWS Lake Formation permissions.
  • Diagnose and resolve performance, reliability, and availability issues across critical data environments.
  • Serve as a technical reference for the team through code reviews, mentoring of less-experienced professionals, and documentation of architecture decisions.
  • Collaborate with data, product, and business stakeholders to understand requirements and translate them into scalable technical solutions.
  • Contribute proactively to the evolution of the data platform, identifying opportunities to improve architecture, automation, reliability, and engineering efficiency.

  Requirements

  • At least 5 years of proven experience working with AWS Cloud in production environments.
  • At least 3 years of hands-on Snowflake experience, including data modeling, query optimization, warehouse management, and cost optimization.
  • Proven production experience with Apache Iceberg, preferably 2+ years, including partitioning, schema evolution, snapshots, time travel, MERGE operations, maintenance, and file compaction.
  • Strong experience with Apache Spark at scale, particularly PySpark, including job tuning and troubleshooting of skew and shuffle-related issues.
  • Solid knowledge of AWS data services, including S3, Glue ETL, Glue Data Catalog, EMR, Athena, Lambda, and Step Functions.
  • Advanced SQL skills, with the ability to develop complex queries and optimize workloads involving large volumes of data.
  • Strong understanding of data modeling and Data Warehouse, Data Lake, and Lakehouse architectures.
  • Proficiency in Python for automation and data engineering tasks.
  • Experience with data pipeline orchestration tools such as Airflow, Step Functions, dbt, or equivalent technologies.
  • Strong familiarity with Git, version control practices, automated testing, and code review processes.
  • Technical English proficiency sufficient to read and understand technical documentation.
  • Ability to work autonomously and drive complex technical deliveries with limited supervision.
  • Strong analytical and problem-solving skills, with the ability to investigate and resolve complex data engineering challenges.
  • Clear communication skills and the ability to collaborate effectively with business stakeholders and multidisciplinary teams.
  • A proactive, collaborative mindset and willingness to propose and implement technical improvements.
  • Mandatory: proven professional experience with AWS for at least 5 years and Snowflake for at least 3 years.
  • Mandatory: ability to present a PowerPoint case demonstrating practical experience and work with Snowflake.

  Nice-to-have qualifications:

  • SnowPro Core or Advanced certification.
  • AWS certifications such as Solutions Architect, Data Engineer, or Data Analytics.
  • Experience with Terraform or other Infrastructure as Code technologies.
  • Production experience with dbt.
  • Experience with open catalogs such as Glue Data Catalog, Polaris/Open Catalog, or Unity, particularly in multi-engine environments.
  • Experience with streaming technologies such as Kinesis, Kafka/MSK, or Snowpipe Streaming.
  • Familiarity with data observability and quality tools such as dbt tests, Great Expectations, or Monte Carlo.

  Benefits

  • Fully remote work model within Brazil.
  • Opportunity to work with modern AWS, Snowflake, Apache Iceberg, Spark, and Lakehouse technologies.
  • Technical ownership of a strategic data engineering ecosystem.
  • Opportunity to act as a technical reference, mentor other professionals, and influence architecture decisions.
  • Continuous exposure to complex data, cloud, analytics, and AI-driven technology challenges.
  • Collaborative environment with multidisciplinary teams and opportunities for continuous professional development.
  • Work on innovative technology solutions designed to create measurable business impact.