Senior Data Engineer

Overview

Senior Data Engineers play a crucial role in helping our client organizations manage and leverage their data effectively. Their responsibilities include Data Architecture Design; Data Warehousing; Data Quality Assurance; Scalability and Performance Optimization; and Data Security. As senior-level individual contributors, SDEs are also responsible for recommending and implementing data engineering tools and technologies that best suit the client’s needs. Because our clients are mostly US-based organizations, we look for the ability to communicate with professional proficiency in English, verbally and in writing.

Responsibilities

• Data Engineering Leadership: You are responsible for bridging technical gaps to lead and oversee various data engineering projects for our data solutions engagements. You possess advanced data engineering knowledge and expertise, and your assignments, projects, and programs are of significant scope and/or complexity where effective decision-making requires navigating ambiguous or uncertain conditions.
• Cross-Team Collaboration: You are responsible for contributing to alignment across the organization through influence and by translating context across different teams. You have a strong, foundational understanding of other Data Team functions, while also being able to translate these concepts for less technical colleagues.
• Project Enablement: You are responsible for ensuring that large and/or more complex engineering projects are delivered in alignment with the appropriate business outcomes. As a senior+ member of the Data Engineering function, you serve as a mentor to data solutions team members, data engineering managers, and other data engineers, and may be embedded on a client team to unblock and upskill them.

Tools & Technologies

• Programming languages (e.g. SQL, Python)
• Data Processing (e.g. Apache Spark, dbt)
• Cloud-based data warehouses (e.g., Snowflake, Google BigQuery)
• Data orchestration (e.g., Apache Airflow, Azure Data Factory, Prefect)

Technical Skills

• Data Movement. Optimizes data pipelines to achieve low latency, ensuring timely processing and delivery of data. Can design and implement real-time and /or micro-batch data processing solutions that handle high-volume data with low latency requirements.
• Data Warehousing. Acts as an internal resource for data warehousing, providing help with advanced techniques and strategies for novel scenarios.
• Programming. Has an authoritative or deep holistic understanding, deals with routine matters intuitively, able to go beyond existing interpretations, achieves excellence with ease.
• Technical Management: Is able to be a technical leader across a set of related team's domains, consistently pushing boundaries and exploring gaps in understanding.
• Data Infrastructure: You can design, build, and maintain the underlying data infrastructure for a complex, large-scale system that integrates with multiple disparate data sources.

Bonus points for:

• Data Modeling and Transformation: Comfortable with modeling and transforming extremely large and complex datasets in a performant and cost-efficient manner.
• MLOps: Understands and can execute the steps required to deploy a trained model into a production environment for offline or online scoring.

Skills & Qualifications

• Proven experience as a Data Engineer or related role, with a focus on designing and developing data pipelines.
• Strong programming skills in Python and SQL. Experience with Scala and Rust is a plus but not required.
• Deep knowledge of data warehousing and ETL/ELT processes.
• Intermediate / expert proficiency with common data integration / orchestration platforms (e.g., Fivetran, Azure Data Factory, Apache Airflow)
• Deep experience with at least one cloud data warehouse (Snowflake, BigQuery, Databricks, or similar).
• Experience with streaming solutions such as Spark Streaming, Kafka, or Flink is desirable but not required.
• Strong experience with at least one cloud platform such as AWS, Azure, or Google Cloud.
• Familiarity with machine learning operations (MLOps) techniques and platforms is a plus but not required.
• Experience mentoring and advising other engineers
• Ability to define, explain, and sell long-term vision for technical architecture, technical approaches / procedures, etc.
• Excellent communication and collaboration skills.

Physical Requirements

• Frequent sitting at a desk performing work on a computer
• Reasonable accommodations may be made to enable individuals with disabilities to perform the essential functions

Compensation Range: $145,000 - 160,000 annually

Please note that compensation packages are finalized after the interview process is concluded. We use a competency-based approach to base pay, which means it is based on the competencies and skills demonstrated for this role.