Data Engineer

We are seeking to hire a Data Engineer to our team!

Client

CTEC develops and delivers innovative customer-centric technologies and solutions that support the Office of Personnel Management’s (OPM) Health and Insurance business unit and Office of the Chief Information Officer (OCIO).

Duties and Responsibilities

• Data Pipeline & ETL Development: Design, develop, and maintain scalable ETL pipelines and data workflows to ingest, transform, and integrate data from legacy systems and external sources into modern cloud-based data platforms.
• Cloud Data Platform & Databricks Implementation: Build, optimize, and maintain data processing solutions using Azure Databricks and lakehouse architectures to support analytical, operational, and reporting use cases.
• Data Migration Support: Support phased data migration from legacy databases and ETL tools to Azure Databricks environments, including transformation documentation and data mapping.
• Lakehouse & Data Layer Design: Implement layered lakehouse data architectures (e.g., bronze, silver, gold layers) in Databricks to support data quality, performance, and downstream reporting needs.
• Python & PySpark Development: Develop data processing notebooks, workflows, and distributed data transformations using Python and PySpark within Databricks environments.
• Data Quality & Validation: Develop data validation, reconciliation, and testing processes to ensure data accuracy, completeness, and consistency across data domains.
• Data Integration & Interoperability: Integrate Databricks data platforms with analytics and reporting tools to enable business intelligence and operational dashboards.
• Data Governance & Security Implementation: Support data governance initiatives including metadata management, data catalog integration, encryption, access controls, and compliance with federal data protection requirements.
• DevOps & CI/CD Support: Maintain source control and CI/CD pipelines for Databricks and data engineering workflows, supporting automated promotion across environments.
• Technical Collaboration: Work closely with data architects, solution architects, business analysts, and reporting teams to implement approved data solutions.
• Operational Support & Troubleshooting: Provide ongoing support for Databricks workflows, resolve pipeline failures, and troubleshoot complex data processing issues.
• Mentorship & Knowledge Sharing: Provide guidance to junior data engineers and contribute to documentation and team enablement.
• Works under minimal supervision with minor guidance from senior personnel.

Skills & Work Experience

• Professional Experience: 7+ years of experience in data engineering, E