Senior Data Engineer
About the role
Every day, somewhere in the world, important decisions are made. Whether it is a private equity company deciding to invest millions into a business or a large corporation implementing a new strategic direction, these decisions impact employees, customers and other stakeholders.
Consulting and private equity firms come to proSapient when they need to discover knowledge to help them make great decisions and succeed in their goals. It is our mission to support them in their discovery of knowledge.
We help our clients find industry experts who can provide their knowledge via interview or survey; we curate this knowledge in a market-leading software platform; and we help clients surface knowledge they already have through expansive knowledge management.
About the role:
We are looking for a Senior Data Engineer to build and own those pipelines. This is a hands-on, delivery-focused role on a team that ships frequently, working directly with the Lead Search Engineer, an AI/ML Engineer and a PM measured on match quality and CSAT.
Key Duties & Responsibilities:
Pipelines & Data Products
- Build and operate batch and real-time pipelines feeding the matching and search systems.
- Work closely with real-time data streams and search infrastructure using tools like Kafka and Elasticsearch.
- Model the datasets the product layer consumes — candidate features, interaction events, ranking signals — and keep the contracts stable.
- Own the pipelines you build in production: monitoring, alerting, backfills, and the on-call reality of a broken 3am job.
Feedback Loops & Instrumentation
- Instrument recruiter and client behaviour so matching quality can be measured and improved.
- Build the event pipelines behind our CSAT and workflow metrics.
- Conduct and analyse experiments to rapidly iterate on the quality of data processing pipelines and make experiment data queryable without a bespoke analysis every time.
Quality & Reliability
- Add data quality checks and tests at the boundaries that matter and make failures loud instead of silent.
- Reduce cost and latency where it counts, leaving the rest alone.
- Partner with QA to make data correctness a testable property, rather than a hope.
- Identify and provide useful data sources, internal or external, for our structured search and GenAI tools.