Data Engineer (Elasticsearch + Datawarehousing)

Accountabilities:

• • Design, implement, and optimize Elasticsearch clusters to support high-performance querying, indexing, and data retrieval.
  • Build and manage efficient Elasticsearch indexes, ensuring data is structured and stored appropriately for performance and scalability.
  • Design and optimize data storage solutions, including data warehouses, data lakes, and lakehouse environments.
  • Integrate structured and unstructured data from multiple internal and external sources to create unified, analysis-ready datasets.
  • Develop data pipelines and transformation processes that maintain data accuracy, consistency, completeness, and reliability.
  • Gather requirements with product managers and stakeholders and translate business needs into effective technical data solutions.
  • Provide technical recommendations during requirements analysis and contribute to solution design.
  • Participate in Agile ceremonies including sprint planning, stand-ups, reviews, and other collaborative development activities.
  • Develop backend components, APIs, microservices, and automation scripts using Python, Java, and relevant frameworks.
  • Conduct unit and integration testing to validate functionality, reliability, security, and performance.
  • Diagnose and resolve defects, code quality issues, performance bottlenecks, and data-related problems.
  • Maintain clear technical documentation covering data processes, tools, systems, and development practices.
  • Identify opportunities to optimize existing code, improve scalability, strengthen security, and enhance maintainability.
  • Stay current with emerging cloud, data engineering, and distributed processing technologies and incorporate relevant improvements into solutions.
  • Collaborate effectively with engineers, testers, product managers, and other cross-functional stakeholders throughout project delivery.

  Requirements

  • Bachelor’s degree in Computer Science, Engineering, or a related technical discipline.
  • At least 3 years of professional experience in data engineering or a closely related role.
  • Strong hands-on proficiency with Elasticsearch and Python.
  • Experience working with relational databases such as MySQL or PostgreSQL and NoSQL technologies such as MongoDB.
  • Strong understanding of data warehousing and data lakehouse principles, database architecture, ORM concepts, and data processing.
  • Experience with technologies and frameworks such as Flask, Databricks, Pandas, Spark, PySpark, or similar data engineering tools.
  • Familiarity with machine learning and data analysis libraries such as Scikit-learn or OpenCV is advantageous.
  • Experience using Java to develop or enhance backend systems, particularly where integration with Elasticsearch and databases is involved.
  • Ability to develop APIs, microservices, and automation scripts for data and backend workflows.
  • Familiarity with tools and libraries including logging, requests, subprocess, regex, and pytest.
  • Experience with the ELK stack, Redis, and distributed task queues is a plus.
  • Strong understanding of concurrent and parallel processing concepts.
  • Familiarity with at least one major cloud data engineering ecosystem, such as AWS, Azure, or GCP, with the ability to adapt quickly to different ETL/ELT tools.
  • Experience with Git and collaborative version-control workflows.
  • Comfortable working with Linux environments and creating shell scripts.
  • Solid understanding of software engineering principles, design patterns, testing practices, and maintainable development.
  • Strong analytical and problem-solving abilities with excellent attention to detail.
  • Effective written and verbal communication skills and the ability to collaborate across multidisciplinary teams.
  • Adaptability, curiosity, and willingness to learn new technologies as project requirements evolve.

  Benefits

  • Fully remote position available across India.
  • Opportunity to work on diverse, impactful data engineering projects and complex client data challenges.
  • Hands-on exposure to Elasticsearch, data warehousing, data lakes, cloud platforms, distributed processing, and modern data technologies.
  • Opportunity to work across both structured and unstructured data environments.
  • Collaborative Agile working environment with cross-functional engineering and product teams.
  • Opportunities for continuous technical learning and exposure to emerging tools and technologies.
  • Scope to contribute to scalable, secure, and high-performance data infrastructure.
  • Supportive team culture focused on empowerment, leadership, innovation, teamwork, and technical excellence.
  • Inclusive workplace with equal employment opportunities based on skills, qualifications, and experience.
  • Compensation structured according to experience, expertise, and skills.