We are looking for a lead data engineer with team management experience to work in our team.
We offer you an opportunity to work in the international team of about 20 software engineers and data scientists who work on a modern SaaS system that automates data processing. The Client is a big data analytics company in the pharmaceutical sector. The automation process starts from getting a buyer’s request for data processing and goes all the way to receiving data analysis results through the system.
Position Summary: You will set up, improve, expand and maintain the data storage, data infrastructures and data pipelines needed by the Client. This is a “hands on” technical role. The role will also have strategic and architectural aspects. Finally, the role also involves management of a small team of 1-3 engineers.
You will need to be able to work in a US/UK-based multicultural environment and speak freely with your colleagues on a daily basis.
Data architecture: Design data management systems for the Client’s Platform. Enable the Client’s platform to ingest, integrate, and manage all the required sources of data for the Client’s platform to meet business, data science and stakeholder requirements. The work of a data architect may need in-depth knowledge of AWS, SQL, NoSQL, and Postgres, among other systems and tools.
Data Storage: Design and maintain the Client’s data lake, data warehouses and database systems. Ensure that all systems function seamlessly for all Client’s stakeholders. Optimize Client’s data systems and storage for speed. Ensure that updates don’t interfere with workflow, and sensitive information is secure.
Data Engineering: Responsible for building a robust, integrated data infrastructure for the Client’s platform. Develops, constructs, tests and maintains architectures, such as databases and large-scale processing systems of the Platform. The ability to recommend and implement ways to improve data reliability, efficiency, and quality.
Develop, construct, test and maintain Client’s data architectures (such as databases, ETL pipelines and large processing systems).
Ensure Client’s data architecture will support the requirements of the business.
Enable the capability of adding new data sources for the Platform.
Employee a variety of languages and tools (e.g. scripting languages) to marry Client’s data systems together.
Recommend ways to improve data reliability, efficiency and quality.
Ensure Client’s platform data security and compliance.
Managing, growing and developing the team of data engineers.
Ensure a culture of accountability, high performance and ethical behaviour.
Education and Experience
Education: Bachelor’s / Masters degree in Computer Science, Computer Engineering or equivalent is preferred. Various certification and training in AWS is a bonus.
Experience: 5+ years of experience as Data Engineer required. Team Management experience is required.
Extensive experience in data platform technology through all data lifecycle stages covering: sourcing, enhancing, persisting and using data.
Have defined and delivered a data technology roadmap to achieve the strategic goals of a data led business.
Have worked effectively in a global, matrix environment and demonstrated experience of satisfying internal customers for their data needs.
Have worked effectively in a global data team with delivery team located in a different geographical location.
Have worked effectively in an agile, sprint-based delivery model.
Have automated deployment of data platform technology changes in a cloud-based environment (ideally AWS).
Can assess and conduct cost benefit analysis of various technologies related to this field and ability to assess new ones coming in the market.
In depth domain experience building high performing data platform architectures, systems and pipelines in the following domains:
Cloud (Ideally AWS).
Data Platform (Redshift, Airflow, Glue, Athena, EMR, Spark, Lake Formation or equivalent).
Data Modelling (KNIME or equivalent).
Data Transformation/ETL (Glue, KNIME or equivalent).
Languages: (Python or equivalent).
Databases: (RDS, PostgreSQL or equivalent).
Proven track record of leading teams to successfully deliver on multiple projects simultaneously.
Demonstrate and promote a culture of commercial awareness.
Experience of developing and mentoring staff and leading them to excellent performance as well as offering career development opportunities.
They are technically excellent, have an agile process in place and are always looking for "win/win" opportunities. The team is incredibly honest, hard working and has been a source of new ideas and improvements.