Data Engineer

Responsibilities

  During project definition

• Design of data ingestion chains
• Design of data preparation chains
• Basic ML algorithm design
• Data product design
• Design of NOSQL data models
• Design of data visualizations
• Participation in the selection of services / solutions to be used according to the uses
• Participation in the development of a data toolbox

  During the iterative realization phase

• Implementation of data ingestion chains
• Implementation of data preparation chains
• Implementation of basic ML algorithms
• Implementation of data visualizations
• Using ML framework
• Implementation of data products
• Exposure of data products
• Setting up NoSQL databases
• Implementation in distributed mode of treatments
• Use of functional languages
• Debugging distributed processes and algorithms
• Identification and cataloging of reusable elements
• Contribution to the evolution of labor standards
• Contribution and opinion on data processing problems

  During integration and deployment

• Participation in problem solving.

Experience

• Mastery in SQL
• Knowledge of Elastic Search
• Expertise in the implementation of end-to-end data processing chains
• Mastery of distributed development
• Basic knowledge and interest in the development of ML algorithms
• Knowledge of the ingestion framework
• Knowledge of Spark and its different modules
• Mastery of Scala and / or Python
• Knowledge of the AWS or GCP ecosystem
• Knowledge of the ecosystem of NOSQL databases
• Knowledge in the construction of APIs of data products
• Knowledge of Dataviz tools and libraries
• Ability to debug Spark and distributed systems
• Popularization of complex systems
• Knowledge of algorithm complexity
• Mastering the use of notebooks data
• Expertise in data testing strategies
• Strong problem solving, intelligence, initiative and ability to withstand pressure
• Excellent interpersonal skills and a great sense of communication (ability to go into detail).