Senior Data Engineer
23/05/23
160 – 180 PLN/HOUR
Remote
The Data Team works on building and deploying novel analytical algorithms based on the
data from our IoT products. The person in this position will work on the full stack; from data
pipelines and processing to creating user facing applications.
data from our IoT products. The person in this position will work on the full stack; from data
pipelines and processing to creating user facing applications.
Your responsibilities:
- Build and run Azure based Data Platform.
- Handle multiple Terabytes of different domains data with focus on IoT data using Stream and Batch processing.
- Craft and maintain world-class, high traffic analytical applications and services with a team of experienced data engineers and scientists.
- Develop data models and process raw data at scale to make it available for analysis.
- Build scalable data pipelines using state-of-the-art technologies in the Microsoft Azure Cloud.
- Implement, run and deploy algorithms in cooperation with data scientists using the Hadoop ecosystem (e.g.Spark, Hive, HDFS) and the Microsoft Azure Stack).
What we expect:
- Certification University degree in Computer Science or related fields of study (Mathematics, Physics)
- At least 5 Years of experience in architecting and building Data Lake and Enterprise Analytics Solutions
- Ability to optimize ‘Big Data’ data pipelines, design Data Lake architecture and data sets
- Experience with Design and Architecture of relational SQL and NoSQL databases
- Advanced hands-on SQL, Python, pySpark and HDFS knowledge and experience working with relational databases for data querying and retrieval
- Experience with Design and Architecture of Azure big data frameworks: Azure Databricks, Azure Data Factory, Azure ML, SQL Data Warehouse, HDInsights, Stream Analytics, Azure Data Explorer – or their open source counterparts: Kafka, Spark, Blob Storages.
- Knowledge of DevOps Processes (including CI/CD) and infrastructure as a code fundamentals
- Knowledge of Data Quality issues and methods of dealing with them
Nice to have:
- experience in designing, building and integrating REST APIs
- Fluent in English, German language will be considered as a plus
Over 60% of our candidates get invited to an interview with our Clients.
Apply with the form below and we will reach out to you in the next 24h
Apply now!