As Data Engineer you'll be working with alongside data architects to take data throughout its lifecycle - acquisition, exploration, data cleaning, integration, analysis, interpretation and visualization. You will be creating the pipeline for data processing, data visualization, and analytics products, including automated services, and APIs.
You seek to grow your expertise in the different infrastructures, tools and applications, and stages of advanced analytic workflows. You are inventive and passionate about streamlining and automating data acquisition and possess a highly structured approach to problem solving.
You will be the go-to person for end-to-end data handling, management and analytics processes.
- Ingest data-sources into our data management platforms.
- Structure data into a scalable and easily understood architecture.
- Work in a multi-disciplined team where you'll turn data discoveries and ideas into models and insights. You'll find how to leverage the data and the models to create and improve products for our customers, in lean development cycles.
- Be able to implement/build methodologies as well as (understand how to) scale them together with the businesses;
- Maintain a good, current and demonstrable knowledge of adjacent application and market developments both for inspiration and for benchmarking the concepts.
- Hands-on experience incl. solid programming to implement pipelines integrating database management systems, cleaning data and improving its data quality
- Experience with Microsoft data management tools and the Azure platform environment
- Curious, proactive, fast learner able to quickly picking-up new areas
- Experience with agile methodologies
- Perfect communication skills
- Can Do approach!
Who we're looking for?
Essential Experience Required
- Proven hands on business intelligence development or data engineering experience;
- Extensive ETL experience;
- Experience in data warehouse design and data lake concepts and practices;
- Exposure working in a Microsoft Azure Data Platform environment;
o Azure Data Factory
o Azure Storage (Blob or Data Lake)
o SQL Azure DB
o Apache Spark/Pyspark
o PythonNICE TO HAVE
- Working on cloud-based big data solutions using Hadoop/Spark;
- SSAS cube development;
- Enterprise BI reporting - Power BI;
- Azure DevOps - CI/CD.