02 Nov
Sr. Data Engineer - Cloud
Vacancy expired!
job summary:This role is for a senior data engineer that will join a team responsible for managing a growing cloud-based data ecosystem consisting of a metadata driven data lake and databases that support real time analytics, extracts, and reporting.
Required:- The right candidate will have a solid background in data engineering and should have a few years of experience on a major cloud platform such as Azure.
- Databricks
- Apache Spark
- Scala programming
- Azure
- Data engineering experience - 5 years
- Cloud platform experience - 2 years
- Version Control (Git or equivalent) - 2 years
- Data Integration Tools (Spark/Databricks or equivalent) 2 years
- Version Control (Git or equivalent) 2 years
- Scripting (Linux/Unix Shell scripting or equivalent) 2 years
- Netezza experience
- Building and maintaining a data processing framework on Azure using Databricks
- Writing code in Apache Spark/Scala
- Working with existing Databricks Delta Lake tables to optimize for CDC performance using techniques
- Working with existing Databricks Notebooks to optimize or address performance concerns
- Create new Databricks Notebooks or stand-alone Apache Spark/Scala code as needed
- Willingness to learn existing on-premise data management tools as required, such as Ab Initio
- working in a smoke free environment
- Experience level: Experienced
- Minimum 3 years of experience
- Education: No Degree Required
- Azure
- Databricks
- Apache Spark
- Scala
Vacancy expired!