08 Mar
Sr Data Engineer with Java
Vacancy expired!
- Strong demonstrable experience in data integration and data warehouse projects
- Extensive data engineering experience with Spark (2.4) preferably with Java
- Experience putting end-to-end data pipelines together and well verse with services on AWS
- Experience with real time applications for consuming and publishing data to queue/stream such as SQS, Kinesis or any other pub-sub model
- Experience with Batch processing of data having high data volume
- Experience with Java is a must
- Experience developing and configuring spark application, testing and enable activation based on prioritized business use cases
- Experience defining data sets and strategies
- Experience in No SQL database preferably Cassandra and Dynamo DB
- Data modeling expertise
- Experience in performance tuning of spark application, error handling and setting up monitoring of jobs
- Experience in implementing CICD pipeline through Jenkins
- Experience in Prometheus, Grafana is a plus
- Certification in AWS is a plus
- Experience supporting ML teams and some knowledge of SparkML is a plus
- Experience working on ETL tools such as Informatica, Talend is a plus
Vacancy expired!