12 Jun
Sr. Data Engineer
Vacancy expired!
- Product delivery
- Build, test and operate stable, scalable data pipelines that cleanse, structure and integrate disparate data sets into a readable and accessible format for end-user facing reports, data sciences and ad-hoc analyses
- Develop a deep understanding of the product roadmap for the squad, including future features to be developed
- Contribute to high-level estimation and participate in laying out the development sequences, challenging the product roadmap and identifying areas where technical debt can be reduced or avoided
- Complete independently complex development tasks and actively contribute to pushing code to production
- Write testable, efficient, and reusable code suitable for continuous integration and deployment, respecting best practices and development standards
- Review Unified Modeling Language (UML) diagrams and technical documentation
- Be accountable for code quality by conducting adequate testing
- Be accountable for performance, reliability, scalability and resilience of at least one technical component owned by the squad through SLAs and monitoring
- Solve complex technical problems and mentor/support other technical staff on data modelling and ETL related issues
- Contribute to cross-squad initiatives, acting as a change agent amongst peers to foster adoption of new processes or technical solutions
- Review Pull Requests to guide and upskill other Data Engineers on various technical topics
- Actively contribute to our client’s University (the internal peer learning platform) to promote continuous learning
- Participate in the onboarding of new Data Engineers
- Contribute to solution designs, challenging other members on technical decisions and explaining the technical design to junior developers so they can write documentation for the rest of the team
- Participate in HR recruiting events, helping to identify and recruit top developers
- Bachelor’s degree in Computer Science, Engineering, or a related technical field, Master’s degree an asset
- A minimum of 5 years of Functional Programming and/or Object Oriented Programming (OOP) experience
- A minimum of 3 years experience writing and optimizing SQL queries
- A minimum of 3 years experience with Apache Spark for big data processing
- Extensive knowledge of Python programming language and its data manipulation libraries (Pandas and Numpy)
- Expertise in data modeling and an advanced understanding of data architecture
- Expertise with RDBMS and NoSQL databases at scale
- Experience with Apache Airflow or other similar data pipelining and workflow scheduling framework (Luigi, Azkaban)
- Ability to use containers, orchestration frameworks, and other DevOps tools (Kubernetes, Terraform, Giant Swarm, etc.)
- Proficiency with cloud resources (AWS/Google Cloud/Azure) with the ability to operate them for the components owned. Certification is an asset.
- Knowledge of the AWS services (Redshift, Glue, Athena, S3, etc.) an asset
- Knowledge of big data technology (Databricks, Hadoop, Hive, Pig, Presto) an asset
- Familiarity with continuous integration and automated pipeline tools (Jenkins, Travis, etc.)
- Proficiency in Git
- Highly analytical and detail-oriented
- Ability to coach and mentor junior employees to achieve personal and professional goals
- Team player with a high sense of accountability and ownership
- Ability to influence and drive change
- Solution-oriented mindset and can-do attitude to overcome challenges
- Ability to thrive in a fast-paced environment and master frequently changing technologies and techniques
Vacancy expired!