02 Nov
Senior Data Engineer
Vacancy expired!
- Design and implement data processing pipelines
- Integrate data from multiple data sources, develop cross-platform ETL processes
- Data Validation and Verification
- Analyze data, solve problems, and implement solutions for ensuring data quality and delivery
- Create systems for data acquisition and wrangling
- Develop new tools and processes for managing our data workflows and data infrastructure
- Collaborate with our Engineering and Data Science teams on building, maintaining and monitoring the database infrastructure
- Collaborate with product managers, data scientists, business users and other engineers to define requirements and design solutions.
- Discover and analyze data from the web (census, open data, commercial vendors)
- Expert in reporting, analytics, and databases
- Data ingestion, ETL and storage
- Interest in pulling data from many sources
- Experience in big data, data mining and statistical analysis
- Cloud computing, especially AWS technologies (S3, EC2, etc.)
- Comfortable choosing technologies that fit the application (e.g. MySQL versus PostgreSQL, Hadoop versus Cassandra)
- More then 5 years of experience in object-oriented development with Python
- Other languages like Scala, C, Java, or similar are a plus
- Experience with spark
- Expertise with SQL
- Familiarity with Docker
- Machine Learning libraries and frameworks like scikit-learn, Tensorflow, Pytorch a plus
- Deploying algorithms at scale
Vacancy expired!