19 Jun
Software Engineer (Data Pipelining)
Vacancy expired!
- Maintain centralized data repository, including updating python code for comparing and analysing datasets as necessary
- Run pipeline to process above data, generate synthesis and produce output files (python command)
- Work with team members to automate data pipeline
- Interface with crowd vendors to debug any data issues occurring in the pipeline
- Execution data pipelines for metrics: given a metric requirement and access to where the data lives, create apipeline so that it’s made available for consumption (by dashboards, Tableau, etc)
- Maintain metrics database (MongoDB)
- Work with internal and external APIs to obtain and process data
- Fluent in Python, especially for data processing.
- Experience working with Python’s pandas, numpy and json libraries.
- Experience integrating with external APsI and HTTP requests (in Python): GET/POST calls, pagination,error handling, asynchronous requests, load handling
- Experience with MongoDB and pymongo
- Data visualization in Python a plus.
- Experience with CI/CD tools a plus but not required.
- Experience with linguistics or natural language processing a plus but not required.
Vacancy expired!