21 Apr
Manager, Platform Engineering (Big Data)
Vacancy expired!
- Manage the creation of a data platform that ingests tens of thousands of datasets, supports petabyte-scale storage and compute, and delivers billions of real-time queries per month, while maintaining cost effectiveness and implementing appropriate data safeguards.
- Lead an engineering team, both as a people manager for a team of direct reports and as a technical leader responsible for key software and system architecture decisions.
- Collaborate with other engineering teams, as well as product managers, project managers, data scientists, and others, to align the data platform with business needs.
- Work with senior leadership to translate platform opportunities into an actionable roadmap, maintain KPIs to track progress, and deliver new platform capabilities on-time and on-budget.
- Facilitate and participate in team activities such as design sessions, code reviews and sprint ceremonies.
- Serve as a mentor for team members, as well as for other teams building on top of the platform.
- Adhere to best practices around versioning, automated testing, dependency management, system reliability, containerization, infrastructure-as-code, auto-scaling, data security, etc.
- Investigate and resolve technical and non-technical issues, resolving critical incidents in a timely manner and with a thorough root cause analysis.
- Contribute to company’s technology strategy as a member of its architectural leadership team.
- 3 or more years as a manager or architect of a big data compute and storage platform
- B.S. in Computer Science (or equivalent)
- 8 or more years of experience in software engineering or systems architecture roles
- 1 or more years managing at least three direct reports
- Preferably experience with geospatial data, graph data and raster data
- Distributed data processing systems, including Spark and Dask
- Data lake storage formats, ideally including Parquet and Hudi
- Relational, graph and document databases systems
- Search and cache layers, including Elasticsearch, Redis and Memcached
- Low-latency models for delivering data lake data at web speed
- Real-time data streaming systems, including Kafka
- Data lake strategies for metadata, ontology, governance, authorization, etc.
- Automated infrastructure scaling and management systems, such as Kubernetes
- Data ingestion automation pipelines, such as Airflow or Prefect
- Infrastructure-as-code, such as with Chef and Terraform
- Hands-on knowledge of AWS infrastructure from solutions architecture to cost management
- SOA, REST, OpenAPI, GraphQL, gRPC, microservices and other API-related concepts
- Modern practices around agile development, release management, continuous integration, system reliability, cloud architecture, authN/Z and data security
- Fundamentals of computer science and software engineering.
- Manage a high-performance software and systems engineering team, including defining tasks, reviewing designs, facilitating sprint ceremonies and managing releases
- Execute on a data platform strategy in collaboration with team members, architects, product managers and other groups across the business
- Clearly communicate decision points, opportunities, and outcomes to senior leadership
- Exercise discretion and independent judgment on all projects and responsibilities
- Contribute as a software engineer and systems architect to meet team objectives
- Mentor team members on technical and non-technical topics
- Stay up to date on emerging technologies, standards, and protocols
Vacancy expired!