18 May
Vice President, Engineering - Data Platform
Vacancy expired!
- Lead the creation of a next generation data platform to ingest tens of thousands of datasets, support petabyte-scale storage and compute, and deliver billions of real-time queries per month, while maintaining cost effectiveness and implementing appropriate data safeguards.
- Manage an organization of over a half dozen software and data engineering teams responsible for company’s data ingestion, storage, compute and API services.
- Identify opportunities to improve the performance and scale of APIs, the velocity and efficiency of data ingestion, the connectivity and linking of datasets, and the extraction of natural language and imagery sources.
- Work with senior leadership to translate company’s platform vision and strategy into an actionable roadmap, maintain KPIs to track process, and deliver on-time and on-budget.
- Collaborate with the application engineering, product management, project management, data science and market-facing teams to align the data platform with business needs.
- Define standards and practices around automation, system reliability, data architecture, process management, containerization, infrastructure-as-code, auto-scaling, data security, etc.
- Serve as a mentor for team members, an evangelist of the data platform for other engineering teams, and a translator between engineering and the business. This will include facilitating and participating in design sessions, code reviews and sprint ceremonies, as well as giving presentations on company’s data platform for technical and non-technical audiences.
- Investigate and resolve technical and non-technical issues, including leading and participating within incident management processes and root cause analyses.
- Contribute to company’s technology strategy as a member of its architectural leadership team.
- 5 or more years of experience managing software engineering teams
- 3 or more years of experience with big data systems
- 3 or more years of experience with cloud architecture
- 10 or more years of experience in software engineering
- B.S. in Computer Science (or equivalent)
- Big data architecture and systems, including distributed data processing systems (such as Spark or Dask), distributed data storage systems (such as Parquet or HDFS), low-latency data lake query architectures (such as Alluxio) and real-time streaming systems (such as Kafka)
- Data engineering techniques for big data, including data automation frameworks (such as Airflow or Prefect), metadata management (such as Amundsen) and process management strategies
- Relational, graph, document, key-value and fuzzy data systems, as well as accompanying data models, architectures, and principles
- Data lake design strategies for metadata, ontology, governance, authorization, etc.
- API design principles and practices, including microservice architecture, RESTful endpoint design, GraphQL graph design, OpenAPI service definition, and API testing methodologies.
- Infrastructure management and automation, such as Kubernetes, Terraform and Chef
- Test automation for data quality, data flow, and API endpoints
- Cloud infrastructure management, ideally with past experience in AWS, including both technical aspects, such as solutions architecture, and non-technical aspects, such as financial planning
- Modern practices around agile development, release management, continuous integration, system reliability, cloud architecture, authN/Z and data security
- Organizational leadership, including managing roadmaps, budgets and human resources, as well as defining processes, developing KPIs and managing knowledge
- Fundamentals of computer science and software engineering
- Execute on a data platform strategy in collaboration with team members, architects, product managers and other groups across the business
- Manage high-performance software and data engineering teams, including defining tasks, reviewing designs, facilitating sprint ceremonies, managing releases, hiring to expand the team’s competencies, and guiding professional development to grow the team from within
- Clearly communicate decision points, opportunities, and outcomes to senior leadership
- Exercise discretion and independent judgment on all projects and responsibilities
- Contribute as a technology leader, including hands on when necessary, to meet team objectives
- Mentor team members on technical and non-technical topics
- Stay up to date on emerging technologies, standards, and protocols
Vacancy expired!