-
ID
#12462965 -
Job type
Permanent -
Salary
Depends on Experience -
Source
Task Management Inc -
Date
2021-04-19 -
Deadline
2021-06-18
Software / Systems Architect (Data Platform) - Software Co.
California, Irvine, 92602 Irvine USAPermanent
Vacancy expired!
- Design systems that support ingestion, storage, compute and web-speed delivery of billions of monthly API calls on petabyte-scale spatial, graph and raster data ingested from thousands of sources, while maintaining cost effectiveness and implementing appropriate data safeguards.
- Identity and implement new capabilities within the platform that create new opportunities for both real-time web speed queries and for long-running asynchronous analysis.
- Design elegant and intuitive REST and GraphQL APIs that support a range of use cases from basic queries to complex user-defined compute pipelines.
- Implement new models for linking and modeling datasets through spatial and relational models, machine learning, natural language processing, computer vision and other techniques, and increase the velocity of data ingestion and processing while reducing human touch points.
- Align the data platform with the use cases of its consumers, including web applications, API integrations, data scientists, BI analysts and others.
- Ensure that systems are manageable, maintainable, reliable, scalable and secure, designing best practices around infrastructure automation, cloud scaling, quality assurance, monitoring, logging, data governance, security and privacy, etc.
- Collaborate with managers of data platform engineering teams to ensure that systems are built as designed and interoperate effectively.
- Participate in team activities such as design sessions, core reviews and sprint ceremonies.
- Serve as a mentor for engineers across the business.
- Adhere to best practices around versioning, automated testing, dependency management, system reliability, containerization, infrastructure-as-code, auto-scaling, data security, etc.
- Investigate and resolve technical and non-technical issues, resolving critical incidents in a timely manner and with a thorough root cause analysis.
- Contribute to company’s technology strategy as a member of its architectural leadership team.
- 5 or more years of experience building distributed big data systems, with 2 or more of these years in an architect or management role
- B.S. in Computer Science (or equivalent)
- 8 or more years of experience in software engineering and/or systems architecture
- Preferably experience with geospatial data, graph data and raster data
- Distributed data processing systems, including Spark and Dask
- Data lake storage formats, ideally including Parquet and Hudi
- Relational, graph and document databases systems
- Search and cache layers, including Elasticsearch, Redis and Memcached
- Low-latency models for delivering data lake data at web speed
- Real-time data streaming systems, including Kafka
- Data lake strategies for metadata, ontology, governance, authorization, etc.
- Automated infrastructure scaling and management systems, such as Kubernetes
- Data ingestion automation pipelines, such as Airflow or Prefect
- Infrastructure-as-code, such as with Chef and Terraform
- Hands-on knowledge of AWS infrastructure from solutions architecture to cost management
- SOA, REST, OpenAPI, GraphQL, gRPC, microservices and other API-related concepts
- Modern practices around agile development, release management, continuous integration, system reliability, cloud architecture, authN/Z and data security
- Familiarity with spatial techniques/standards such as Geohash, Quadkey, H3, WMS and WMTS
- Familiarity with data modeling, graph theory, knowledge graphs and ontology design
- Familiarity with AuthN/Z standards and practices, including with OAuth 2.0, SAML and OIDC
- API design fundamentals, including REST, GraphQL and gRPC
- Data system fundamentals such as partitioning, optimization, indexing, query planning, etc.
- Computer science and software engineering fundamentals
- Design high-performance data ingestion, storage, compute and delivery systems that serve a variety of consumers from web apps and APIs to data scientists and business intelligence
- Execute on a data platform strategy in collaboration with team members, architects, product managers and other groups across the business
- Ensure interoperability of systems designed between multiple teams across the organization
- Clearly communicate decision points, opportunities, and outcomes to senior leadership
- Exercise discretion and independent judgment on all projects and responsibilities
- Contribute to development of systems and software to meet team objectives
- Mentor team members on technical and non-technical topics
- Stay up to date on emerging technologies, standards, and protocols
Vacancy expired!
Report job