Presentation
High-Performance Data-Intensive Science: Clouds, Containers, Workflows, and the Data Lifecycle
Speaker
Event Type
Focus Session
AI/Machine Learning/Deep Learning
Clouds and Distributed Computing
Containerized HPC
Exascale Systems
HPC workflows
TimeMonday, June 17th4pm - 4:30pm
LocationPanorama 3
DescriptionThe HPC environments of tomorrow will have to flexibly accommodate new application domains and dynamic connections with peer facilities. We already see the need to accommodate data-driven applications at scale for AI and campaigns with interleaved simulation and analytics. Data availability from simulations or observational data sources is particularly relevant to enable AI applications at scale. Computing facilities have taken the logical step of instantiating Cloud and Container orchestration frameworks adjacent to the core HPC systems. Such constructs enable key classes of workflows that provide persistence of control while supporting traditional HPC paradigms of computing. We report on our early experiences using such an approach and discuss how this can apply to create seamless connections within and across facilities. The underlying data lifecycle provides the backdrop for such data-intensive scientific explorations.
Speaker
Group Leader, Advanced Data and Workflow