Description
eFlows4HPC aims at designing and implementing a European workflow platform that enables the design of complex applications that integrate HPC processes, data analytics and artificial intelligence, making use of the HPC resources in an easy, efficient and responsible way as well as enabling the accessibility and reusability of applications to reduce the time to solution.
As the main outcome, the project will deliver the eFlows4HPC software stack which integrates different components to provide an overall workflow management system. One of the core functionalities of the software stack is the definition of the complex workflows that combine HPC, HPDA and ML frameworks and the integration of large volumes of data from different sources and locations.
On top of this software stack, the project will build an HPC Workflow as a Service (HPCWaaS) platform to facilitate the reusability of these complex workflows in federated HPC infrastructure. The goal is to provide methodologies and tools that enable sharing and reuse of existing workflows and that assist when adapting workflow templates to create new workflow instances.
The HPCWaaS platform and the eFlows4HPC software stack will be validated by use cases organised in three pillars which represent the main sectors that the project targets. The eFlows4HPC project has received funding from the European High-Performance Computing Joint Undertaking (JU) under grant agreement No 955558. The JU receives support from the European Union’s Horizon 2020 research and innovation programme and Spain, Germany, France, Italy, Poland, Switzerland, Norway.
Goals
Funded by the European High-Performance Computing Joint Undertaking (EuroHPC JU) and the participating states, the eFlows4HPC project will create a European workflow platform for the design of complex applications that integrate HPC processes, data analytics, and artificial intelligence. Coordinated by the Barcelona Supercomputing Center (BSC), the project aims to demonstrate the novel technologies through use cases of three application pillars with high industrial and social relevance: manufacturing, climate, and urgent computing for natural hazards.
Nowadays, developers lack tools that enable the development of complex workflows involving HPC simulation and modelling with data analytics and machine learning. With an overall funding of €7,6M, eFlows4HPC aims to deliver a workflow software stack and an additional set of services for the integration of HPC simulation and modelling with big data analytics and machine learning, applied in scientific and industrial domains.
In order to achieve these goals, an ambitious and diverse group of partners came together from seven countries with expertise in different technical aspects: supercomputing and acceleration, workflow management and orchestration, numerical simulation, machine learning and big data analytics, data management and storage; topped with advanced skills in the pillar workflows’ areas of manufacturing, climate, and urgent computing for natural hazards.
Partners
The eFlows4HPC consortium comprises 16 partners from seven different countries with expertise in technical aspects: supercomputing and acceleration, workflow management and orchestration, machine learning, big data analytics, data management, and storage; together with expertise in the pillar workflows’ areas: manufacturing, climate, urgent computing. Find out more about our partners below:
This project has received funding from the European High-Performance Computing Joint Undertaking (JU) under grant agreement No 955558. The JU receives support from the European Union’s Horizon 2020 research and innovation programme and Spain, Germany, France, Italy, Poland, Switzerland, Norway.