{"id":2309,"date":"2023-08-10T10:02:53","date_gmt":"2023-08-10T09:02:53","guid":{"rendered":"https:\/\/blogs.qub.ac.uk\/dipsa\/?page_id=2309"},"modified":"2025-02-28T04:41:22","modified_gmt":"2025-02-28T04:41:22","slug":"ms-biographs","status":"publish","type":"page","link":"https:\/\/blogs.qub.ac.uk\/dipsa\/ms-biographs\/","title":{"rendered":"MS-BioGraphs: Trillion-Scale Public Real-World Sequence Similarity Graph Datasets"},"content":{"rendered":"\n

Project Statement<\/strong><\/h2>\n\n\n\n

Progress in High-Performance Computing in general, and High-Performance Graph Processing in particular is highly dependent on the availability of publicly-accessible, relevant, and realistic data sets.
We (i) investigate and optimize the process of generating large sequence similarity graphs as an HPC challenge and (ii) demonstrate this process in creating MS-BioGraphs, a new family of publicly-available real-world graph datasets with up to 2.5 trillion edges, that is, 6.6 times greater than the largest graph published recently.
The largest graph is created by matching (i.e., all-to-all similarity aligning) 1.7 billion protein sequences. The MS-BioGraphs family includes also seven subgraphs with different sizes and direction types.<\/p>\n\n\n\n

Project Steps<\/strong><\/h2>\n\n\n\n

(i) Creating and validating the MS-BioGraphs by designing and engineering parallel and distributed algorithms and data structures to optimize performance and cluster utilization,
(ii) Extending the WebGraph<\/a> framework with parallel compression for MS-BioGraphs as edge-weighted graphs, and
(iii) Analyzing the structural characteristics of MS-BioGraphs by comparing to other real-world graphs.<\/p>\n\n\n\n

Datasets on IEEE DataPort<\/strong><\/h2>\n\n\n\n

DOI: https:\/\/doi.org\/10.21227\/gmd9-1534<\/a><\/p>\n\n\n\n

Validation & Sample Code<\/strong><\/h2>\n\n\n\n

Please visit the MS-BioGraphs Validation<\/a> post.
The ParaGrapher library (https:\/\/blogs.qub.ac.uk\/DIPSA\/ParaGrapher\/<\/a>) may be used to access MS-BioGraphs in C\/C++.<\/p>\n\n\n\n

Datasets, Source Code, and Publications<\/strong><\/h2>\n\n\n