{"id":2798,"date":"2024-02-13T06:53:25","date_gmt":"2024-02-13T06:53:25","guid":{"rendered":"https:\/\/blogs.qub.ac.uk\/dipsa\/?page_id=2798"},"modified":"2025-11-12T09:31:49","modified_gmt":"2025-11-12T09:31:49","slug":"paragrapher","status":"publish","type":"page","link":"https:\/\/blogs.qub.ac.uk\/dipsa\/paragrapher\/","title":{"rendered":"ParaGrapher: Graph Loading API and Library"},"content":{"rendered":"\n

Project Statement<\/strong><\/h2>\n\n\n\n

Several graph frameworks exist and each one optimizes particular graph algorithms for different optimization metrics and is implemented in a different programming language using different parallelization libraries\/techniques. As each framework has implemented its own format of graphs, it becomes time-consuming, inaccurate, and sometimes impossible to compare the execution of graph algorithms across different frameworks for a wide range of graph datasets. Moreover, designing a new graph framework requires investing time for implementing solutions for loading the graphs (as input datasets to the algorithms), while it is the execution of graph algorithms that is the main optimization target of the graph frameworks and not loading the graphs.<\/p>\n\n\n\n

It may be better to separate the process of loading graphs from designing high-performance graph algorithms. We present ParaGrapher, an API and a library for loading graphs. ParaGrapher supports (i) synchronous (blocking) and asynchronous (non-blocking) loading of (ii) simple, vertex- and edge-weighted graphs (iii) in different formats including WebGraph and MatrixMarket. In this way, ParaGrapher helps progressing High-Performance Graph Processing by simplifying the evaluation and refinement of new (and previous) falsifiable contributions on a wider range of datasets.<\/p>\n\n\n\n

ParaGrapher in a Few Bullet Points<\/strong><\/h2>\n\n\n\n

ParaGrapher
(i) provides an API for reading different graph formats from storage and
(ii) implements a library for this API for accessing graphs.<\/p>\n\n\n\n

This allows new graph processing frameworks
(i) to have immediate access for reading graphs and
(ii) to evaluate the new\/previous contributions on a wide range of graph datasets.<\/p>\n\n\n\n

Asynchronous reading of a graph in ParaGrapher:
(i) the user asks reading a range of vertices (and their edges),
(ii) ParaGrapher calls the callback function defined by the user upon completing each block of requested vertices, and
(iii) the user (in callback function) informs ParaGrapher when no further access to a block (i.e., its buffer) is required.<\/p>\n\n\n\n

ParaGrapher library
(i) is not responsible for allocating, releasing, or managing memory and is just responsible for loading block(s) of edges,
(ii) returns the read data as read-only buffers to the user or by populating memory allocated by the user, and
(iii) parallelizes reading and passes blocks of read data to a user-defined callback function on a new thread.<\/p>\n\n\n\n

For now, ParaGrapher supports
(i) WebGraph format by implementing a C-consumser Java-producer algorithm using the shared-memory interface (\/dev\/shm). For more details please refer to its post.
(ii) Other formats: on its way.<\/p>\n\n\n\n

Source Code<\/strong><\/h2>\n\n\n\n

https:\/\/github.com\/DIPSA-QUB\/ParaGrapher<\/a><\/strong><\/p>\n\n\n\n

API Documentation<\/strong><\/h2>\n\n\n\n

Please refer to the Wiki, https:\/\/github.com\/DIPSA-QUB\/ParaGrapher\/wiki\/API-Documentation<\/a>, or download the PDF file using https:\/\/github.com\/DIPSA-QUB\/ParaGrapher\/raw\/main\/doc\/api.pdf<\/a> .<\/p>\n\n\n\n

<\/h2>\n\n\n\n

Publications<\/strong><\/h2>\n\n\n