Large Twitter network graph
Hello all, I am in need of help and suggestions on a task for my study. I have a dataset with over 1 million nodes that I am trying to graph. My interest is in visualising the degree of connections, centrality and betweenness. Used Networkx module in python but the code for drawing the graph runs for days with no error or output. I have tried out a smaller sample of the dataset and it worked nicely. Any suggestions on a tool/software that can handle such node size, or an alternative approach would be greatly appreciated. Thank you in advance. Samuel Olaniran PhD Student, Department of Media Studies, University of the Witwatersrand
Hi Samuel, Depending on the bus speed/processing capacity of your computer, visualizing large samples can be (obviously) problematic. However, some viz software (such as Gephi, UCNet, Netlyzer, etc.) can be tweaked (settings) to accommodate large node mass visualization. I assumes your best bet is to use these software solely designed for visualization, it helps with computing resource management (a list of useful tools has been compiled by SocialMedia Lab at Ryerson https://socialmedialab.ca/apps/). Similarly, check out Deen Freelon’s TSM tools on GitHub ...in case you’re working with Twitter data. It comes in well recommended for preparing node tables for Gephi parsing. HTH Best wishes Martins Olu’ Badmus LIS PhD Candidate Western University Rm 4039 FNB London, Ontario. Canada N6A 5B9 Skype: olubadmus007 I acknowledge that Western University is located on the traditional lands of the Anishinaabek, Haudenosaunee, Lūnaapéewak and Attawandaron peoples, on lands connected with the London Township and Sombra Treaties of 1796 and the Dish with One Spoon Covenant Wampum. This land continues to be home to diverse Indigenous peoples (e.g. First Nations, Métis and Inuit) whom we recognize as contemporary stewards of the land and vital contributors of our society. From: Olaniran Psalmuel<mailto:psalmuel35@gmail.com> Sent: September 7, 2020 5:14 PM To: air-l@listserv.aoir.org<mailto:air-l@listserv.aoir.org> Subject: [Air-L] Large Twitter network graph Hello all, I am in need of help and suggestions on a task for my study. I have a dataset with over 1 million nodes that I am trying to graph. My interest is in visualising the degree of connections, centrality and betweenness. Used Networkx module in python but the code for drawing the graph runs for days with no error or output. I have tried out a smaller sample of the dataset and it worked nicely. Any suggestions on a tool/software that can handle such node size, or an alternative approach would be greatly appreciated. Thank you in advance. Samuel Olaniran PhD Student, Department of Media Studies, University of the Witwatersrand _______________________________________________ The Air-L@listserv.aoir.org mailing list is provided by the Association of Internet Researchers http://aoir.org Subscribe, change options or unsubscribe at: http://listserv.aoir.org/listinfo.cgi/air-l-aoir.org Join the Association of Internet Researchers: http://www.aoir.org/
Depending on the structure of the graph, it is also worth considering the sfdp engine in GraphViz for the visualization portion - while not as good at teasing out fine structure as other algorithms, it is one of the more scalable layout and software rendering algorithms for large graphs and we've used it for some very large Twitter graphs. Back in 2012 we routinely performed layout and rendering of billion-edge graphs in under 30 minutes on a single core on an SGI UV2 and GraphViz makes it quite straightforward to enrich the graphs with external calculated attributes that can be used to affect the visuals. Today we still make heavy use of its rasterizer for geographic visualization for rendering high-resolution graphs containing tens to hundreds of billions of points and it is still one of the more efficient software rasterizers out there and have continued to use it for our largest social graph layout needs, while using Gephi for small graphs. Kalev On Mon, Sep 7, 2020 at 9:42 PM Oluwole Badmus <obadmus@uwo.ca> wrote:
Hi Samuel,
Depending on the bus speed/processing capacity of your computer, visualizing large samples can be (obviously) problematic. However, some viz software (such as Gephi, UCNet, Netlyzer, etc.) can be tweaked (settings) to accommodate large node mass visualization. I assumes your best bet is to use these software solely designed for visualization, it helps with computing resource management (a list of useful tools has been compiled by SocialMedia Lab at Ryerson https://socialmedialab.ca/apps/).
Similarly, check out Deen Freelon’s TSM tools on GitHub ...in case you’re working with Twitter data. It comes in well recommended for preparing node tables for Gephi parsing.
HTH
Best wishes Martins Olu’ Badmus LIS PhD Candidate Western University Rm 4039 FNB London, Ontario. Canada N6A 5B9 Skype: olubadmus007
I acknowledge that Western University is located on the traditional lands of the Anishinaabek, Haudenosaunee, Lūnaapéewak and Attawandaron peoples, on lands connected with the London Township and Sombra Treaties of 1796 and the Dish with One Spoon Covenant Wampum. This land continues to be home to diverse Indigenous peoples (e.g. First Nations, Métis and Inuit) whom we recognize as contemporary stewards of the land and vital contributors of our society.
From: Olaniran Psalmuel<mailto:psalmuel35@gmail.com> Sent: September 7, 2020 5:14 PM To: air-l@listserv.aoir.org<mailto:air-l@listserv.aoir.org> Subject: [Air-L] Large Twitter network graph
Hello all,
I am in need of help and suggestions on a task for my study. I have a dataset with over 1 million nodes that I am trying to graph. My interest is in visualising the degree of connections, centrality and betweenness. Used Networkx module in python but the code for drawing the graph runs for days with no error or output. I have tried out a smaller sample of the dataset and it worked nicely. Any suggestions on a tool/software that can handle such node size, or an alternative approach would be greatly appreciated. Thank you in advance.
Samuel Olaniran PhD Student, Department of Media Studies, University of the Witwatersrand _______________________________________________ The Air-L@listserv.aoir.org mailing list is provided by the Association of Internet Researchers http://aoir.org Subscribe, change options or unsubscribe at: http://listserv.aoir.org/listinfo.cgi/air-l-aoir.org
Join the Association of Internet Researchers: http://www.aoir.org/
_______________________________________________ The Air-L@listserv.aoir.org mailing list is provided by the Association of Internet Researchers http://aoir.org Subscribe, change options or unsubscribe at: http://listserv.aoir.org/listinfo.cgi/air-l-aoir.org
Join the Association of Internet Researchers: http://www.aoir.org/
Pajek is software that can handle very large networks (http://mrvar.fdv.uni-lj.si/pajek/), specifically the xxl and 3xl versions: http://mrvar.fdv.uni-lj.si/pajek/PajekXXL.htm Hope this helps. Maurice ________________________________________________ Maurice Vergeer www.mauricevergeer.nl ________________________________________________ ________________________________________ Van: Air-L <air-l-bounces@listserv.aoir.org> namens Olaniran Psalmuel <psalmuel35@gmail.com> Verzonden: maandag 7 september 2020 23:14 Aan: air-l@listserv.aoir.org Onderwerp: [Air-L] Large Twitter network graph Hello all, I am in need of help and suggestions on a task for my study. I have a dataset with over 1 million nodes that I am trying to graph. My interest is in visualising the degree of connections, centrality and betweenness. Used Networkx module in python but the code for drawing the graph runs for days with no error or output. I have tried out a smaller sample of the dataset and it worked nicely. Any suggestions on a tool/software that can handle such node size, or an alternative approach would be greatly appreciated. Thank you in advance. Samuel Olaniran PhD Student, Department of Media Studies, University of the Witwatersrand _______________________________________________ The Air-L@listserv.aoir.org mailing list is provided by the Association of Internet Researchers http://aoir.org Subscribe, change options or unsubscribe at: http://listserv.aoir.org/listinfo.cgi/air-l-aoir.org Join the Association of Internet Researchers: http://www.aoir.org/
participants (4)
-
kalev leetaru -
Olaniran Psalmuel -
Oluwole Badmus -
Vergeer, M.R.M. (Maurice)