Favorite Link Analysis Software
Do list members have a preferred program for analyzing links between webpages? In particular, I'm looking for something that can take a blog (or really any website), spider through the site, and record where the links go to. While there are no lack of computer science papers on such software, those are usually published when the programs are in pre-alpha, so I was hopping that someone knows of a close-to-stable program. While working on a mac would be preferable, I can learn to live with a Windows or Linux one. Any advice for the list for link analysis software that works and isn't a huge pain to set up and use? Thanks in advance Ben Spigel Graduate Student Department of Geography The Ohio State University
Do list members have a preferred program for analyzing links between webpages? In particular, I'm looking for something that can take a blog (or really any website), spider through the site, and record where the links go to.
We usually end up re-writing/tweaking code to do link extraction (e.g. weblogs, FOAF, vlogs, whatever) on a project-by-project basis. The data being collected usually differs enough that one would want to do significant tweaking for each new project. Once you have the links out, of course, you can do whatever sort of analysis your heart desires. Most of the link-extraction code that i've written is in shell scripts and outputs plain text files full of data. Pretty low to the ground stuff, mostly absent of portability problems. I find that this is a not uncommon approach to the problem. Once you have a pretty well-refined idea of what data you are looking for and what patterns in the data are exploitable for your data collection, getting a piece of code pinned down to grab the data is usually the least of your problems. --e
While there are no lack of computer science papers on such software, those are usually published when the programs are in pre-alpha, so I was hopping that someone knows of a close-to-stable program. While working on a mac would be preferable, I can learn to live with a Windows or Linux one.
Any advice for the list for link analysis software that works and isn't a huge pain to set up and use?
Thanks in advance
Ben Spigel Graduate Student Department of Geography The Ohio State University
have you tried issuecrawler? it's at http://issuecrawler.net platform independent, through it uses SVG files to output. -- Barry Saunders http://investigativeblog.net http://blogopalypse.blogspot.com/ http://del.icio.us/new_media --------------------- PhD Candidate // researcher http://creativeindustries.qut.edu.au ph: 07 3138 0155 skype: barry_saunders (CRICOS No. 00213J) On 7/3/07, elw@stderr.org <elw@stderr.org> wrote:
Do list members have a preferred program for analyzing links between webpages? In particular, I'm looking for something that can take a blog (or really any website), spider through the site, and record where the links go to.
We usually end up re-writing/tweaking code to do link extraction (e.g. weblogs, FOAF, vlogs, whatever) on a project-by-project basis. The data being collected usually differs enough that one would want to do significant tweaking for each new project.
Once you have the links out, of course, you can do whatever sort of analysis your heart desires.
Most of the link-extraction code that i've written is in shell scripts and outputs plain text files full of data. Pretty low to the ground stuff, mostly absent of portability problems.
I find that this is a not uncommon approach to the problem. Once you have a pretty well-refined idea of what data you are looking for and what patterns in the data are exploitable for your data collection, getting a piece of code pinned down to grab the data is usually the least of your problems.
--e
While there are no lack of computer science papers on such software, those are usually published when the programs are in pre-alpha, so I was hopping that someone knows of a close-to-stable program. While working on a mac would be preferable, I can learn to live with a Windows or Linux one.
Any advice for the list for link analysis software that works and isn't a huge pain to set up and use?
Thanks in advance
Ben Spigel Graduate Student Department of Geography The Ohio State University
The air-l@listserv.aoir.org mailing list is provided by the Association of Internet Researchers http://aoir.org Subscribe, change options or unsubscribe at: http://listserv.aoir.org/listinfo.cgi/air-l-aoir.org
Join the Association of Internet Researchers: http://www.aoir.org/
I've said enough lately, but here's a couple of tools: http://lexiurl.wlv.ac.uk/ http://socscibot.wlv.ac.uk/ Associated with the University of Wolverhampton, UK, I believe there are some limitations on their use. Cheers, Denise Denise N. Rall, PhD Southern Cross University, Lismore NSW 2480 AUSTRALIA Tues: Room T2.17, +61 (0)2 6620 3577 Mobile 0438 233 344 http://www.scu.edu.au/schools/rsm/staff/pages/drall/ Virtual member, Cybermetrics Group, University of Wolverhampton, UK http://cybermetrics.wlv.ac.uk/index.html ____________________________________________________________________________________ Be a better Heartthrob. Get better relationship answers from someone who knows. Yahoo! Answers - Check it out. http://answers.yahoo.com/dir/?link=list&sid=396545433
participants (4)
-
Barry Saunders -
Ben Spigel -
Denise N. Rall -
elw＠stderr.org