Re: [Air-L] Tool for collecting Instagram images/websites/data?
Hi Todd, Rainer and all Here’s a blog post on the Harkive site that (I hope) explains the process I described, whereby data from different social media sources can be automatically written to a single database. The post includes a short video walkthrough of the process. http://harkive.org/datcolzap/ Hope you find this useful Kind regards Craig
On 19 Sep 2016, at 13:02, Todd O'Neill <Todd.O'Neill@mtsu.edu> wrote:
Craig
Would you mind sharing your Zapier workflow with us?
Cheers!
Todd O'Neill Assistant Professor, New Media Electronic Media Communication Department College of Media and Entertainment Middle Tennessee State University todd.oneill@mtsu.edu LinkedIn: toddoneill | FaceBook: OneillTodd | Twitter: mtsunewmedia
On Sep 19, 2016, at 2:10 AM, Craig Hamilton <Craig.Hamilton@bcu.ac.uk> wrote:
Hi Rainer,
When collecting data for my Harkive project (which collects on the #harkive hashtag across various platforms, including Instagram, Twitter, etc), I recently started using Zapier, which I found solved a lot of time-consuming issues.
The main advantage for my purposes was that it significantly reduced data cleaning/organising post collection. As I’m sure you are aware, the API of each service uses slightly different terms for common elements of the data available. EG: On Twitter the text content of a Tweet is contained with the <text> element, whereas on Tumblr the text written by a user is in the <body> element. Not only that, but date/time stamps often appear in different formats. What Zapier allowed me to do was collect from the various APIs, reformat to common data formats, and then write all data to specific columns in a central GoogleDocs Spreadsheet - so, all commonly formatted date/time stamps appeared in a single column, all usernames in another, and so on, regardless of the service they originated from. I was also able to create a new entry in each row that labelled each entry as originating from Twitter, Tumblr, Instagram, etc. This was all in an attempt to have my data rendered as ‘Tidy’, in the Hadley Wickham sense, automatically.
The disadvantage of Zapier is that there is a charge. I was collecting over a short period of time so I was able to keep this cost quite low. If you are collecting a lot of data over a sustained period of time, you may find it prohibitively expensive.
Let me know if you would like to know more - I am happy to share my rough workflow notes with you, or can show you in Berlin during the conference.
Kind regards Craig
On 18 Sep 2016, at 10:24, Maurice Vergeer <m.vergeer@maw.ru.nl> wrote:
maybe the R package instaR: https://github.com/pablobarbera/instaR/blob/master/examples.R For longterm applications you could learn web scraping with R package rvest (there ar more options for web scraping) The quick-and-dirty method is search a tag on Instagram with a browser, scroll down multiple times to load as much pictures as you need, then save the resulting Instagram page in a new specific folder, which then contains all pictures with that specific tag.
hope that helps
On Sat, Sep 17, 2016 at 5:27 PM, Rainer Hillrichs <hillrichs@uni-mannheim.de
wrote:
Dear all,
I searched on the list and on the web but couldn't find anything: I'm looging for a tool that collects Instagram images, websites, and data associated with a specific tag. Basically, I want to type in a tag and end up with a folder full of images, websites, and a table with data (e.g. user name, date posted, URL, other tags). I already suspect that is a lot to ask for ;-) Even a simpler tool would be a good start! As long as I don't have to to end up saving individual images, websites, and typiing/copying stuff into a table.
Suggestions very much appreciated! Rainer
-- Dr. Rainer Hillrichs Universität Mannheim https://uni-mannheim.academia.edu/RainerHillrichs _______________________________________________ The Air-L@listserv.aoir.org mailing list is provided by the Association of Internet Researchers http://aoir.org Subscribe, change options or unsubscribe at: http://listserv.aoir.org/ listinfo.cgi/air-l-aoir.org
Join the Association of Internet Researchers: http://www.aoir.org/
-- ________________________________________________ Maurice Vergeer To contact me, see http://mauricevergeer.nl/node/5 To see my publications, see http://mauricevergeer.nl/node/1 ________________________________________________ _______________________________________________ The Air-L@listserv.aoir.org mailing list is provided by the Association of Internet Researchers http://aoir.org Subscribe, change options or unsubscribe at: http://listserv.aoir.org/listinfo.cgi/air-l-aoir.org
Join the Association of Internet Researchers: http://www.aoir.org/
_______________________________________________ The Air-L@listserv.aoir.org mailing list is provided by the Association of Internet Researchers http://aoir.org Subscribe, change options or unsubscribe at: http://listserv.aoir.org/listinfo.cgi/air-l-aoir.org
Join the Association of Internet Researchers: http://www.aoir.org/
participants (1)
-
Craig Hamilton