Re: [Air-L] Searching Instagram/Twitter by keyword and narrowing to specific focus
Dear Natalie, I do not know any tool that would allow you to combine the two parameters (hashtag and geographical location) when searching for posts on Instagram. I collected data pre Dec 1st 2018 API change via Netlytic and am not sure Netlytic' access is still valid but I do recommend checking that out. Steps I recommend your friend takes: 1) It is very crucial to think carefully about what hashtag will be useful for your research. Better it is not a BIG hashtag. 2) Collect metadata using the hashtag(s) and dates as parameters. 3) Use longitude-latitude to detect locations yourself. Will be much more reliable that way. I played with scraping the number of followers after these steps and see if I can come up with a formula to make sense of number of likes and comments under each post but then I decided with the bots and fake accounts no finding will be reliable. So I gave up… If anyone has thoughts and solutions for that I would be glad to hear. Best, Ayşenur _____________________________________________ Ayşenur Benevento Faculty of Humanities and Social Sciences Abdullah Gül University On 28 Mar 2019 05:38 +0300, Natalie Rock <drnatalierock@gmail.com>, wrote: Community: This is a question from a colleague. Its out of my wheelhouse but probably not yours! He is interested in tracking dialogue on Twitter and Instagram about renewable energy, especially in light of the Green New Deal, but with the search focused on change makers (i.e. those over a certain number of followers). He has set up an account with Keyhole but their searches are not useful. Keyhole doesn't allow you to specify the search by geography; nor does it allow you to indicate which profiles to search. When he searches "renewable energy" (keyword and hashtag) he gets hundreds of thousands of returns, but many are unrelated to his focus. So, do you know a platform that enables: -searching/scraping by geography -specifying searches according to number of followers and which is the same or below Keyhole's current price range of $370/month? Thank you! Natalie _______________________________________________ The Air-L@listserv.aoir.org mailing list is provided by the Association of Internet Researchers http://aoir.org Subscribe, change options or unsubscribe at: http://listserv.aoir.org/listinfo.cgi/air-l-aoir.org Join the Association of Internet Researchers: http://www.aoir.org/
Hi! I have collected several large datasets via the Twitter API, using a small Python program written around the Tweepy library. Basically, I have used real-time streaming with pre-set hashtag filters. Works rather good. The good thing is that you could add a program-inter filter responsible for filtering the stream with your second criteria. In other words: 1) Use the Twitter API to collect tweets containing hashtags you are interested in; 2) Write your Python script so that each collected tweet is checked against your geographical criteria; 3) Only tweets passing (1) and (2) are printed to the output. You should, however, keep in mind that not all tweets contain GIS data. And the location data is not very reliable, either... HTH, Kimmo 2019-03-28, 09:46 +0000, Aysenur Ataman wrote:
Dear Natalie,
I do not know any tool that would allow you to combine the two parameters (hashtag and geographical location) when searching for posts on Instagram. I collected data pre Dec 1st 2018 API change via Netlytic and am not sure Netlytic' access is still valid but I do recommend checking that out.
Steps I recommend your friend takes:
1) It is very crucial to think carefully about what hashtag will be useful for your research. Better it is not a BIG hashtag.
2) Collect metadata using the hashtag(s) and dates as parameters.
3) Use longitude-latitude to detect locations yourself. Will be much more reliable that way.
I played with scraping the number of followers after these steps and see if I can come up with a formula to make sense of number of likes and comments under each post but then I decided with the bots and fake accounts no finding will be reliable. So I gave up… If anyone has thoughts and solutions for that I would be glad to hear.
Best, Ayşenur
_____________________________________________ Ayşenur Benevento Faculty of Humanities and Social Sciences Abdullah Gül University On 28 Mar 2019 05:38 +0300, Natalie Rock <drnatalierock@gmail.com>, wrote: Community:
This is a question from a colleague. Its out of my wheelhouse but probably not yours!
He is interested in tracking dialogue on Twitter and Instagram about renewable energy, especially in light of the Green New Deal, but with the search focused on change makers (i.e. those over a certain number of followers).
He has set up an account with Keyhole but their searches are not useful. Keyhole doesn't allow you to specify the search by geography; nor does it allow you to indicate which profiles to search. When he searches "renewable energy" (keyword and hashtag) he gets hundreds of thousands of returns, but many are unrelated to his focus. So, do you know a platform that enables:
-searching/scraping by geography -specifying searches according to number of followers
and which is the same or below Keyhole's current price range of $370/month?
Thank you! Natalie _______________________________________________ The Air-L@listserv.aoir.org mailing list is provided by the Association of Internet Researchers http://aoir.org Subscribe, change options or unsubscribe at: http://listserv.aoir.org/listinfo.cgi/air-l-aoir.org
Join the Association of Internet Researchers: http://www.aoir.org/ _______________________________________________ The Air-L@listserv.aoir.org mailing list is provided by the Association of Internet Researchers http://aoir.org Subscribe, change options or unsubscribe at: http://listserv.aoir.org/listinfo.cgi/air-l-aoir.org
Join the Association of Internet Researchers: http://www.aoir.org/
-searching/scraping by geography *>> no longer possible (academic access to historical premium Twitter data with geo enhancements) and it never really was unless you were willing to use a non-random, non-representative samples derived from the less than 1% of Twitter users who manually opt in to share geolocation data. I agree there are other ways to impute location from other elements in the metadata (sometimes, for example, the author bio) but there is a huge error rate and also quite incomplete data.* -specifying searches according to number of followers *>> This was formerly done in advance through historical metadata filters and is currently done now after the data is collected. Here the question is, do you mean at the time of collection or the time of analysis. If you mean time of collection, it is in the metadata. If you mean at the time of analysis, you need to be viewing the Tweets in the Twitter display to see the live count. Once you put Tweets in a spreadsheet or any other platform not connected to the Twitter display, you kill the connection to the mothership, and all the images, embeds, threads, live RT counts, and live likes, etc. In other words, you kill the Twitter data rendering it into a degraded post-Tweet **detritus**, and no longer the living, breathing network property we are all supposed to be interested in.* - Keyhole's current price range of $370/month *>> That's a lot. DiscoverText starts at $24/month for students and $49/month for faculty.* On Thu, Mar 28, 2019 at 5:46 AM Aysenur Ataman <aataman@gradcenter.cuny.edu> wrote:
Dear Natalie,
I do not know any tool that would allow you to combine the two parameters (hashtag and geographical location) when searching for posts on Instagram. I collected data pre Dec 1st 2018 API change via Netlytic and am not sure Netlytic' access is still valid but I do recommend checking that out.
Steps I recommend your friend takes:
1) It is very crucial to think carefully about what hashtag will be useful for your research. Better it is not a BIG hashtag.
2) Collect metadata using the hashtag(s) and dates as parameters.
3) Use longitude-latitude to detect locations yourself. Will be much more reliable that way.
I played with scraping the number of followers after these steps and see if I can come up with a formula to make sense of number of likes and comments under each post but then I decided with the bots and fake accounts no finding will be reliable. So I gave up… If anyone has thoughts and solutions for that I would be glad to hear.
Best, Ayşenur
_____________________________________________ Ayşenur Benevento Faculty of Humanities and Social Sciences Abdullah Gül University On 28 Mar 2019 05:38 +0300, Natalie Rock <drnatalierock@gmail.com>, wrote: Community:
This is a question from a colleague. Its out of my wheelhouse but probably not yours!
He is interested in tracking dialogue on Twitter and Instagram about renewable energy, especially in light of the Green New Deal, but with the search focused on change makers (i.e. those over a certain number of followers).
He has set up an account with Keyhole but their searches are not useful. Keyhole doesn't allow you to specify the search by geography; nor does it allow you to indicate which profiles to search. When he searches "renewable energy" (keyword and hashtag) he gets hundreds of thousands of returns, but many are unrelated to his focus. So, do you know a platform that enables:
-searching/scraping by geography -specifying searches according to number of followers
and which is the same or below Keyhole's current price range of $370/month?
Thank you! Natalie _______________________________________________ The Air-L@listserv.aoir.org mailing list is provided by the Association of Internet Researchers http://aoir.org Subscribe, change options or unsubscribe at: http://listserv.aoir.org/listinfo.cgi/air-l-aoir.org
Join the Association of Internet Researchers: http://www.aoir.org/ _______________________________________________ The Air-L@listserv.aoir.org mailing list is provided by the Association of Internet Researchers http://aoir.org Subscribe, change options or unsubscribe at: http://listserv.aoir.org/listinfo.cgi/air-l-aoir.org
Join the Association of Internet Researchers: http://www.aoir.org/
-- Dr. Stuart W. Shulman Founder and CEO, Texifter Cell: 413-992-8513 LinkedIn: http://www.linkedin.com/in/stuartwshulman
participants (3)
-
Aysenur Ataman -
Kimmo Elo -
Shulman, Stu