Twitter data related to a link
Hello list, I have been trying to figure out how to get data about spread of a link on Twitter. I see a lot of tools that can provide data on hashtags, users, search terms but haven't figured out how to search for links on twitter. Any help is appreciated. Devayani
Hi Devayani, I think it is possible to do it with python and/or R. I use R with rtweet For my Twitter work. I haven't tracked a link myself but I suspect you can do it with the following code in R. #--------------- library(rtweet) api_token = rtweet::create_token(app = "app_name",consumer_key = "consumer_key",consumer_secret = "consumer_secret_key",access_token = "access_token_key",access_secret = "access_secret_key",set_renv = F) link = "your_link" link_tweets = search_tweets(q = link,n = 18000,type = "recent", include_rts = TRUE,retryonratelimit = T, token = api_token) write_as_csv(x = link_tweets,file_name = "link_tweets.csv",prepend_ids = T,na = "",fileEncoding = "UTF-8") #------------------ You need to create a developer account to connect to API. It's free and it is not a big hassle. Hope it helps ?? Best, Sina
-----Original Message----- From: Air-L <air-l-bounces@listserv.aoir.org> On Behalf Of Devayani Tirthali Sent: Friday, February 14, 2020 9:04 AM To: Air-L@listserv.aoir.org Subject: [Air-L] Twitter data related to a link
Hello list,
I have been trying to figure out how to get data about spread of a link on Twitter. I see a lot of tools that can provide data on hashtags, users, search terms but haven't figured out how to search for links on twitter. Any help is appreciated.
Devayani _______________________________________________ The Air-L@listserv.aoir.org mailing list is provided by the Association of Internet Researchers http://aoir.org Subscribe, change options or unsubscribe at: http://listserv.aoir.org/listinfo.cgi/air-l-aoir.org
Join the Association of Internet Researchers: http://www.aoir.org/
Hi Devayani, Sina’s right—any collection tool that is based on a search term will accept a URL just as easily as a hashtag. I’ve successfully used TAGS (https://tags.hawksey.info) to collect the sharing of a URL over time, and that rtweet code should work beautifully for a one-time search. Spencer
On Feb 14, 2020, at 05:00, Sina Furkan Özdemir <sina.ozdemir@ntnu.no> wrote:
Hi Devayani,
I think it is possible to do it with python and/or R. I use R with rtweet For my Twitter work. I haven't tracked a link myself but I suspect you can do it with the following code in R.
#--------------- library(rtweet)
api_token = rtweet::create_token(app = "app_name",consumer_key = "consumer_key",consumer_secret = "consumer_secret_key",access_token = "access_token_key",access_secret = "access_secret_key",set_renv = F)
link = "your_link"
link_tweets = search_tweets(q = link,n = 18000,type = "recent", include_rts = TRUE,retryonratelimit = T, token = api_token)
write_as_csv(x = link_tweets,file_name = "link_tweets.csv",prepend_ids = T,na = "",fileEncoding = "UTF-8") #------------------
You need to create a developer account to connect to API. It's free and it is not a big hassle.
Hope it helps ??
Best, Sina
-----Original Message----- From: Air-L <air-l-bounces@listserv.aoir.org> On Behalf Of Devayani Tirthali Sent: Friday, February 14, 2020 9:04 AM To: Air-L@listserv.aoir.org Subject: [Air-L] Twitter data related to a link
Hello list,
I have been trying to figure out how to get data about spread of a link on Twitter. I see a lot of tools that can provide data on hashtags, users, search terms but haven't figured out how to search for links on twitter. Any help is appreciated.
Devayani _______________________________________________ The Air-L@listserv.aoir.org mailing list is provided by the Association of Internet Researchers https://nam04.safelinks.protection.outlook.com/?url=http%3A%2F%2Faoir.org&data=02%7C01%7Cspencer.greenhalgh%40uky.edu%7C119a05d5587c4442b95b08d7b134c944%7C2b30530b69b64457b818481cb53d42ae%7C0%7C1%7C637172712589151885&sdata=60fvjyojl6WVLtnvZ%2FN%2FeFs0AUn0gNLNDnnNbswLjtI%3D&reserved=0 Subscribe, change options or unsubscribe at: https://nam04.safelinks.protection.outlook.com/?url=http%3A%2F%2Flistserv.aoir.org%2Flistinfo.cgi%2Fair-l-aoir.org&data=02%7C01%7Cspencer.greenhalgh%40uky.edu%7C119a05d5587c4442b95b08d7b134c944%7C2b30530b69b64457b818481cb53d42ae%7C0%7C1%7C637172712589151885&sdata=67R9iJODo9HKg4oT3kyLjc7uFYaCZc2ExSfuiL%2Fv%2FeA%3D&reserved=0
Join the Association of Internet Researchers: https://nam04.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.aoir.org%2F&data=02%7C01%7Cspencer.greenhalgh%40uky.edu%7C119a05d5587c4442b95b08d7b134c944%7C2b30530b69b64457b818481cb53d42ae%7C0%7C1%7C637172712589161876&sdata=L8fX26c%2BdHwS0CFbl50JFYmiJ2fY6PQD2ji83mKxbf0%3D&reserved=0
The Air-L@listserv.aoir.org mailing list is provided by the Association of Internet Researchers https://nam04.safelinks.protection.outlook.com/?url=http%3A%2F%2Faoir.org&data=02%7C01%7Cspencer.greenhalgh%40uky.edu%7C119a05d5587c4442b95b08d7b134c944%7C2b30530b69b64457b818481cb53d42ae%7C0%7C1%7C637172712589161876&sdata=zOZW4%2BIc6kjxwkkFqzL1LnqUoloDNTeGpWFwPQiCTKA%3D&reserved=0 Subscribe, change options or unsubscribe at: https://nam04.safelinks.protection.outlook.com/?url=http%3A%2F%2Flistserv.aoir.org%2Flistinfo.cgi%2Fair-l-aoir.org&data=02%7C01%7Cspencer.greenhalgh%40uky.edu%7C119a05d5587c4442b95b08d7b134c944%7C2b30530b69b64457b818481cb53d42ae%7C0%7C1%7C637172712589161876&sdata=ovMbih2j5pnV2WMY8A%2BO%2FSGGTEp4fL8nS48R8B3Ni6w%3D&reserved=0
Join the Association of Internet Researchers: https://nam04.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.aoir.org%2F&data=02%7C01%7Cspencer.greenhalgh%40uky.edu%7C119a05d5587c4442b95b08d7b134c944%7C2b30530b69b64457b818481cb53d42ae%7C0%7C1%7C637172712589161876&sdata=L8fX26c%2BdHwS0CFbl50JFYmiJ2fY6PQD2ji83mKxbf0%3D&reserved=0
Hi Spencer, I tried TAGS. The problem is it takes the main website url: for example https://www.altnews.in <https://www.altnews.in/did-bjp-lose-36-seats-by-a-margin-of-less-than-2000-votes-in-delhi-assembly-elections/> but not the actual url https://www.altnews.in/did-bjp-lose-36-seats-by-a-margin-of-less-than-2000-v... I have downloaded some data with website as the search term and trying to figure out if I can group the tweets based on the specific links. Unfortunately twitter gives the tiny URL rather than the actual URL so the only way I can think of right now is to go through each and tag it myself which is going to take a while. Would love to hear any other ideas. Devayani PS: I forgot to use reply all before. Realized it now. On Fri, Feb 14, 2020 at 5:13 PM Greenhalgh, Spencer P. < spencer.greenhalgh@uky.edu> wrote:
Hi Devayani,
Sina’s right—any collection tool that is based on a search term will accept a URL just as easily as a hashtag. I’ve successfully used TAGS ( https://tags.hawksey.info) to collect the sharing of a URL over time, and that rtweet code should work beautifully for a one-time search.
Spencer
On Feb 14, 2020, at 05:00, Sina Furkan Özdemir <sina.ozdemir@ntnu.no> wrote:
Hi Devayani,
I think it is possible to do it with python and/or R. I use R with rtweet For my Twitter work. I haven't tracked a link myself but I suspect you can do it with the following code in R.
#--------------- library(rtweet)
api_token = rtweet::create_token(app = "app_name",consumer_key = "consumer_key",consumer_secret = "consumer_secret_key",access_token = "access_token_key",access_secret = "access_secret_key",set_renv = F)
link = "your_link"
link_tweets = search_tweets(q = link,n = 18000,type = "recent", include_rts = TRUE,retryonratelimit = T, token = api_token)
write_as_csv(x = link_tweets,file_name = "link_tweets.csv",prepend_ids = T,na = "",fileEncoding = "UTF-8") #------------------
You need to create a developer account to connect to API. It's free and it is not a big hassle.
Hope it helps ??
Best, Sina
-----Original Message----- From: Air-L <air-l-bounces@listserv.aoir.org> On Behalf Of Devayani Tirthali Sent: Friday, February 14, 2020 9:04 AM To: Air-L@listserv.aoir.org Subject: [Air-L] Twitter data related to a link
Hello list,
I have been trying to figure out how to get data about spread of a link on Twitter. I see a lot of tools that can provide data on hashtags, users, search terms but haven't figured out how to search for links on twitter. Any help is appreciated.
Devayani _______________________________________________ The Air-L@listserv.aoir.org mailing list is provided by the Association of Internet Researchers https://nam04.safelinks.protection.outlook.com/?url=http%3A%2F%2Faoir.org&data=02%7C01%7Cspencer.greenhalgh%40uky.edu%7C119a05d5587c4442b95b08d7b134c944%7C2b30530b69b64457b818481cb53d42ae%7C0%7C1%7C637172712589151885&sdata=60fvjyojl6WVLtnvZ%2FN%2FeFs0AUn0gNLNDnnNbswLjtI%3D&reserved=0 Subscribe, change options or unsubscribe at: https://nam04.safelinks.protection.outlook.com/?url=http%3A%2F%2Flistserv.aoir.org%2Flistinfo.cgi%2Fair-l-aoir.org&data=02%7C01%7Cspencer.greenhalgh%40uky.edu%7C119a05d5587c4442b95b08d7b134c944%7C2b30530b69b64457b818481cb53d42ae%7C0%7C1%7C637172712589151885&sdata=67R9iJODo9HKg4oT3kyLjc7uFYaCZc2ExSfuiL%2Fv%2FeA%3D&reserved=0
Join the Association of Internet Researchers:
https://nam04.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.aoir.org%2F&data=02%7C01%7Cspencer.greenhalgh%40uky.edu%7C119a05d5587c4442b95b08d7b134c944%7C2b30530b69b64457b818481cb53d42ae%7C0%7C1%7C637172712589161876&sdata=L8fX26c%2BdHwS0CFbl50JFYmiJ2fY6PQD2ji83mKxbf0%3D&reserved=0 _______________________________________________ The Air-L@listserv.aoir.org mailing list is provided by the Association of Internet Researchers https://nam04.safelinks.protection.outlook.com/?url=http%3A%2F%2Faoir.org&data=02%7C01%7Cspencer.greenhalgh%40uky.edu%7C119a05d5587c4442b95b08d7b134c944%7C2b30530b69b64457b818481cb53d42ae%7C0%7C1%7C637172712589161876&sdata=zOZW4%2BIc6kjxwkkFqzL1LnqUoloDNTeGpWFwPQiCTKA%3D&reserved=0 Subscribe, change options or unsubscribe at: https://nam04.safelinks.protection.outlook.com/?url=http%3A%2F%2Flistserv.aoir.org%2Flistinfo.cgi%2Fair-l-aoir.org&data=02%7C01%7Cspencer.greenhalgh%40uky.edu%7C119a05d5587c4442b95b08d7b134c944%7C2b30530b69b64457b818481cb53d42ae%7C0%7C1%7C637172712589161876&sdata=ovMbih2j5pnV2WMY8A%2BO%2FSGGTEp4fL8nS48R8B3Ni6w%3D&reserved=0
Join the Association of Internet Researchers:
Hello Davayani, Have you tried using the “url:” operator? url: Performs a tokenized match on the expanded URLs of a Tweet. Tokens and phrases containing punctuation or special characters should be double-quoted (e.g. url:"/developer"). While generally not recommended, the operator can also match on a specific protocol, enclosed in double-quotes (e.g. url:"https://developer.twitter.com"). entities.urls.expanded_url I haven’t used it except a the domain level “url: altnews, url: in” Regards, Brenda
The Story of Goldilocks and Three Twitter’s APIs: A Pilot Study on Twitter Data Sources and Disclosure https://www.mdpi.com/1660-4601/17/3/864 Open access and very timely. On Sun, Feb 16, 2020 at 4:34 AM Brenda Moon <brenda@moon.net.au> wrote:
Hello Davayani,
Have you tried using the “url:” operator? url: Performs a tokenized match on the expanded URLs of a Tweet. Tokens and phrases containing punctuation or special characters should be double-quoted (e.g. url:"/developer"). While generally not recommended, the operator can also match on a specific protocol, enclosed in double-quotes (e.g. url:" https://developer.twitter.com"). entities.urls.expanded_url
I haven’t used it except a the domain level “url: altnews, url: in”
Regards,
Brenda _______________________________________________ The Air-L@listserv.aoir.org mailing list is provided by the Association of Internet Researchers http://aoir.org Subscribe, change options or unsubscribe at: http://listserv.aoir.org/listinfo.cgi/air-l-aoir.org
Join the Association of Internet Researchers: http://www.aoir.org/
-- Dr. Stuart W. Shulman Founder and CEO, Texifter Cell: 413-992-8513 LinkedIn: http://www.linkedin.com/in/stuartwshulman
Hi Brenda, I started collecting data using TAGS. Currently, it is working fine I think with specific links as well as the domain. The search for specific links started working when I removed http and started the link from www. Thanks for the article Stu. I have been looking for some information on how different methods of data collection on twitter affect the data I collect. So as you said it is timely. On Sun, Feb 16, 2020 at 3:04 PM Brenda Moon <brenda@moon.net.au> wrote:
Hello Davayani,
Have you tried using the “url:” operator? url: Performs a tokenized match on the expanded URLs of a Tweet. Tokens and phrases containing punctuation or special characters should be double-quoted (e.g. url:"/developer"). While generally not recommended, the operator can also match on a specific protocol, enclosed in double-quotes (e.g. url:" https://developer.twitter.com"). entities.urls.expanded_url
I haven’t used it except a the domain level “url: altnews, url: in”
Regards,
Brenda
Hi Devayani, If you use R check up this guide https://www.earthdatascience.org/courses/earth-analytics/get-data-using-apis... which is one of many out there. I use R and rtweet almost on a daily basis for my own research. Using rtweet search functions one can get up to 90 fields per tweet, several of them covering URLs, including actual source, if available. Hope this helps. Cheers, Raul On 2/14/20 5:00 AM, Sina Furkan Özdemir wrote:
Hi Devayani,
I think it is possible to do it with python and/or R. I use R with rtweet For my Twitter work. I haven't tracked a link myself but I suspect you can do it with the following code in R.
#--------------- library(rtweet)
api_token = rtweet::create_token(app = "app_name",consumer_key = "consumer_key",consumer_secret = "consumer_secret_key",access_token = "access_token_key",access_secret = "access_secret_key",set_renv = F)
link = "your_link"
link_tweets = search_tweets(q = link,n = 18000,type = "recent", include_rts = TRUE,retryonratelimit = T, token = api_token)
write_as_csv(x = link_tweets,file_name = "link_tweets.csv",prepend_ids = T,na = "",fileEncoding = "UTF-8") #------------------
You need to create a developer account to connect to API. It's free and it is not a big hassle.
Hope it helps ??
Best, Sina
-----Original Message----- From: Air-L <air-l-bounces@listserv.aoir.org> On Behalf Of Devayani Tirthali Sent: Friday, February 14, 2020 9:04 AM To: Air-L@listserv.aoir.org Subject: [Air-L] Twitter data related to a link
Hello list,
I have been trying to figure out how to get data about spread of a link on Twitter. I see a lot of tools that can provide data on hashtags, users, search terms but haven't figured out how to search for links on twitter. Any help is appreciated.
Devayani _______________________________________________ The Air-L@listserv.aoir.org mailing list is provided by the Association of Internet Researchers http://aoir.org Subscribe, change options or unsubscribe at: http://listserv.aoir.org/listinfo.cgi/air-l-aoir.org
Join the Association of Internet Researchers: http://www.aoir.org/
The Air-L@listserv.aoir.org mailing list is provided by the Association of Internet Researchers http://aoir.org Subscribe, change options or unsubscribe at: http://listserv.aoir.org/listinfo.cgi/air-l-aoir.org
Join the Association of Internet Researchers: http://www.aoir.org/
Hi Devayani, maybe give Facepager a try: https://github.com/strohne/Facepager There is a search preset for hashtags that can be used to search for links. The linked Twitter documentation might be helpful as well. Disclaimer: I'm an author of Facepager. Best Jakob Am 15. Februar 2020 16:06:49 MEZ schrieb Raul Zambrano <raulza@gmail.com>:
Hi Devayani,
If you use R check up this guide
https://www.earthdatascience.org/courses/earth-analytics/get-data-using-apis...
which is one of many out there.
I use R and rtweet almost on a daily basis for my own research.
Using rtweet search functions one can get up to 90 fields per tweet, several of them covering URLs, including actual source, if available.
Hope this helps.
Cheers, Raul
On 2/14/20 5:00 AM, Sina Furkan Özdemir wrote:
Hi Devayani,
I think it is possible to do it with python and/or R. I use R with rtweet For my Twitter work. I haven't tracked a link myself but I suspect you can do it with the following code in R.
#--------------- library(rtweet)
api_token = rtweet::create_token(app = "app_name",consumer_key = "consumer_key",consumer_secret = "consumer_secret_key",access_token = "access_token_key",access_secret = "access_secret_key",set_renv = F)
link = "your_link"
link_tweets = search_tweets(q = link,n = 18000,type = "recent", include_rts = TRUE,retryonratelimit = T, token = api_token)
write_as_csv(x = link_tweets,file_name = "link_tweets.csv",prepend_ids = T,na = "",fileEncoding = "UTF-8") #------------------
You need to create a developer account to connect to API. It's free and it is not a big hassle.
Hope it helps ??
Best, Sina
-----Original Message----- From: Air-L <air-l-bounces@listserv.aoir.org> On Behalf Of Devayani Tirthali Sent: Friday, February 14, 2020 9:04 AM To: Air-L@listserv.aoir.org Subject: [Air-L] Twitter data related to a link
Hello list,
I have been trying to figure out how to get data about spread of a link on Twitter. I see a lot of tools that can provide data on hashtags, users, search terms but haven't figured out how to search for links on twitter. Any help is appreciated.
Devayani _______________________________________________ The Air-L@listserv.aoir.org mailing list is provided by the Association of Internet Researchers http://aoir.org Subscribe, change options or unsubscribe at: http://listserv.aoir.org/listinfo.cgi/air-l-aoir.org
Join the Association of Internet Researchers: http://www.aoir.org/
The Air-L@listserv.aoir.org mailing list is provided by the Association of Internet Researchers http://aoir.org Subscribe, change options or unsubscribe at: http://listserv.aoir.org/listinfo.cgi/air-l-aoir.org
Join the Association of Internet Researchers: http://www.aoir.org/
The Air-L@listserv.aoir.org mailing list is provided by the Association of Internet Researchers http://aoir.org Subscribe, change options or unsubscribe at: http://listserv.aoir.org/listinfo.cgi/air-l-aoir.org
Join the Association of Internet Researchers: http://www.aoir.org/
-- Jakob Jünger University of Greifswald Institute of Political Science and Communication Studies Lohmeyer-Platz 3 17487 Greifswald, Germany Room: 3.16 (3rd floor) Phone : +49 3834 420 3444 or +49 173 860 8056 Email: jakob.juenger@uni-greifswald.de Web: http://www.ipk.uni-greifswald.de/
participants (7)
-
Brenda Moon -
Devayani Tirthali -
Greenhalgh, Spencer P. -
Jakob Jünger -
Raul Zambrano -
Shulman, Stu -
Sina Furkan Özdemir