Update on simple historical Twitter hashtag searches
Dear Air-L, I thought I should follow up with a small but perhaps not so insignificant update on my query to the list from January (see below) where I outlined my difficulties in accessing 'historical' Twitter data, specifically a simple search of all tweets using a specific hashtag over around a two year period. Without getting too prematurely excited, it does seem that Twitter has listened to complaints that this rather basic search functionality should be provided to general users, not just big data analysis companies. Today when I searched for a specific hashtag in the ‘discover’ tab, I noticed that a user can indeed now go back (apparently) indefinitely and read all tweets using a particular hashtag. Very best, Scott On Mon, Jan 28, 2013 at 10:22 AM, Scott Rodgers <rodgers_scott at hotmail.com>wrote:
Hello all,
Apologies if I am about to jump in with a question discussed previously
(if so, I am happy to look through the archives) but I have been spending a
lot of time trying get a sense of how one can access 'historical' Twitter
data. A collaborator and I have a fairly small-scale project, for which
we'd like to look back about 2 years at maybe 1-3 specific hashtags all
related to a local neighbourhood campaign. From what I have garnered from a
lot of looking around, the only way to get at historical data is by
purchasing services from third parties with access to the Twitter fire
hose. The problem there is that such companies only seem to offer much more
extensive services than we need - we do not need, for instance, full-scale
quantitative analytics of tweet data, mood, and the like. We simply want to
generate a list of tweets using a certain hashtag over a certain period,
for the purposes of qualitative analysis (it would be linked with
interviews and other secondary information)
.
Any help or pointers would be much appreciated.
Very best,
Scott Rodgers
www.publiclysited.com
Hmm, just tested the search API with an old hashtag and it looks like it's still only trawling up to a week back in time. Maybe Twitter hasn't rolled out historical access to the API yet, or maybe I'm just #doinitwrong. ~DEEN On 3/8/2013 11:20 AM, Scott Rodgers wrote:
Dear Air-L,
I thought I should follow up with a small but perhaps not so insignificant update on my query to the list from January (see below) where I outlined my difficulties in accessing 'historical' Twitter data, specifically a simple search of all tweets using a specific hashtag over around a two year period.
Without getting too prematurely excited, it does seem that Twitter has listened to complaints that this rather basic search functionality should be provided to general users, not just big data analysis companies. Today when I searched for a specific hashtag in the ‘discover’ tab, I noticed that a user can indeed now go back (apparently) indefinitely and read all tweets using a particular hashtag.
Very best,
Scott
On Mon, Jan 28, 2013 at 10:22 AM, Scott Rodgers
<rodgers_scott at hotmail.com>wrote:
Hello all, Apologies if I am about to jump in with a question discussed previously
(if so, I am happy to look through the archives) but I have been spending a
lot of time trying get a sense of how one can access 'historical' Twitter
data. A collaborator and I have a fairly small-scale project, for which
we'd like to look back about 2 years at maybe 1-3 specific hashtags all
related to a local neighbourhood campaign. From what I have garnered from a
lot of looking around, the only way to get at historical data is by
purchasing services from third parties with access to the Twitter fire
hose. The problem there is that such companies only seem to offer much more
extensive services than we need - we do not need, for instance, full-scale
quantitative analytics of tweet data, mood, and the like. We simply want to
generate a list of tweets using a certain hashtag over a certain period,
for the purposes of qualitative analysis (it would be linked with
interviews and other secondary information) . Any help or pointers would be much appreciated.
Very best, Scott Rodgers www.publiclysited.com
_______________________________________________ The Air-L@listserv.aoir.org mailing list is provided by the Association of Internet Researchers http://aoir.org Subscribe, change options or unsubscribe at: http://listserv.aoir.org/listinfo.cgi/air-l-aoir.org
Join the Association of Internet Researchers: http://www.aoir.org/
-- Deen Freelon, Ph.D. Assistant Professor American University School of Communication Office: Asbury 228A dfreelon@gmail.com http://dfreelon.org
It's working for me on a bunch of different hashes. Make sure you're searching from the discover tab and you select "Tweets: all" at the top of the list (rather than "top"). It only loads one screen, so you have to continually scroll to load more (a limiting factor, imho). More bad news: does not work in Tweetdeck and not easily exportable. tnx, kc ____________________________________________ Katy Culver Assistant Professor School of Journalism & Mass Communication Associate Director Center for Journalism Ethics University of Wisconsin-Madison kbculver@wisc.edu 608-575-4082 On Mar 8, 2013, at 12:18 PM, Deen Freelon <dfreelon@gmail.com> wrote:
Hmm, just tested the search API with an old hashtag and it looks like it's still only trawling up to a week back in time. Maybe Twitter hasn't rolled out historical access to the API yet, or maybe I'm just #doinitwrong. ~DEEN
On 3/8/2013 11:20 AM, Scott Rodgers wrote:
Dear Air-L,
I thought I should follow up with a small but perhaps not so insignificant update on my query to the list from January (see below) where I outlined my difficulties in accessing 'historical' Twitter data, specifically a simple search of all tweets using a specific hashtag over around a two year period.
Without getting too prematurely excited, it does seem that Twitter has listened to complaints that this rather basic search functionality should be provided to general users, not just big data analysis companies. Today when I searched for a specific hashtag in the ‘discover’ tab, I noticed that a user can indeed now go back (apparently) indefinitely and read all tweets using a particular hashtag.
Very best,
Scott
On Mon, Jan 28, 2013 at 10:22 AM, Scott Rodgers
<rodgers_scott at hotmail.com>wrote:
Hello all, Apologies if I am about to jump in with a question discussed previously
(if so, I am happy to look through the archives) but I have been spending a
lot of time trying get a sense of how one can access 'historical' Twitter
data. A collaborator and I have a fairly small-scale project, for which
we'd like to look back about 2 years at maybe 1-3 specific hashtags all
related to a local neighbourhood campaign. From what I have garnered from a
lot of looking around, the only way to get at historical data is by
purchasing services from third parties with access to the Twitter fire
hose. The problem there is that such companies only seem to offer much more
extensive services than we need - we do not need, for instance, full-scale
quantitative analytics of tweet data, mood, and the like. We simply want to
generate a list of tweets using a certain hashtag over a certain period,
for the purposes of qualitative analysis (it would be linked with
interviews and other secondary information) . Any help or pointers would be much appreciated.
Very best, Scott Rodgers www.publiclysited.com
_______________________________________________ The Air-L@listserv.aoir.org mailing list is provided by the Association of Internet Researchers http://aoir.org Subscribe, change options or unsubscribe at: http://listserv.aoir.org/listinfo.cgi/air-l-aoir.org
Join the Association of Internet Researchers: http://www.aoir.org/
-- Deen Freelon, Ph.D. Assistant Professor American University School of Communication Office: Asbury 228A dfreelon@gmail.com http://dfreelon.org
_______________________________________________ The Air-L@listserv.aoir.org mailing list is provided by the Association of Internet Researchers http://aoir.org Subscribe, change options or unsubscribe at: http://listserv.aoir.org/listinfo.cgi/air-l-aoir.org
Join the Association of Internet Researchers: http://www.aoir.org/
Hi, amazing, I just checked and indeed I get results that go years back! Best --u At 13:18 Uhr -0500 8.3.2013, Deen Freelon wrote:
Hmm, just tested the search API with an old hashtag and it looks like it's still only trawling up to a week back in time. Maybe Twitter hasn't rolled out historical access to the API yet, or maybe I'm just #doinitwrong. ~DEEN
On 3/8/2013 11:20 AM, Scott Rodgers wrote:
Dear Air-L,
I thought I should follow up with a small but perhaps not so insignificant update on my query to the list from January (see below) where I outlined my difficulties in accessing 'historical' Twitter data, specifically a simple search of all tweets using a specific hashtag over around a two year period.
Without getting too prematurely excited, it does seem that Twitter has listened to complaints that this rather basic search functionality should be provided to general users, not just big data analysis companies. Today when I searched for a specific hashtag in the 'discover' tab, I noticed that a user can indeed now go back (apparently) indefinitely and read all tweets using a particular hashtag.
Very best,
Scott
On Mon, Jan 28, 2013 at 10:22 AM, Scott Rodgers
<rodgers_scott at hotmail.com>wrote:
Hello all, Apologies if I am about to jump in with a question discussed previously
(if so, I am happy to look through the archives) but I have been spending a
lot of time trying get a sense of how one can access 'historical' Twitter
data. A collaborator and I have a fairly small-scale project, for which
we'd like to look back about 2 years at maybe 1-3 specific hashtags all
related to a local neighbourhood campaign. From what I have garnered from a
lot of looking around, the only way to get at historical data is by
purchasing services from third parties with access to the Twitter fire
hose. The problem there is that such companies only seem to offer much more
extensive services than we need - we do not need, for instance, full-scale
quantitative analytics of tweet data, mood, and the like. We simply want to
generate a list of tweets using a certain hashtag over a certain period,
for the purposes of qualitative analysis (it would be linked with
interviews and other secondary information) . Any help or pointers would be much appreciated.
Very best, Scott Rodgers www.publiclysited.com
_______________________________________________ The Air-L@listserv.aoir.org mailing list is provided by the Association of Internet Researchers http://aoir.org Subscribe, change options or unsubscribe at: http://listserv.aoir.org/listinfo.cgi/air-l-aoir.org
Join the Association of Internet Researchers: http://www.aoir.org/
-- Deen Freelon, Ph.D. Assistant Professor American University School of Communication Office: Asbury 228A dfreelon@gmail.com http://dfreelon.org
_______________________________________________ The Air-L@listserv.aoir.org mailing list is provided by the Association of Internet Researchers http://aoir.org Subscribe, change options or unsubscribe at: http://listserv.aoir.org/listinfo.cgi/air-l-aoir.org
Join the Association of Internet Researchers: http://www.aoir.org/
I would bet it's the former - precisely to improve the search experience for (web) users while keeping the choke on API-enabled data access and storage? By the way: I note that search in the official iOS app doesn't have the "all" search results function, which is very far from ideal in many circumstances. For example, in a crisis situation there are times when I might want to view all tweets for a search (a location name for example) not just all the popular or socially "relevant" tweets. On 08/03/2013, at 10:19, "Deen Freelon" <dfreelon@gmail.com> wrote:
Hmm, just tested the search API with an old hashtag and it looks like it's still only trawling up to a week back in time. Maybe Twitter hasn't rolled out historical access to the API yet, or maybe I'm just #doinitwrong. ~DEEN
On 3/8/2013 11:20 AM, Scott Rodgers wrote:
Dear Air-L,
I thought I should follow up with a small but perhaps not so insignificant update on my query to the list from January (see below) where I outlined my difficulties in accessing 'historical' Twitter data, specifically a simple search of all tweets using a specific hashtag over around a two year period.
Without getting too prematurely excited, it does seem that Twitter has listened to complaints that this rather basic search functionality should be provided to general users, not just big data analysis companies. Today when I searched for a specific hashtag in the ‘discover’ tab, I noticed that a user can indeed now go back (apparently) indefinitely and read all tweets using a particular hashtag.
Very best,
Scott
On Mon, Jan 28, 2013 at 10:22 AM, Scott Rodgers
<rodgers_scott at hotmail.com>wrote:
Hello all, Apologies if I am about to jump in with a question discussed previously
(if so, I am happy to look through the archives) but I have been spending a
lot of time trying get a sense of how one can access 'historical' Twitter
data. A collaborator and I have a fairly small-scale project, for which
we'd like to look back about 2 years at maybe 1-3 specific hashtags all
related to a local neighbourhood campaign. From what I have garnered from a
lot of looking around, the only way to get at historical data is by
purchasing services from third parties with access to the Twitter fire
hose. The problem there is that such companies only seem to offer much more
extensive services than we need - we do not need, for instance, full-scale
quantitative analytics of tweet data, mood, and the like. We simply want to
generate a list of tweets using a certain hashtag over a certain period,
for the purposes of qualitative analysis (it would be linked with
interviews and other secondary information) . Any help or pointers would be much appreciated.
Very best, Scott Rodgers www.publiclysited.com
_______________________________________________ The Air-L@listserv.aoir.org mailing list is provided by the Association of Internet Researchers http://aoir.org Subscribe, change options or unsubscribe at: http://listserv.aoir.org/listinfo.cgi/air-l-aoir.org
Join the Association of Internet Researchers: http://www.aoir.org/
-- Deen Freelon, Ph.D. Assistant Professor American University School of Communication Office: Asbury 228A dfreelon@gmail.com http://dfreelon.org
_______________________________________________ The Air-L@listserv.aoir.org mailing list is provided by the Association of Internet Researchers http://aoir.org Subscribe, change options or unsubscribe at: http://listserv.aoir.org/listinfo.cgi/air-l-aoir.org
Join the Association of Internet Researchers: http://www.aoir.org/
I checked this with #nickcleggsfault - around three years old and the origin is known. It brought up a lot of tweets, back to the very day the meme started but not back to the very first use of the hashtag. Be wary if using it that it may not be gathering all of the tweets/may still have a time limit on how far back you can search. Cassian
From: je.burgess@qut.edu.au To: dfreelon@gmail.com Date: Sat, 9 Mar 2013 05:06:25 +1000 CC: air-l@listserv.aoir.org Subject: Re: [Air-L] Update on simple historical Twitter hashtag searches
I would bet it's the former - precisely to improve the search experience for (web) users while keeping the choke on API-enabled data access and storage?
By the way:
I note that search in the official iOS app doesn't have the "all" search results function, which is very far from ideal in many circumstances. For example, in a crisis situation there are times when I might want to view all tweets for a search (a location name for example) not just all the popular or socially "relevant" tweets.
On 08/03/2013, at 10:19, "Deen Freelon" <dfreelon@gmail.com> wrote:
Hmm, just tested the search API with an old hashtag and it looks like it's still only trawling up to a week back in time. Maybe Twitter hasn't rolled out historical access to the API yet, or maybe I'm just #doinitwrong. ~DEEN
On 3/8/2013 11:20 AM, Scott Rodgers wrote:
Dear Air-L,
I thought I should follow up with a small but perhaps not so insignificant update on my query to the list from January (see below) where I outlined my difficulties in accessing 'historical' Twitter data, specifically a simple search of all tweets using a specific hashtag over around a two year period.
Without getting too prematurely excited, it does seem that Twitter has listened to complaints that this rather basic search functionality should be provided to general users, not just big data analysis companies. Today when I searched for a specific hashtag in the ‘discover’ tab, I noticed that a user can indeed now go back (apparently) indefinitely and read all tweets using a particular hashtag.
Very best,
Scott
On Mon, Jan 28, 2013 at 10:22 AM, Scott Rodgers
<rodgers_scott at hotmail.com>wrote:
Hello all, Apologies if I am about to jump in with a question discussed previously
(if so, I am happy to look through the archives) but I have been spending a
lot of time trying get a sense of how one can access 'historical' Twitter
data. A collaborator and I have a fairly small-scale project, for which
we'd like to look back about 2 years at maybe 1-3 specific hashtags all
related to a local neighbourhood campaign. From what I have garnered from a
lot of looking around, the only way to get at historical data is by
purchasing services from third parties with access to the Twitter fire
hose. The problem there is that such companies only seem to offer much more
extensive services than we need - we do not need, for instance, full-scale
quantitative analytics of tweet data, mood, and the like. We simply want to
generate a list of tweets using a certain hashtag over a certain period,
for the purposes of qualitative analysis (it would be linked with
interviews and other secondary information) . Any help or pointers would be much appreciated.
Very best, Scott Rodgers www.publiclysited.com
_______________________________________________ The Air-L@listserv.aoir.org mailing list is provided by the Association of Internet Researchers http://aoir.org Subscribe, change options or unsubscribe at: http://listserv.aoir.org/listinfo.cgi/air-l-aoir.org
Join the Association of Internet Researchers: http://www.aoir.org/
-- Deen Freelon, Ph.D. Assistant Professor American University School of Communication Office: Asbury 228A dfreelon@gmail.com http://dfreelon.org
_______________________________________________ The Air-L@listserv.aoir.org mailing list is provided by the Association of Internet Researchers http://aoir.org Subscribe, change options or unsubscribe at: http://listserv.aoir.org/listinfo.cgi/air-l-aoir.org
Join the Association of Internet Researchers: http://www.aoir.org/
The Air-L@listserv.aoir.org mailing list is provided by the Association of Internet Researchers http://aoir.org Subscribe, change options or unsubscribe at: http://listserv.aoir.org/listinfo.cgi/air-l-aoir.org
Join the Association of Internet Researchers: http://www.aoir.org/
Thanks for that Cassian, For the hashtags I'm looking at, I don't know the precise origin so have no way to know how compete the results from these search queries are. Now I know to be cautious. Slightly frustrating, of course. I'd like to exist in a Twitter world where 'show all' really means all. Scott
From: cassian@hotmail.co.uk To: je.burgess@qut.edu.au; dfreelon@gmail.com Date: Tue, 12 Mar 2013 10:19:22 +0000 CC: air-l@listserv.aoir.org Subject: Re: [Air-L] Update on simple historical Twitter hashtag searches a warning
I checked this with #nickcleggsfault - around three years old and the origin is known. It brought up a lot of tweets, back to the very day the meme started but not back to the very first use of the hashtag. Be wary if using it that it may not be gathering all of the tweets/may still have a time limit on how far back you can search.
Cassian
From: je.burgess@qut.edu.au To: dfreelon@gmail.com Date: Sat, 9 Mar 2013 05:06:25 +1000 CC: air-l@listserv.aoir.org Subject: Re: [Air-L] Update on simple historical Twitter hashtag searches
I would bet it's the former - precisely to improve the search experience for (web) users while keeping the choke on API-enabled data access and storage?
By the way:
I note that search in the official iOS app doesn't have the "all" search results function, which is very far from ideal in many circumstances. For example, in a crisis situation there are times when I might want to view all tweets for a search (a location name for example) not just all the popular or socially "relevant" tweets.
On 08/03/2013, at 10:19, "Deen Freelon" <dfreelon@gmail.com> wrote:
Hmm, just tested the search API with an old hashtag and it looks like it's still only trawling up to a week back in time. Maybe Twitter hasn't rolled out historical access to the API yet, or maybe I'm just #doinitwrong. ~DEEN
On 3/8/2013 11:20 AM, Scott Rodgers wrote:
Dear Air-L,
I thought I should follow up with a small but perhaps not so insignificant update on my query to the list from January (see below) where I outlined my difficulties in accessing 'historical' Twitter data, specifically a simple search of all tweets using a specific hashtag over around a two year period.
Without getting too prematurely excited, it does seem that Twitter has listened to complaints that this rather basic search functionality should be provided to general users, not just big data analysis companies. Today when I searched for a specific hashtag in the ‘discover’ tab, I noticed that a user can indeed now go back (apparently) indefinitely and read all tweets using a particular hashtag.
Very best,
Scott
On Mon, Jan 28, 2013 at 10:22 AM, Scott Rodgers
<rodgers_scott at hotmail.com>wrote:
Hello all, Apologies if I am about to jump in with a question discussed previously
(if so, I am happy to look through the archives) but I have been spending a
lot of time trying get a sense of how one can access 'historical' Twitter
data. A collaborator and I have a fairly small-scale project, for which
we'd like to look back about 2 years at maybe 1-3 specific hashtags all
related to a local neighbourhood campaign. From what I have garnered from a
lot of looking around, the only way to get at historical data is by
purchasing services from third parties with access to the Twitter fire
hose. The problem there is that such companies only seem to offer much more
extensive services than we need - we do not need, for instance, full-scale
quantitative analytics of tweet data, mood, and the like. We simply want to
generate a list of tweets using a certain hashtag over a certain period,
for the purposes of qualitative analysis (it would be linked with
interviews and other secondary information) . Any help or pointers would be much appreciated.
Very best, Scott Rodgers www.publiclysited.com
_______________________________________________ The Air-L@listserv.aoir.org mailing list is provided by the Association of Internet Researchers http://aoir.org Subscribe, change options or unsubscribe at: http://listserv.aoir.org/listinfo.cgi/air-l-aoir.org
Join the Association of Internet Researchers: http://www.aoir.org/
-- Deen Freelon, Ph.D. Assistant Professor American University School of Communication Office: Asbury 228A dfreelon@gmail.com http://dfreelon.org
_______________________________________________ The Air-L@listserv.aoir.org mailing list is provided by the Association of Internet Researchers http://aoir.org Subscribe, change options or unsubscribe at: http://listserv.aoir.org/listinfo.cgi/air-l-aoir.org
Join the Association of Internet Researchers: http://www.aoir.org/
The Air-L@listserv.aoir.org mailing list is provided by the Association of Internet Researchers http://aoir.org Subscribe, change options or unsubscribe at: http://listserv.aoir.org/listinfo.cgi/air-l-aoir.org
Join the Association of Internet Researchers: http://www.aoir.org/
_______________________________________________ The Air-L@listserv.aoir.org mailing list is provided by the Association of Internet Researchers http://aoir.org Subscribe, change options or unsubscribe at: http://listserv.aoir.org/listinfo.cgi/air-l-aoir.org
Join the Association of Internet Researchers: http://www.aoir.org/
participants (6)
-
Cassian Vian -
Deen Freelon -
Jean Burgess -
Katy Culver -
Scott Rodgers -
Ulf-Dietrich Reips