Dear Muira, I would like to push back a little against something you are saying in your message, namely that Webrecorder (or any other tool for that matter, including the YouTube Data Tools mentioned below, which I develop and maintain) allows you to create a "reliable archive". I think that this is a problematic starting point, in particular for data retrieved from corporately owned platforms via data interfaces designed for purposes other than research goals. The YouTube API, for example is highly reactive to privacy settings, allows admins to delete and moderate comments, and does not always provide all comments for thread chains for a variety of reasons. I would argue that the "reliability" of data needs to be established for each research project sparately and cannot be delegated to a tool. This validation (which includes an understanding of limitations) can be done, for example, through comparison between data and interface, by using different API credentials for retrieval, and by supporting findings through two or more approaches. More specifically, for any data retrieved from corporate platforms, research should - in my view - include robust data critique. My colleagues and I have tried to sketch such a process for Facebook API data here: http://journals.sagepub.com/doi/abs/10.1177/2053951715614980 best wishes, Bernhard --- Bernhard Rieder | Associate Professor | New Media and Digital Culture University of Amsterdam | Turfdraagsterpad 9 | 1012 XT Amsterdam | The Netherlands http://thepoliticsofsystems.net | http://labs.polsys.net | https://www.digitalmethods.net | @RiederB

On 17 Jan 2018, at 14:52, Muira McCammon <muira.n.mccammon@gmail.com> wrote:

Hi, Philippa!

This is a frustrating issue, to be sure.

Though I am not very well-acquainted with YouTube's framework/structure and know nothing of the Digital Methods Initiative, this is what I can offer in terms of advice for future research: Webrecorder.io, an open-source project by Rhizome at the New Museum, gives users 5GB of space, which you can use to create your own high fidelity web archives. To give you a sense of how this works, last evening, to test things out, I went to YouTube and made a web archive <https://webrecorder.io/muira/youtubetest/20180117032525/https://www.youtube.com/watch?v=GwN-WixHkAw> of this video <https://www.youtube.com/watch?v=GwN-WixHkAw> of NPR's Tiny Desk Concert featuring Dessa; as you can see, I included the comments in the archive I created.

What Webrecorder would let you do is create a reliable archive of the YouTube comments you want to incorporate into your research. That archive would then be accessible, and you could link to it, if need be, long after the comments disappeared and/or were deleted. Normally, if you were just interested in archiving static material (that is, the comments and not say, video content), I would suggest you use Perma.cc, an archiving tool created by the Harvard Law Library Innovation Lab. The problem there is two-fold: (#1) you'd only get 10 links a month, which isn't much, and (#2) based on my own use, it just doesn't deal well with any YouTube content, including, the comments. I should say here that there are other web archiving tools out there, but Webrecorder.io and Perma.cc are what I know best. Others on this listserv may have other/better suggestions!

If you go down this road, some folks on this listserv might urge you to grapple with some of the ethical issues that come with creating web archives of YouTube user comments, esp. those that users later delete deliberately; maybe someone will weigh in on that matter. Indeed, perhaps someone on this listserv has written about the ethics of web archiving YouTube comments!

That being said, at a technical level, if you're trying to keep track of thousands of comments or hundreds of YouTube pages, you will probably hit the 5GB data limit on Webrecorder sooner than later. Good news is that Rhizome <http://rhizome.org/> just received a grant <https://news.artnet.com/art-world/rhizome-1-million-mellon-foundation-grant-1197608> of $1 million, so perhaps they'll bump up the space they give new users. (In full disclosure, I should say that I received a microgrant from Rhizome back in 2017, so I am, in more ways than one, a fan of what they do. Happy to help anyone here figure out how to use their tool.)

Lastly, I'd be curious to know if anyone here, apart from myself, is actively using Webrecorder in their research, especially if it's qualitative and engaging with deletion on social media platforms. Would love to talk to you, if you exist!

All my best, Muira McCammon

Muira McCammon Ph.D. Student <https://www.asc.upenn.edu/people/students/muira-mccammon> / Journalist <http://muira.me/> Signal: Available upon request Annenberg School for Communication University of Pennsylvania

On Tue, Jan 16, 2018 at 5:22 PM, Philippa Smith <philippa.smith@aut.ac.nz> wrote:

...

Hi Everyone

My latest research project involves linguistic analysis of comments/replies under a specific youtube video. Interestingly I cut and pasted the comments from a particular youtube url in July last year and saved them in a word document. However in going back to update this in 2018 I find that those original comments are no longer accessible in that the site does not automatically load more comments beyond the last three months. This leave me with an incomplete data set. I find this situation rather intriguing. If anyone who is better informed than I am might enlighten me as to whether this just reflects a change in youtube policy or might relate to settings of the youtube poster - that would be helpful. And if there is any solution - even better!

Kind regards

Philippa

[AUT]<https://www.aut.ac.nz/>

Philippa Smith Senior Lecturer ICDC Auckland University of Technology

[QS rankings]<https://www.aut.ac.nz/qs>

[Facebook]<https://www.aut.ac.nz/facebook>[Twitter]<https:// www.aut.ac.nz/twitter>[YouTube]<https://www.aut.ac.nz/youtube>

P 09 921 9999 ext 8276 E philippa.smith@aut.ac.nz<mailto: philippa.smith@aut.ac.nz> W aut.ac.nz<https://www.aut.ac.nz/>

[https://www.aut.ac.nz/__data/assets/image/0011/571817/teal-strip.gif]

_______________________________________________ The Air-L@listserv.aoir.org mailing list is provided by the Association of Internet Researchers http://aoir.org Subscribe, change options or unsubscribe at: http://listserv.aoir.org/ listinfo.cgi/air-l-aoir.org

Join the Association of Internet Researchers: http://www.aoir.org/

-- *Muira McCammon* *@muira_mccammon <https://twitter.com/muira_mccammon>*

Recently out: "Persisting in Dark Times: Lessons from a War Crimes Researcher <http://howwegettonext.com/persisting-in-dark-times-lessons-from-a-war-crimes-researcher-8b3504f4b169>" in *How We Get to Next * _______________________________________________ The Air-L@listserv.aoir.org mailing list is provided by the Association of Internet Researchers http://aoir.org Subscribe, change options or unsubscribe at: http://listserv.aoir.org/listinfo.cgi/air-l-aoir.org

Join the Association of Internet Researchers: http://www.aoir.org/

Hi All – As always I am grateful for the goodwill and willingness of AOIR e-listers to offer advice and help with questions I pose. Thanks to all of you who have responded to my query about ‘lost’ youtube comments. I hope others will also benefit from these posts and the conversation that has ensued either with the suggested techniques to use (web archiving with WebRecorder or trying API), the thoughts about “robust data critique”, or the ethics involved with analysis ie should you include a comment in your data from someone who might have deleted it later (and how would you know that they were the one to take this action anyway?). While I will follow up the various suggestions – there is the possibility that the ‘lost’ comments I’m looking for might have been purposely deleted by the ‘owners’ of the youtube clip (if that can be done through settings?) or even done by youtube itself? While some of the comments I was looking at were inappropriate, not all were. This does raise an issue concerning free speech and censorship and if you offer someone the opportunity to voice their opinion on the internet, do you also have the right to take that away (if it isn’t hate speech/abuse)? … but then that is a whole different matter to consider. Thanks everyone – Philippa [AUT]<https://www.aut.ac.nz/> Philippa Smith Senior Lecturer ICDC Auckland University of Technology [QS rankings]<https://www.aut.ac.nz/qs> [Facebook]<https://www.aut.ac.nz/facebook>[Twitter]<https://www.aut.ac.nz/twitter>[YouTube]<https://www.aut.ac.nz/youtube> P 09 921 9999 ext 8276 E philippa.smith@aut.ac.nz<mailto:philippa.smith@aut.ac.nz> W aut.ac.nz<https://www.aut.ac.nz/> [https://www.aut.ac.nz/__data/assets/image/0011/571817/teal-strip.gif] From: Muira McCammon [mailto:muira.n.mccammon@gmail.com] Sent: Thursday, 18 January 2018 5:09 a.m. To: Bernhard Rieder <berno.rieder@gmail.com> Cc: Air list (air-l@listserv.aoir.org) <air-l@listserv.aoir.org>; Philippa Smith <philippa.smith@aut.ac.nz>; muira.mccammon@asc.upenn.edu Subject: Re: [Air-L] Lost youtube comments Hi, Dr. Rieder et al., Thank you for this prompt response. I'd like to start off by saying that I do not disagree with your points. In my initial email, I was a bit sloppy in that I did not contextualize what I meant by reliable archive. What I meant to imply, is that I find Webrecorder's archives to be reliable in the sense that as a tool, I have found that it works well, that, for example, I have never had issues in accessing the archives that I have made; it is dependable in this regard. I have some degree of trust in the creators of the tool. I also have reason to believe that the web team of Webrecorder is not maliciously manipulating the archives that I make. I've found some web archiving tools to be particularly shoddy; many lack funding, many lack technical support teams, some appear to be on the verge of going under. Of course, it's still possible, one day in my lifetime, that Webrecorder will cease to function, and that's a concern. I'm in complete agreement that when data is retrieved from corporate platforms, research should include robust data critique, and I appreciate the link! I also believe that when data is archived from corporate platforms, researchers should endeavor to include a robust critique of the archiving tool used - in part because many of the tools tend to work/function differently. Again, sorry for any confusion in my initial post. Best, Muira On Wed, Jan 17, 2018 at 10:22 AM, Bernhard Rieder <berno.rieder@gmail.com<mailto:berno.rieder@gmail.com>> wrote: Dear Muira, I would like to push back a little against something you are saying in your message, namely that Webrecorder (or any other tool for that matter, including the YouTube Data Tools mentioned below, which I develop and maintain) allows you to create a "reliable archive". I think that this is a problematic starting point, in particular for data retrieved from corporately owned platforms via data interfaces designed for purposes other than research goals. The YouTube API, for example is highly reactive to privacy settings, allows admins to delete and moderate comments, and does not always provide all comments for thread chains for a variety of reasons. I would argue that the "reliability" of data needs to be established for each research project sparately and cannot be delegated to a tool. This validation (which includes an understanding of limitations) can be done, for example, through comparison between data and interface, by using different API credentials for retrieval, and by supporting findings through two or more approaches. More specifically, for any data retrieved from corporate platforms, research should - in my view - include robust data critique. My colleagues and I have tried to sketch such a process for Facebook API data here: http://journals.sagepub.com/doi/abs/10.1177/2053951715614980 best wishes, Bernhard --- Bernhard Rieder | Associate Professor | New Media and Digital Culture University of Amsterdam | Turfdraagsterpad 9 | 1012 XT Amsterdam | The Netherlands http://thepoliticsofsystems.net | http://labs.polsys.net | https://www.digitalmethods.net | @RiederB

On 17 Jan 2018, at 14:52, Muira McCammon <muira.n.mccammon@gmail.com<mailto:muira.n.mccammon@gmail.com>> wrote:

Hi, Philippa!

This is a frustrating issue, to be sure.

Though I am not very well-acquainted with YouTube's framework/structure and know nothing of the Digital Methods Initiative, this is what I can offer in terms of advice for future research: Webrecorder.io, an open-source project by Rhizome at the New Museum, gives users 5GB of space, which you can use to create your own high fidelity web archives. To give you a sense of how this works, last evening, to test things out, I went to YouTube and made a web archive <https://webrecorder.io/muira/youtubetest/20180117032525/https://www.youtube.com/watch?v=GwN-WixHkAw<https://webrecorder.io/muira/youtubetest/20180117032525/https:/www.youtube.com/watch?v=GwN-WixHkAw>> of this video <https://www.youtube.com/watch?v=GwN-WixHkAw> of NPR's Tiny Desk Concert featuring Dessa; as you can see, I included the comments in the archive I created.

What Webrecorder would let you do is create a reliable archive of the YouTube comments you want to incorporate into your research. That archive would then be accessible, and you could link to it, if need be, long after the comments disappeared and/or were deleted. Normally, if you were just interested in archiving static material (that is, the comments and not say, video content), I would suggest you use Perma.cc, an archiving tool created by the Harvard Law Library Innovation Lab. The problem there is two-fold: (#1) you'd only get 10 links a month, which isn't much, and (#2) based on my own use, it just doesn't deal well with any YouTube content, including, the comments. I should say here that there are other web archiving tools out there, but Webrecorder.io and Perma.cc are what I know best. Others on this listserv may have other/better suggestions!

If you go down this road, some folks on this listserv might urge you to grapple with some of the ethical issues that come with creating web archives of YouTube user comments, esp. those that users later delete deliberately; maybe someone will weigh in on that matter. Indeed, perhaps someone on this listserv has written about the ethics of web archiving YouTube comments!

That being said, at a technical level, if you're trying to keep track of thousands of comments or hundreds of YouTube pages, you will probably hit the 5GB data limit on Webrecorder sooner than later. Good news is that Rhizome <http://rhizome.org/> just received a grant <https://news.artnet.com/art-world/rhizome-1-million-mellon-foundation-grant-1197608> of $1 million, so perhaps they'll bump up the space they give new users. (In full disclosure, I should say that I received a microgrant from Rhizome back in 2017, so I am, in more ways than one, a fan of what they do. Happy to help anyone here figure out how to use their tool.)

Lastly, I'd be curious to know if anyone here, apart from myself, is actively using Webrecorder in their research, especially if it's qualitative and engaging with deletion on social media platforms. Would love to talk to you, if you exist!

All my best, Muira McCammon

Muira McCammon Ph.D. Student <https://www.asc.upenn.edu/people/students/muira-mccammon> / Journalist <http://muira.me/> Signal: Available upon request Annenberg School for Communication University of Pennsylvania

On Tue, Jan 16, 2018 at 5:22 PM, Philippa Smith <philippa.smith@aut.ac.nz<mailto:philippa.smith@aut.ac.nz>> wrote:

...

Hi Everyone

My latest research project involves linguistic analysis of comments/replies under a specific youtube video. Interestingly I cut and pasted the comments from a particular youtube url in July last year and saved them in a word document. However in going back to update this in 2018 I find that those original comments are no longer accessible in that the site does not automatically load more comments beyond the last three months. This leave me with an incomplete data set. I find this situation rather intriguing. If anyone who is better informed than I am might enlighten me as to whether this just reflects a change in youtube policy or might relate to settings of the youtube poster - that would be helpful. And if there is any solution - even better!

Kind regards

Philippa

[AUT]<https://www.aut.ac.nz/>

Philippa Smith Senior Lecturer ICDC Auckland University of Technology

[QS rankings]<https://www.aut.ac.nz/qs>

[Facebook]<https://www.aut.ac.nz/facebook>[Twitter]<https:// <https://%0b>>> www.aut.ac.nz/twitter<http://www.aut.ac.nz/twitter>>[YouTube]<https://www.aut.ac.nz/youtube>

P 09 921 9999 ext 8276 E philippa.smith@aut.ac.nz<mailto:philippa.smith@aut.ac.nz><mailto: philippa.smith@aut.ac.nz<mailto:philippa.smith@aut.ac.nz>> W aut.ac.nz<http://aut.ac.nz><https://www.aut.ac.nz/>

[https://www.aut.ac.nz/__data/assets/image/0011/571817/teal-strip.gif]

_______________________________________________ The Air-L@listserv.aoir.org<mailto:Air-L@listserv.aoir.org> mailing list is provided by the Association of Internet Researchers http://aoir.org Subscribe, change options or unsubscribe at: http://listserv.aoir.org/ listinfo.cgi/air-l-aoir.org<http://air-l-aoir.org>

Join the Association of Internet Researchers: http://www.aoir.org/

-- *Muira McCammon* *@muira_mccammon <https://twitter.com/muira_mccammon>*

Recently out: "Persisting in Dark Times: Lessons from a War Crimes Researcher <http://howwegettonext.com/persisting-in-dark-times-lessons-from-a-war-crimes-researcher-8b3504f4b169>" in *How We Get to Next * _______________________________________________ The Air-L@listserv.aoir.org<mailto:Air-L@listserv.aoir.org> mailing list is provided by the Association of Internet Researchers http://aoir.org Subscribe, change options or unsubscribe at: http://listserv.aoir.org/listinfo.cgi/air-l-aoir.org

Join the Association of Internet Researchers: http://www.aoir.org/

-- Muira McCammon @muira_mccammon<https://twitter.com/muira_mccammon> Recently out: "Persisting in Dark Times: Lessons from a War Crimes Researcher<http://howwegettonext.com/persisting-in-dark-times-lessons-from-a-war-crimes-researcher-8b3504f4b169>" in How We Get to Next