Dear Air-L colleagues,
My name is Todd Ensz. I run OwnedByDanes LLC, a small data company, and I want to let people here know about a resource I've built. In the interest of transparency: I license it commercially (mainly to AI developers), and I also offer academic licenses and free samples.
It is a cleaned, documented Usenet archive spanning 1980-2013: about 405 million posts across 18,332 newsgroups in nine hierarchies. A threaded edition reconstructs 190.8 million conversations from the original Message-ID/References headers, so each post carries a thread_id, a position and a depth.
One example of what that makes possible. The comp.os.minix thread "What would you like to see most in minix?" opens in August 1991 with the post in which Linus Torvalds first mentions his hobby operating system, followed by seven replies over three days. Then nothing for twenty years. In August 2011 replies started arriving again: 78 posts through May 2013, mostly short thank-you notes, many in languages other than English, from people who had found the thread. The same thread holds a technical conversation and its later life as a kind of pilgrimage site.
Limits you should know about: Usenet is unmoderated and contains sensitive material. Display names are kept as posted; email addresses are removed, so about 8% of posts have an author field of "[redacted]" (the header held only an address). Threads are rebuilt from headers, so nesting is imperfect and some parent posts are missing. Cross-posted copies are retained (about 15.8% of posts).
I'm aware that the commercialization of Usenet archives is itself a contested question, and I'd welcome critique, particularly of the documentation and the ethics. [If you wrote posts in the archive and would like them removed from the hosted dataset, write to me and I will take care of it.]
Todd Ensz
OwnedByDanes LLC