Managed Projects

WikiHadoop

  Analyzed about 2 hours ago

WikiHadoop is a set of Hadoop modules focusing onto processing Wikipedia's TB-scale XML dump files. Wikipedia XML dumps with complete edit histories have been difficult to process because of its exceptional size and structure. While a "page" is a common processing unit, one Wikipedia page may ... [More] contain more than gigabytes of text when the edit history is very long. WikiHadoop provides an InputFormat for Hadoop Streaming Interface that processes Wikipedia bzip2 XML dumps in a streaming manner. Using this InputFormat, the content of every page is fed to a mapper via standard input and output without using too much memory. Thanks to Hadoop Streaming, mappers can be implemented in any language. [Less]

4.92K lines of code

0 current contributors

about 13 years since last commit

1 users on Open Hub

Inactive
0.0
 
I Use This

RevDiffSearch

  Analyzed about 19 hours ago

Full text search system designed for Wikipedia revision diffs. Implemented as Lucene indexer & searcher for retrieving revision diffs of Wikipedia and others.

3.1K lines of code

0 current contributors

over 12 years since last commit

1 users on Open Hub

Inactive
0.0
 
I Use This
Licenses: No declared licenses

TumblrTagCommand

  Analyzed 1 day ago

TumblrTagCommand supports fast tagging of many Tumblr posts at once, in CUI style, without page transitions. You can just type 'p' to mark/unmark a post, and type 'g' to tag. Minibuffer & LDRize are required.

281 lines of code

0 current contributors

over 16 years since last commit

0 users on Open Hub

Inactive
0.0
 
I Use This

TinyClassifier

  Analyzed about 13 hours ago

a tiny machine learning library for scripting

1.65K lines of code

0 current contributors

almost 15 years since last commit

0 users on Open Hub

Inactive
0.0
 
I Use This
Licenses: No declared licenses

filteroid

  Analyzed about 17 hours ago

129 lines of code

0 current contributors

over 16 years since last commit

0 users on Open Hub

Inactive
0.0
 
I Use This
Licenses: No declared licenses

ligtran

  Analyzed about 13 hours ago

text conversion by composing & decomposing ligatures

2.47K lines of code

0 current contributors

about 12 years since last commit

0 users on Open Hub

Inactive
0.0
 
I Use This
Licenses: No declared licenses