Open Hub
Black Duck Open Hub

Tags : Browse Projects

Select a tag to browse associated projects and drill deeper into the tag cloud.

sweetie-bot

Compare

  Analyzed about 13 hours ago

Automated package repository cannibaliser

440 lines of code

0 current contributors

over 10 years since last commit

1 users on Open Hub

Inactive
5.0
 
0 Reviews
I Use This
Mostly written in shell script
Licenses: AGPL3_or_...
Tags automation console converter crawler gnu_linux gnupony operating_system os package_management packages packaging spike

html-snapshots

Compare

  Analyzed about 17 hours ago

A selector-based html snapshot tool using PhantomJS that sources sitemap.xml, robots.txt, or arbitrary input

6.31K lines of code

1 current contributors

1 day since last commit

1 users on Open Hub

High Activity
0.0
 
0 Reviews
I Use This
Mostly written in JavaScript
Licenses: mit
Tags ajax crawler html javascript robots_txt seo singlepageapplication sitemap sitemapxml snapshots

polipus

Compare

  Analyzed about 3 hours ago

Polipus: distributed and scalable web-crawler framework

2.45K lines of code

0 current contributors

almost 9 years since last commit

1 users on Open Hub

Inactive
0.0
 
0 Reviews
I Use This
Mostly written in Ruby
Licenses: No declared licenses
Tags crawler

Corpusexplorer.SDK.Extern

Compare

  Analyzed about 18 hours ago

Dieses Projekt ist Teil des Software Development Kit - des CorpusExplorers (CorpusExplorer SDK) [Weitere Informationen finden Sie hier]. Das SDK sowie alle Teile können kostenlos für Forschungs- und Bildungsprojekte genutzt werden. Dieser Teil des SDK steht unter der GPL-3.0-Lizenz. Sie können ... [More] dieses Projekt nutzen um: - Den CorpusExplorer zu erweitern. - Ihr eigenes Programm mit dem CorpusExplorer zu verbinden (API-Schnittstelle). - Oder unabhängig vom CorpusExplorer ihr eigenes Programm zu entwickeln/erweitern und so auf bewährte Lösungen zurückzugreifen. [Less]

33.8K lines of code

0 current contributors

about 7 years since last commit

1 users on Open Hub

Inactive
5.0
 
0 Reviews
I Use This
Mostly written in C#
Licenses: GPL2
Tags crawler importer linguistic linguistics nlp scraper webscraper xmlparser

Wikipedia-API

Compare

  Analyzed about 24 hours ago

Python wrapper for Wikipedia

2.14K lines of code

2 current contributors

17 days since last commit

1 users on Open Hub

Moderate Activity
0.0
 
0 Reviews
I Use This
Mostly written in Python
Licenses: mit
Tags api bot crawler mediawiki mediawikiapi python wiki wikipedia wikipedia-mining

Spatie Crawler

Compare

  Analyzed 1 day ago

An easy to use, powerful crawler implemented in PHP. Can execute JavaScript.

2.16K lines of code

0 current contributors

about 1 month since last commit

1 users on Open Hub

Low Activity
0.0
 
0 Reviews
I Use This
Mostly written in PHP
Licenses: No declared licenses
Tags crawler GoogleChrome Guzzle headless javascript php puppeteer spatie

Arachnid Crawler

Compare

  Analyzed about 15 hours ago

Crawl all unique internal links found on a given website, and extract SEO related information - supports javascript based sites

1.41K lines of code

0 current contributors

over 1 year since last commit

0 users on Open Hub

Very Low Activity
0.0
 
0 Reviews
I Use This
Mostly written in PHP
Licenses: No declared licenses
Tags crawler php spider

Zeitcrawler

Compare

  Analyzed about 17 hours ago

A specialized crawler for the German newspaper 'Die Zeit'. Starting from the front page or from a given list of links, the crawler retrieves newspaper articles and gathers new links to explore as it goes, stripping the text of each article out of the HTML formatting and saving it into a raw text ... [More] file. The project includes scripts to convert it into the XML format for further use with natural language processing tools. [Less]

1.64K lines of code

0 current contributors

about 10 years since last commit

0 users on Open Hub

Inactive
0.0
 
0 Reviews
I Use This
Mostly written in Perl
Licenses: gpl3
Tags academic computational_linguistics corpus corpus_linguistics crawler digital_humanities natural_language_processing nlp perl unix webcrawler xml

Équipe Crawler

Compare

  Analyzed about 7 hours ago

A specialized crawler for the French sport newspaper L'Équipe. Starting from the front page or from a given list of links, the crawler retrieves newspaper articles and gathers new links to explore as it goes, stripping the text of each article out of the HTML formatting and saving it into a raw ... [More] text file. The project includes scripts to convert it into the XML format for further use with natural language processing tools. [Less]

401 lines of code

0 current contributors

over 11 years since last commit

0 users on Open Hub

Inactive
0.0
 
0 Reviews
I Use This
Mostly written in Perl
Licenses: gpl3
Tags academic computational_linguistics corpus corpus_linguistics crawler digital_humanities natural_language_processing nlp perl unix webcrawler xml

German Political Speeches Corpus-Builder

Compare

  Analyzed about 7 hours ago

Tools to crawl German official speeches repositories in order to gather a corpus. More information to come. A complete version of the corpus including a visualization tool is available here : http://purl.org/corpus/german-speeches

1.08K lines of code

0 current contributors

over 10 years since last commit

0 users on Open Hub

Inactive
0.0
 
0 Reviews
I Use This
Mostly written in Perl
Licenses: gpl3
Tags academic computational_linguistics corpus corpus_linguistics crawler digital_humanities natural_language_processing nlp perl unix webcrawler xml