Grub

New version of Grub Next Generation C# client is available. Changes since last release (0.8.9): - update Spanish translations (author: Víctor Hermida Prada) - fixed bug with count unmodified URL's as a invalid - fixed bugs with crash client on ... [More] invalid URL #439: Crash all the time if i close #C Client in console (with CTRL+C) #455: Crashed without any interaction :-/ #456: Crashed without any interaction :-/ ... sry that i cannot give more informations - fixed bug with crash client on change amount of working crawlers during crawling - fixed bug with not exit from program when any crawler is in waiting queue - fixed URL to create new account - fixed bug with crash program on show informations about selected crawler read more [Less]

Next beta version of Grub Next Generation C# upload server is available. Changes since last release (0.7): - switch from SQLite to MySQL (URL's list and users database) - some code optimization - added to uploader options to run only moving or ... [More] sending to Solr .arc files - added saving Solr response to log file on failed sent URL's to Solr - fixed bug with crash uploader on invalid HTML tag - fixed bug with crash program on convert .arc file to Solr XML You can download source code and zip compressed server from project page. read more [Less]

Third beta version of Grub Next Generation C# upload server is available. Changes since last release (0.6): - added support for Solr (converting .arc files to Solr Update XML Schema, sending URL's to Solr) - added support for URL's database - upload ... [More] URL's to database every hour - fixed bug with not deleting empty directories - fixed bug with crash program on invalid char in .arc file You can download source code and zip compressed server from project page. read more [Less]

New version of Grub Next Generation C# client is available. Changes since last release (0.8.8): - fixed bug with invalid .arc file entry on response HTTP status code 304 - fixed bug with very slow crawling with If-Modified-Since header If you want ... [More] start your adventure with Grub - you can download new version from project page. If you only update from previous version, you can run autoupdater for download new files. read more [Less]

A new version of the GrubNG python client is available. This version supports the new dispatcher. Changelog since version 0.4: Added variable to control the number of uploader threads Changed verbose mode to default. Use -q flag to disable it Added ... [More] debug flag to save workunits and arc files on errors Reupload the arc file in case of a network error (10 times) File close() on everything to prevent 'Too many open files' errors Changed to new grub.org dispatcher address Fixed compressed responce data bug Fixed bug about custom headers Fixed bug about incomplete arc files treated as complete Direct Download Happy crawling ;) read more [Less]

New version of Grub Next Generation C# client is available. Changes since last release (0.8.7): - update Spanish and Galician translations (author: Víctor Hermida Prada) - added French translation (author François Blondel) - fixed option to run/pause ... [More] selected crawler - fixed bug with crash client during start - fixed bug on crash client when sitemap file not exist #401: Grub bug in Windows Xp - fixed bug with not re-send .arc file on upload failure - added support for META ROBOTS tag - small code optimization read more [Less]

Second beta version of Grub Next Generation C# upload server is available. Changes since last release (0.5): - code cleanup and refactoring - move uploading and checking .arc files to separate program - fixed bug with count connected clients - fixed ... [More] server answer on invalid POST request - fixed bug with crash server on copy not existing file - fixed bug with crash server on unfinished files download - fixed bug #377: Connection gets reset instead of closed after disk full error - added simple connections log - added option to move .arc files on disk (not only to Hadoop) read more [Less]

Today we start works on indexing crawled by Grub volunteers data. First effects of our work you can find at: http://soap.grub.org This is very basic web search interface with small index (at this moment around 166 000 of URL's). For indexing we use ... [More] Solr. All other code (current web interface, converter from .arc files to solr xml upload) you can find in our repository in perl directory. [Less]

New version of Grub Next Generation C# client is available. Changes since last release (0.8.6): - update Spanish translation (authors: DiegoJ, Dennis Tobar) - update German translation (authors: HennR, noxin) - update Galician translation (author: ... [More] Victor Hermida Prada) - added option to enable/disable creating sitemaps during crawling - fixed bug with not quit from program when paused - fixed bug with crash client when antivirus move or delete .arc file #376: Grub crashed after Avast moved a Trojan to quarantine folder read more [Less]

Again almost all Grub servers works. At this moment not working only sending sitemaps to Grub. New dispatch server address is: http://soap.grub.org/cgi-bin/dispatch.cgi Additional, new workunits have set new User-Agent HTTP header for clients: User-Agent: GrubNG 0.1 (http://grub.org) read more