cpollett

Yioop! is a PHP search engine. Yioop! can be configured as either a general purpose search engine for the whole Web or it can be configured to provide search results for a set of URLs or domains. It supports indexing several file formats such as HTML, PDF, DOC, PPT, RTF, RSS, XML, SVG, PNG, JPG ... [More] , BMP, and GIF. The Yioop! crawler can be deployed on one or many machines. On reasonably low-end hardware with cable Internet, four machines can download a million pages every couple of days. Crawling respects robots.txt including Crawl-delay. Yioop! crawls are stored in a Web archive format that is easy to move around. Crawling can be done on one machine and the results deployed elsewhere. Yioop! supports mixing of crawls. [Less]