Xidel is a command line tool to download web pages and extract data from them.
It can download files over http/s connections, follow redirections, links, or extracted values, and also process local files.
The data can be extracted using XPath 2.0, XQuery 1.0 expressions, JSONiq, CSS 3 selectors, and custom, pattern-matching templates that are like an annotated version of the processed page.
The extracted values can then be exported as plain text/xml/html/json or assigned to variables to be used in other extract expressions or to be exported to the shell.
There is also an online cgi service for testing.
Commercial Use
Modify
Distribute
Place Warranty
Sub-License
Hold Liable
Distribute Original
Disclose Source
Include Copyright
State Changes
Include License
These details are provided for information only. No information here is legal advice and should not be used as such.