DarkMatter in Cyberspace
  • Home
  • Categories
  • Tags
  • Archives

Download Website with wget


I want to download the Chinese version of "Learn you a Haskell for Great Good" at http://learnyouahaskell-zh-tw.csie.org/, use this command:

wget -U mozilla --limit-rate=200k --no-clobber --convert-links --random-wait -r -p -E -e robots=off http://learnyouahaskell-zh-tw.csie.org/

where:

-U mozilla: pretends to be just like a browser Mozilla is looking at a page;

-r: recursive download all website, -p: download everything even pictures;

-E: gets the right extension of the file, without most html and other files have no extension;

-e robots=off: act like we are not a robot;

You can use "-l" option to specify the recursive depth. The default maximum depth is five layers.

Ref:

LINUX - wget full website - full site download

https://www.gnu.org/software/wget/manual/html_node/Recursive-Download.html



Published

Dec 13, 2013

Last Updated

Dec 13, 2013

Category

Tech

Tags

  • download 3
  • firefox 3
  • wget 1

Contact

  • Powered by Pelican. Theme: Elegant by Talha Mansoor