{"id":198,"date":"2016-01-12T15:34:15","date_gmt":"2016-01-12T15:34:15","guid":{"rendered":"http:\/\/wp-p3d2tsa2xh.pairsite.com\/?p=198"},"modified":"2025-05-21T14:42:18","modified_gmt":"2025-05-21T18:42:18","slug":"paircloud-downloading-files-with-wget","status":"publish","type":"ht_kb","link":"https:\/\/www.pair.com\/support\/kb\/paircloud-downloading-files-with-wget\/","title":{"rendered":"Downloading files with wget"},"content":{"rendered":"

Shared<\/p><\/a>

VPS<\/p><\/a>

Dedicated<\/p><\/a> 

WP Professional<\/p><\/a>

WP Professional Plus<\/p><\/a><\/span><\/p>\n

The wget command is an internet file downloader that can download anything from files and web pages all the way through to entire websites.<\/p>\n

Basic Usage<\/h3>\n

The wget command is in the format of:<\/p>\n

wget [options] url<\/pre>\n
For example, in its most basic form, you would write a command something like this:<\/p>\n
wget http:\/\/www.domain.com\/filename.zip<\/pre>\n
This will download the filename.zip<\/em> file from www.domain.com<\/em> and place it in your current directory.<\/p>\n
Redirecting Output<\/h4>\n
The -O<\/code> option sets the output file name. If the file was called filename-4.0.1.zip<\/em> and you wanted to save it directly to filename.zip<\/em> you would use a command like this:<\/p>\n
 wget -O filename.zip http:\/\/www.domain.com\/filename-4.0.1.zip<\/pre>\n
The wget program can operate on many different protocols with the most common being ftp:\/\/ and http:\/\/.<\/p>\n
Downloading in the background.<\/h3>\n
If you want to download a large file and close your connection to the server you can use the command:<\/p>\n
wget -b url<\/pre>\n
Downloading Multiple Files<\/h3>\n
If you want to download multiple files you can create a text file with the list of target files. Each filename should be on its own line. You would then run the command:<\/p>\n
wget -i filename.txt<\/pre>\n
You can also do this with an HTML file. If you have an HTML file on your server and you want to download all the links within that page you need add --force-html<\/code> to your command.<\/p>\n
To use this, all the links in the file must be full links, if they are relative links you will need to add <base href=\"\/support\/knowledge_base\/\"><\/code> following to the HTML file before running the command:<\/p>\n
wget --force-html -i filename.html<\/pre>\n
Limiting the download speed<\/h3>\n
Usually, you want your downloads to be as fast as possible. However, if you want to continue working while downloading, you want the speed to be throttled.<\/p>\n
To do this use the --limit-rate<\/code> option. You would use it like this:<\/p>\n
wget --limit-rate=200k http:\/\/www.domain.com\/filename.tar.gz<\/code><\/pre>\n
Continuing a failed download<\/h3>\n
If you are downloading a large file and it fails part way through, you can continue the download in most cases by using the -c<\/code> option.<\/p>\n
For example:<\/p>\n
 wget -c http:\/\/www.domain.com\/filename.tar.gz<\/pre>\n
Normally when you restart a download of the same filename, it will append a number starting with .1  to the downloaded file and start from the beginning again.<\/p>\n
Downloading in the background<\/h3>\n
If you want to download in the background use the -b<\/code> option. An example of this is:<\/p>\n
wget -b http:\/\/domain.com\/filename.tar.gz<\/pre>\n
Checking if remote files exist before a scheduled download<\/h3>\n
If you want to schedule a large download ahead of time, it is worth checking that the remote files exist. The option to run a check on files is --spider<\/code>.<\/p>\n
In circumstances such as this, you will usually have a file with the list of files to download inside. An example of how this command will look when checking for a list of files is:<\/p>\n
 wget --spider -i filename.txt<\/pre>\n
However, if it is just a single file you want to check, then you can use this formula:<\/p>\n
wget --spider http:\/\/www.domain.com\/filename.tar.gz<\/pre>\n
Copy an entire website<\/h3>\n
If you want to copy an entire website you will need to use the --mirror<\/code> option. As this can be a complicated task there are other options you may need to use such as -p<\/code>, -P<\/code>, --convert-links<\/code>, --reject<\/code> and --user-agent<\/code>.<\/p>\n
 <\/p>\n\n\n\n\n\n\n\n
-p<\/strong><\/td>\nThis option is necessary if you want all additional files necessary to view the page such as CSS files and images<\/span><\/td>\n<\/tr>\n
-P<\/strong><\/td>\nThis option sets the download directory. Example: <\/span>-P downloaded<\/code><\/span><\/td>\n<\/tr>\n
--convert-links<\/strong><\/td>\nThis option will fix any links in the downloaded files. For example, it will change any links that refer to other files that were downloaded to local ones.<\/span><\/td>\n<\/tr>\n
--reject<\/strong><\/span><\/td>\nThis option prevents certain file types from downloading. If for instance, you wanted all files except flash video files (flv) you would use<\/span> --reject=flv<\/code><\/span><\/td>\n<\/tr>\n
--user-agent<\/strong><\/td>\nThis option is for when a site has protection in place to prevent scraping. You would use this to set your user agent to make it look like you were a normal web browser and not wget.<\/span><\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n
Using all these options to download a website would look like this:<\/p>\n
wget --mirror -p --convert-links -P .\/local-dir --user-agent=\"Mozilla\/5.0 (Windows NT 6.3; WOW64; rv:40.0\" http:\/\/www.domain.com\/<\/pre>\n
\n
TIP: Being Nice<\/h4>\n
It is always best to ask permission before downloading a site belonging to someone else and even if you have permission it is always good to play nice with their server. These two additional options will ensure you don\u2019t harm their server while downloading.<\/p>\n
--wait=15 --limit-rate=50K<\/pre>\n
This will wait 15 seconds between each page and limit the download speed to 50K\/sec.<\/p>\n<\/div>\n
Downloading using FTP<\/h3>\n
If you want to download a file via FTP and a username and password is required, then you will need to use the --ftp-user<\/code> and --ftp-password<\/code> options.<\/p>\n
An example of this might look like:<\/p>\n
wget --ftp-user=USERNAME --ftp-password=PASSWORD ftp:\/\/ftp.domain.com\/filename.tar.gz<\/pre>\n
Retry<\/h3>\n
If you are getting failures during a download, you can use the -t<\/code> option to set the number of retries. Such a command may look like this:<\/p>\n
wget -t 50 http:\/\/www.domain.com\/filename.tar.gz<\/pre>\n
You could also set it to infinite retries using -t inf<\/code>.<\/p>\n
Recursive down to level X<\/h3>\n
If you want to get only the first level of a website, then you would use the -r<\/code> option combined with the -l<\/code> option.<\/p>\n
For example, if you wanted only the first level of website you would use:<\/p>\n
wget -r -l1 http:\/\/www.example.com\/<\/pre>\n
Setting the username and password for authentication<\/h3>\n
If you need to authenticate an HTTP request you use the command:<\/p>\n
wget --http-user=USERNAME --http-password=PASSWORD http:\/\/domain.com\/filename.html<\/pre>\n
wget is a very complicated and complete downloading utility. It has many more options and multiple combinations to achieve a specific task. For more details, you can use the man wget<\/code> command in your terminal\/command prompt to bring up the wget manual. You can also find the wget manual here in webpage format.<\/a><\/p>\n","protected":false},"excerpt":{"rendered":"
  The wget command is an internet file downloader that can download anything from files and web pages all the way through to entire websites. Basic Usage The wget command is in the format of: wget [options] url For example, in its most basic form, you would write a command…<\/p>\n","protected":false},"author":12,"comment_status":"closed","ping_status":"open","template":"","format":"standard","meta":[],"ht-kb-category":[13],"ht-kb-tag":[],"yoast_head":"\nDownload File with wget | wget Download File | wget a File<\/title>\n<meta name=\"description\" content=\"The wget command is an internet file downloader that can download anything from files to entire websites. Learn how to download a file with wget.\" \/>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/www.pair.com\/support\/kb\/paircloud-downloading-files-with-wget\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Download File with wget | wget Download File | wget a File\" \/>\n<meta property=\"og:description\" content=\"The wget command is an internet file downloader that can download anything from files to entire websites. Learn how to download a file with wget.\" \/>\n<meta property=\"og:url\" content=\"https:\/\/www.pair.com\/support\/kb\/paircloud-downloading-files-with-wget\/\" \/>\n<meta property=\"og:site_name\" content=\"Knowledge Base - Pair Networks\" \/>\n<meta property=\"article:publisher\" content=\"https:\/\/www.facebook.com\/pairnetworks\/\" \/>\n<meta property=\"article:modified_time\" content=\"2025-05-21T18:42:18+00:00\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:site\" content=\"@pairNetworks\" \/>\n<meta name=\"twitter:label1\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data1\" content=\"4 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"WebPage\",\"@id\":\"https:\/\/www.pair.com\/support\/kb\/paircloud-downloading-files-with-wget\/\",\"url\":\"https:\/\/www.pair.com\/support\/kb\/paircloud-downloading-files-with-wget\/\",\"name\":\"Download File with wget | wget Download File | wget a File\",\"isPartOf\":{\"@id\":\"https:\/\/www.pair.com\/support\/kb\/#website\"},\"datePublished\":\"2016-01-12T15:34:15+00:00\",\"dateModified\":\"2025-05-21T18:42:18+00:00\",\"description\":\"The wget command is an internet file downloader that can download anything from files to entire websites. Learn how to download a file with wget.\",\"breadcrumb\":{\"@id\":\"https:\/\/www.pair.com\/support\/kb\/paircloud-downloading-files-with-wget\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/www.pair.com\/support\/kb\/paircloud-downloading-files-with-wget\/\"]}]},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/www.pair.com\/support\/kb\/paircloud-downloading-files-with-wget\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\/\/www.pair.com\/support\/kb\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Downloading files with wget\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/www.pair.com\/support\/kb\/#website\",\"url\":\"https:\/\/www.pair.com\/support\/kb\/\",\"name\":\"Knowledge Base - Pair Networks\",\"description\":\"\",\"publisher\":{\"@id\":\"https:\/\/www.pair.com\/support\/kb\/#organization\"},\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/www.pair.com\/support\/kb\/?s={search_term_string}\"},\"query-input\":\"required name=search_term_string\"}],\"inLanguage\":\"en-US\"},{\"@type\":\"Organization\",\"@id\":\"https:\/\/www.pair.com\/support\/kb\/#organization\",\"name\":\"Pair Networks\",\"url\":\"https:\/\/www.pair.com\/support\/kb\/\",\"logo\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/www.pair.com\/support\/kb\/#\/schema\/logo\/image\/\",\"url\":\"https:\/\/www.pair.com\/support\/kb\/wp-content\/uploads\/2022\/02\/Gb_TUkUE_400x400.jpeg\",\"contentUrl\":\"https:\/\/www.pair.com\/support\/kb\/wp-content\/uploads\/2022\/02\/Gb_TUkUE_400x400.jpeg\",\"width\":360,\"height\":360,\"caption\":\"Pair Networks\"},\"image\":{\"@id\":\"https:\/\/www.pair.com\/support\/kb\/#\/schema\/logo\/image\/\"},\"sameAs\":[\"https:\/\/www.facebook.com\/pairnetworks\/\",\"https:\/\/twitter.com\/pairNetworks\",\"https:\/\/www.instagram.com\/pairnetworks\/\",\"https:\/\/www.linkedin.com\/company\/2269676\/\",\"https:\/\/www.pinterest.com\/pairnetworksinc\/pins\/\",\"https:\/\/www.youtube.com\/user\/pairnetworks\"]}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Download File with wget | wget Download File | wget a File","description":"The wget command is an internet file downloader that can download anything from files to entire websites. Learn how to download a file with wget.","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/www.pair.com\/support\/kb\/paircloud-downloading-files-with-wget\/","og_locale":"en_US","og_type":"article","og_title":"Download File with wget | wget Download File | wget a File","og_description":"The wget command is an internet file downloader that can download anything from files to entire websites. Learn how to download a file with wget.","og_url":"https:\/\/www.pair.com\/support\/kb\/paircloud-downloading-files-with-wget\/","og_site_name":"Knowledge Base - Pair Networks","article_publisher":"https:\/\/www.facebook.com\/pairnetworks\/","article_modified_time":"2025-05-21T18:42:18+00:00","twitter_card":"summary_large_image","twitter_site":"@pairNetworks","twitter_misc":{"Est. reading time":"4 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"WebPage","@id":"https:\/\/www.pair.com\/support\/kb\/paircloud-downloading-files-with-wget\/","url":"https:\/\/www.pair.com\/support\/kb\/paircloud-downloading-files-with-wget\/","name":"Download File with wget | wget Download File | wget a File","isPartOf":{"@id":"https:\/\/www.pair.com\/support\/kb\/#website"},"datePublished":"2016-01-12T15:34:15+00:00","dateModified":"2025-05-21T18:42:18+00:00","description":"The wget command is an internet file downloader that can download anything from files to entire websites. Learn how to download a file with wget.","breadcrumb":{"@id":"https:\/\/www.pair.com\/support\/kb\/paircloud-downloading-files-with-wget\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/www.pair.com\/support\/kb\/paircloud-downloading-files-with-wget\/"]}]},{"@type":"BreadcrumbList","@id":"https:\/\/www.pair.com\/support\/kb\/paircloud-downloading-files-with-wget\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/www.pair.com\/support\/kb\/"},{"@type":"ListItem","position":2,"name":"Downloading files with wget"}]},{"@type":"WebSite","@id":"https:\/\/www.pair.com\/support\/kb\/#website","url":"https:\/\/www.pair.com\/support\/kb\/","name":"Knowledge Base - Pair Networks","description":"","publisher":{"@id":"https:\/\/www.pair.com\/support\/kb\/#organization"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/www.pair.com\/support\/kb\/?s={search_term_string}"},"query-input":"required name=search_term_string"}],"inLanguage":"en-US"},{"@type":"Organization","@id":"https:\/\/www.pair.com\/support\/kb\/#organization","name":"Pair Networks","url":"https:\/\/www.pair.com\/support\/kb\/","logo":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.pair.com\/support\/kb\/#\/schema\/logo\/image\/","url":"https:\/\/www.pair.com\/support\/kb\/wp-content\/uploads\/2022\/02\/Gb_TUkUE_400x400.jpeg","contentUrl":"https:\/\/www.pair.com\/support\/kb\/wp-content\/uploads\/2022\/02\/Gb_TUkUE_400x400.jpeg","width":360,"height":360,"caption":"Pair Networks"},"image":{"@id":"https:\/\/www.pair.com\/support\/kb\/#\/schema\/logo\/image\/"},"sameAs":["https:\/\/www.facebook.com\/pairnetworks\/","https:\/\/twitter.com\/pairNetworks","https:\/\/www.instagram.com\/pairnetworks\/","https:\/\/www.linkedin.com\/company\/2269676\/","https:\/\/www.pinterest.com\/pairnetworksinc\/pins\/","https:\/\/www.youtube.com\/user\/pairnetworks"]}]}},"_links":{"self":[{"href":"https:\/\/www.pair.com\/support\/kb\/wp-json\/wp\/v2\/ht-kb\/198"}],"collection":[{"href":"https:\/\/www.pair.com\/support\/kb\/wp-json\/wp\/v2\/ht-kb"}],"about":[{"href":"https:\/\/www.pair.com\/support\/kb\/wp-json\/wp\/v2\/types\/ht_kb"}],"author":[{"embeddable":true,"href":"https:\/\/www.pair.com\/support\/kb\/wp-json\/wp\/v2\/users\/12"}],"replies":[{"embeddable":true,"href":"https:\/\/www.pair.com\/support\/kb\/wp-json\/wp\/v2\/comments?post=198"}],"version-history":[{"count":9,"href":"https:\/\/www.pair.com\/support\/kb\/wp-json\/wp\/v2\/ht-kb\/198\/revisions"}],"predecessor-version":[{"id":9798,"href":"https:\/\/www.pair.com\/support\/kb\/wp-json\/wp\/v2\/ht-kb\/198\/revisions\/9798"}],"wp:attachment":[{"href":"https:\/\/www.pair.com\/support\/kb\/wp-json\/wp\/v2\/media?parent=198"}],"wp:term":[{"taxonomy":"ht_kb_category","embeddable":true,"href":"https:\/\/www.pair.com\/support\/kb\/wp-json\/wp\/v2\/ht-kb-category?post=198"},{"taxonomy":"ht_kb_tag","embeddable":true,"href":"https:\/\/www.pair.com\/support\/kb\/wp-json\/wp\/v2\/ht-kb-tag?post=198"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}