Download files from a Bash Shell

  • HOME
  • >
  • BASH
  • >
  • Download files from a Bash Shell
Last Updated: 

Are you trying to download a file from a Bash script but you are not sure where to start? Let me show you how to use wget, curl or download files with a shell script using Bash Redirections.

Generally you will want to use the pre-installed tool on your platform which is generally wget or curl.

Introduction to wget

GNU Wget is a free utility for non-interactive download of files from the Web. It supports HTTP, HTTPS, and FTP protocols, as well as retrieval through HTTP proxies.

One liner example: wget -O newname.txt http://www.het.brown.edu/guide/UNIX-password-security.txt

Introduction to curl

curl is a tool to transfer data from or to a server, using one of the supported protocols (HTTP, HTTPS, FTP, FTPS, TFTP, DICT, TELNET, LDAP or FILE). The command is designed to work without user interaction.

One liner example: curl -o newname.txt http://www.het.brown.edu/guide/UNIX-password-security.txt

Bash Redirections

Lastly, if you don't have wget, curl or any similar command-line tool, you can use a pure shell script approach by using the /dev/tcp pseudo-device. Since Bash version 2, you can directly open a TCP connection to a given socket.

`/dev/tcp/HOST/PORT'
    If HOST is a valid hostname or Internet address, and PORT is an
     integer port number or service name, Bash attempts to open a TCP
     connection to the corresponding socket.

Below is a sample bash script on how to download a file, from a non-secured HTTP endpoint, without using wget or curl. It defines a bash function named _get and use the pseudo-device to open the TCP connection. You can add this function to your .bashrc for convenience.

_get ()
{
  IFS=/ read proto z host query <<< "$1"
  exec 3< /dev/tcp/$host/80
  {
    echo GET /$query HTTP/1.1
    echo connection: close
    echo host: $host
    echo
  } >&3 
  sed '1,/^$/d' <&3 > $(basename $1)
}

# Example usage
[me@linux ~]$ _get http://www.het.brown.edu/guide/UNIX-password-security.txt
[me@linux ~]$ ls
UNIX-password-security.txt
[me@linux ~]$ type _get
_get is a function
_get () 
{ 
    IFS=/ read proto z host query <<< "$1";
    exec 3< /dev/tcp/$host/80;
    { 
        echo GET /$query HTTP/1.1;
        echo connection: close;
        echo host: $host;
        echo
    } 1>&3;
    sed '1,/^$/d' 0<&3 > $(basename $1)
}
Related bash posts that you may like...
The Complete How To Guide of Bash Functions
Learn how to write shell scripts with bash functions. This guide includes examples and best practices on how to define, call, and debug functions in bash.
How To Create Simple Menu with the Shell Select Loop?
The select loop is not a regular shell loop. It can be used in Bash to generate a simple menu from which a user can select numbered options.
What is the Right Way to do Bash Loops?
Looping over a list of numbers or words is a building block in shell scripts. Learn how to write Bash loops, including for loop, while loop, and until loop.
What is the Best Way to Count Files in a Directory?
Learn how to count the number of files in a directory using the Linux command line ls, find, and a native bash shell solution with globs and arrays.