Download files from a Bash Shell

Last updated: 2020-09-26

Are you trying to download a file from a Bash script but you are not sure where to start? Let me show you how to use wget, curl or download files with a shell script using Bash Redirections.

Generally you will want to use the pre-installed tool on your platform which is generally wget or curl.

Introduction to wget

GNU Wget is a free utility for non-interactive download of files from the Web. It supports HTTP, HTTPS, and FTP protocols, as well as retrieval through HTTP proxies.

One liner example: wget -O newname.txt http://www.het.brown.edu/guide/UNIX-password-security.txt

Introduction to curl

curl is a tool to transfer data from or to a server, using one of the supported protocols (HTTP, HTTPS, FTP, FTPS, TFTP, DICT, TELNET, LDAP or FILE). The command is designed to work without user interaction.

One liner example: curl -o newname.txt http://www.het.brown.edu/guide/UNIX-password-security.txt

Bash Redirections

Lastly, if you don’t have wget, curl or any similar command-line tool, you can use a pure shell script approach by using the /dev/tcp pseudo-device. Since Bash version 2, you can directly open a TCP connection to a given socket.

`/dev/tcp/HOST/PORT'
    If HOST is a valid hostname or Internet address, and PORT is an
     integer port number or service name, Bash attempts to open a TCP
     connection to the corresponding socket.

Below is a sample bash script on how to download a file, from a non-secured HTTP endpoint, without using wget or curl. It defines a bash function named _get and use the pseudo-device to open the TCP connection. You can add this function to your .bashrc for convenience.

_get ()
{
  IFS=/ read proto z host query <<< "$1"
  exec 3< /dev/tcp/$host/80
  {
    echo GET /$query HTTP/1.1
    echo connection: close
    echo host: $host
    echo
  } >&3 
  sed '1,/^$/d' <&3 > $(basename $1)
}

# Example usage
[me@linux ~]$ _get http://www.het.brown.edu/guide/UNIX-password-security.txt
[me@linux ~]$ ls
UNIX-password-security.txt
[me@linux ~]$ type _get
_get is a function
_get () 
{ 
    IFS=/ read proto z host query <<< "$1";
    exec 3< /dev/tcp/$host/80;
    { 
        echo GET /$query HTTP/1.1;
        echo connection: close;
        echo host: $host;
        echo
    } 1>&3;
    sed '1,/^$/d' 0<&3 > $(basename $1)
}