Download files from a Bash Shell

  • HOME
  • >
  • BASH
  • >
  • Download files from a Bash Shell
Last Updated: 
Tags:  bash curl wget

Are you trying to download a file from a Bash script but you are not sure where to start? Let me show you how to use wget, curl or download files with a shell script using Bash Redirections.

Generally you will want to use the pre-installed tool on your platform which is generally wget or curl.

Introduction to wget

GNU Wget is a free utility for non-interactive download of files from the Web. It supports HTTP, HTTPS, and FTP protocols, as well as retrieval through HTTP proxies.

One liner example: wget -O newname.txt http://www.het.brown.edu/guide/UNIX-password-security.txt

Introduction to curl

curl is a tool to transfer data from or to a server, using one of the supported protocols (HTTP, HTTPS, FTP, FTPS, TFTP, DICT, TELNET, LDAP or FILE). The command is designed to work without user interaction.

One liner example: curl -o newname.txt http://www.het.brown.edu/guide/UNIX-password-security.txt

Bash Redirections

Now, how would you do without wget, curl or any other tool? In pure shell script of course! If you have Bash 2.04 or above and /dev/tcp/ pseudo-device enabled, you can open TCP connection to a given socket.

`/dev/tcp/HOST/PORT'
    If HOST is a valid hostname or Internet address, and PORT is an
     integer port number or service name, Bash attempts to open a TCP
     connection to the corresponding socket.

So, here is a sample bash script on how to download a file without wget or curl.

_get ()
{
  IFS=/ read proto z host query <<< "$1"
  exec 3< /dev/tcp/$host/80
  {
    echo GET /$query HTTP/1.1
    echo connection: close
    echo host: $host
    echo
  } >&3 
  sed '1,/^$/d' <&3 > $(basename $1)
}

_get http://www.het.brown.edu/guide/UNIX-password-security.txt

You can add this function to your .bashrc for convenience.

[root@host ~]$ _get http://www.het.brown.edu/guide/UNIX-password-security.txt
[root@host ~]$ ls
UNIX-password-security.txt
[root@host ~]$ type _get
_get is a function
_get () 
{ 
    IFS=/ read proto z host query <<< "$1";
    exec 3< /dev/tcp/$host/80;
    { 
        echo GET /$query HTTP/1.1;
        echo connection: close;
        echo host: $host;
        echo
    } 1>&3;
    sed '1,/^$/d' 0<&3 > $(basename $1)
}
Related bash posts that you may like
What is the Bash Null Command?
Learn about the Bash null command, also known as the POSIX shell colon command. This post cover concrete use cases and pitfalls to avoid.
How To Format Date and Time in Linux, macOS, and Bash?
Find out how to manipulate date and time on linux and macOS systems as well as natively in the Bash shell. This post covers all you need to know to format a date from your shell.
How To Use Option as Meta Key in macOS Terminal?
The Meta Key is a modifier key that can be quite helpful to improve your productivity while working in a terminal and bash. This post cover how to enable from the command line the Meta Key in macOS Terminal.