Looking at downloading a file from a bash script but not sure where to start? Let me show you how to use wget, curl or download files with a shell script using Bash Redirections.

Generally you will want to use the pre-installed tool on your platform which is generally wget or curl.

WGET

GNU Wget is a free utility for non-interactive download of files from the Web. It supports HTTP, HTTPS, and FTP protocols, as well as retrieval through HTTP proxies.

RTFM: info '(wget) Invoking

wget -O newname.txt http://www.het.brown.edu/guide/UNIX-password-security.txt

CURL

curl is a tool to transfer data from or to a server, using one of the supported protocols (HTTP, HTTPS, FTP, FTPS, TFTP, DICT, TELNET, LDAP or FILE). The command is designed to work without user interaction.

RTFM: man curl

curl -o newname.txt http://www.het.brown.edu/guide/UNIX-password-security.txt

Bash Redirections

Now, how would you do without wget, curl or anyother tool? In pure shell script of course! If you have Bash 2.04 or above with /dev/tcp/ pseudo-device enabled, you can open TCP connection to a given socket.

RTFM: info '(bash) Redirections'

`/dev/tcp/HOST/PORT' If HOST is a valid hostname or Internet address, and PORT is an integer port number or service name, Bash attempts to open a TCP connection to the corresponding socket.

So, here is a sample bash script on how to download a file without wget or curl.

_get function
1
2
3
4
5
6
7
8
9
10
11
12
13
14
_get ()
{
  IFS=/ read proto z host query <<< "$1"
  exec 3< /dev/tcp/$host/80
  {
    echo GET /$query HTTP/1.1
    echo connection: close
    echo host: $host
    echo
  } >&3
  sed '1,/^$/d' <&3 > $(basename $1)
}

_get http://www.het.brown.edu/guide/UNIX-password-security.txt

You can add this function to your .bashrc for convenience.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
[root@host ~]# _get http://www.het.brown.edu/guide/UNIX-password-security.txt
[root@host ~]# ls
UNIX-password-security.txt
[root@host ~]# type _get
_get is a function
_get ()
{
    IFS=/ read proto z host query <<< "$1";
    exec 3< /dev/tcp/$host/80;
    {
        echo GET /$query HTTP/1.1;
        echo connection: close;
        echo host: $host;
        echo
    } 1>&3;
    sed '1,/^$/d' 0<&3 > $(basename $1)
}

Comments