How to access a website using command-line from the Terminal

Penetration Testing Process often involves dealing with a variety of tools. These tools may be Command-Line based, GUI Based, readily available, and sometimes the pen tester may have to automate a set of commands.

Introduction

GUI is not often possible, and you are not expected to rely on GUI/Gnome-based tools for testing and exploitation practices. Suppose you have gained a shell on some machine and want to download an exploit, then the web browser is not available. In this tutorial, we will take a look at different tools that are helpful while browsing the website using the command line from the terminal.

Netcat

Netcat is a Swiss army knife for hackers, and It gives you a range of options to make your way through the exploitation phase.

Following is how to access a webpage using the GET method with netcat.
$ nc www.google.com 80
GET / HTTP/1.1
Host:www.google.com

To access any content like video, you can enter the following;

$ nc www.example.com 80
GET /VIDEO HTTP/1.1
Host:www.example.com

You can change the ways to POST, OPTIONS, CONNECT as per your requirements. With HTTP/1.1, the connection doesn’t close after one request. To close the connection, enter;

$ Connection:close

Alternatively, you can also use the following while accessing the webpage;

$ nc www.google.com 80
GET / HTTP/1.1
Host:www.google.com
Connection: close

The above commands will close the connection automatically after the page has been fetched from the webserver.

Wget

wget is another commonly used tool to access the webpage. You can use it to download anything placed on a particular web server.

$ wget http://192.168.43.177

Curl

Curl is another powerful tool used to access the webpages in the command line environment. Enter the following command;

$ curl http://wwww.192.168.43.177/path_to_file/file.html

W3M

w3m is a CLI-based web browser. It lets you view the page source and access the webpage as if you were accessing it in any GUI browser.

You can install it by the following command;

$ sudo apt install w3m

To access a webpage, enter;

$ w3m www.google.com

Lynx

Another useful command-line tool is lynx. You can install it by entering;

$ sudo apt install lynx
To access a webpage, enter;

$ lynx www.google.com

Browsh

Another handy text-based browser is browsh. It is still under construction. You can use it by initiating the ssh connection by;

$ ssh brow.sh

Then you can press CTRL+lto focus on the URL bar. Enter your query, and the Browsh will use Google as a search engine to search and output you the results on the command line.

Here’s an output of the weather query using the Browsh.

Custom HTTP Request

You can also craft your custom HTTP request by entering the following command;

printf "GET /\r\nHost: google.com\r\n\r\n" | netcat google.com 80

The HTTPs request will look like the following;
printf "GET /\r\nHost: google.com\r\n\r\n" | socat - OPENSSL:google.com 443

Conclusion

We have various tools available to access the web pages from the terminal. The terminal also gives us the ability to customize the requests, giving us enhanced capabilities. During exploitation, a pen tester must have some of these tools in the pocket.