Intro
Most communication on the Internet is performed through web requests using the Hypertext Transfer Protocol (HTTP). HTTP is an application-layer protocol designed to enable the retrieval and exchange of resources across the World Wide Web. The term hypertext refers to text that contains references (links) to other resources, enabling non-linear navigation between documents.
HTTP follows a client-server communication model. The client, typically a web browser or command-line tool, initiates a request for a resource. The server processes this request and returns a response containing the requested resource or an error message. By default, HTTP operates over TCP port 80, although servers may be configured to listen on alternative ports.
From a security perspective, HTTP is fundamental. Understanding how requests are constructed, transmitted, and processed is essential for identifying attack surfaces such as parameter manipulation, header abuse, and protocol misuse.
URL
Resources accessed via HTTP are identified using a Uniform Resource Locator (URL). A URL provides a structured way to describe the exact location and access method for a resource.
A URL is composed of multiple components:
| Component | Example | Description |
|---|---|---|
Scheme | http:// https:// | Identifies the protocol to be used. |
User Info | admin:password@ | Optional credentials used for authentication. |
Host | example.com | The hostname or IP address of the server. |
Port | :80 | Optional port number. Defaults to 80 for HTTP and 443 for HTTPS. |
Path | /dashboard.php | The specific resource being requested. |
Query String | ?login=true | Parameters passed to the server. |
Fragment | #status | Client-side reference to a section of the resource. |
Only the scheme and host are strictly required, but additional components provide fine-grained control over how the request is handled.
HTTP Flow
When a user enters a URL into a browser, several steps occur before the page is displayed. First, the browser resolves the domain name via DNS to obtain an IP address. Once resolved, the browser initiates a TCP connection to the server and sends an HTTP request, typically a GET request for the root path /.
The server processes the request and responds with an HTTP response containing a status code and, if successful, the requested content. The browser then parses and renders the content, initiating additional requests as needed for assets such as stylesheets, scripts, and images.
Understanding this flow is critical for penetration testing, as attackers often manipulate intermediate steps to bypass controls or inject malicious input.
cURL
cURL is a command-line tool and library that supports HTTP and many other protocols. It is widely used for automation, scripting, and manual testing of web services.
A simple HTTP request using cURL looks like this:
curl http://example.com
cURL allows full control over methods, headers, cookies, and payloads, making it indispensable for web security testing.
Hypertext Transfer Protocol Secure (HTTPS)
A major limitation of HTTP is that it transmits data in plaintext. This allows attackers to intercept traffic using Man-in-the-Middle (MitM) attacks.
HTTPS addresses this issue by encrypting communication using TLS (Transport Layer Security). All data exchanged between client and server is encrypted, preventing eavesdropping and tampering.
HTTPS Flow
When accessing a site over HTTPS, the client and server perform a TLS handshake. This process includes exchanging cryptographic parameters, validating certificates, and establishing a secure session key. Once the handshake is complete, standard HTTP communication continues over an encrypted channel.
cURL for HTTPS
cURL automatically handles HTTPS connections. If a certificate is invalid or expired, cURL will refuse the connection unless explicitly instructed otherwise.
To ignore certificate validation (for testing purposes only):
curl -k https://example.com
HTTP Requests and Responses
HTTP Request
An HTTP request begins with a request line containing the method, path, and protocol version:
GET /index.html HTTP/1.1
This is followed by headers that provide metadata about the request, such as host, user agent, and cookies.
HTTP Response
The server replies with an HTTP response containing a status line, headers, and an optional body:
HTTP/1.1 200 OK
The body may contain HTML, JSON, binary files, or other content types.
cURL
Verbose output in cURL allows inspection of full requests and responses:
curl -v http://example.com
Browser Developer Tools
Modern browsers include developer tools that allow inspection of network requests. The Network tab is particularly valuable for observing headers, parameters, and responses during application interaction.
HTTP Headers
HTTP headers convey metadata about requests and responses. They are grouped into categories based on purpose.
General Headers
General headers apply to both requests and responses.
Entity Headers
Entity headers describe the content being transmitted, such as content type and length.
Request Headers
Request headers provide context about the client and request.
Response Headers
Response headers describe server behavior and response handling.
Security Headers
Security headers enforce browser-side security policies, reducing exposure to common attacks such as XSS and protocol downgrade attacks.
cURL
To retrieve only response headers:
curl -I https://example.com
HTTP Methods
HTTP Codes and Methods
HTTP defines multiple request methods and status codes to describe actions and outcomes.
Request Methods
| Method | Description |
|---|---|
| GET | Retrieve a resource |
| POST | Submit data |
| PUT | Update a resource |
| DELETE | Remove a resource |
Status Codes
Status codes indicate the result of a request, ranging from informational (1xx) to server errors (5xx).
GET
GET requests retrieve resources and often include parameters in the URL.
HTTP Basic Authentication
Basic authentication transmits credentials encoded in Base64.
curl -u admin:admin http://server/
POST
POST requests transmit data in the request body, making them suitable for form submissions and file uploads.
curl -X POST -d 'username=admin&password=admin' http://server/
CRUD API
APIs commonly implement CRUD operations using HTTP methods.
APIs
APIs expose application functionality programmatically.
CRUD
| Operation | Method |
|---|---|
| Create | POST |
| Read | GET |
| Update | PUT |
| Delete | DELETE |
Read
curl http://server/api.php/city/london
Create
curl -X POST -H 'Content-Type: application/json' -d '{"city":"HTB"}' http://server/api.php/city
Update
curl -X PUT -H 'Content-Type: application/json' -d '{"city":"NewCity"}' http://server/api.php/city/london
Delete
curl -X DELETE http://server/api.php/city/NewCity
Reference
This article is based on my personal study notes from the Information Security Foundations track.
Full repository: https://github.com/lameiro0x/security-foundations-htb-notes