Introduction to Web Proxies

Web proxies sit between a browser or mobile client and the back-end server to capture and inspect HTTP traffic. Unlike full network sniffers that see all local traffic, web proxies focus on web ports such as HTTP/80 and HTTPS/443. This focus makes them ideal for web assessment workflows that require precise request editing and replay.

Web proxies support more than interception, and they become a central tool for application testing. They can map endpoints, review headers, and identify patterns that guide further attacks or validation checks. Common tasks include the following:

  • Vulnerability scanning and validation
  • Web fuzzing and content discovery
  • Crawling and site mapping
  • Request analysis and parameter testing
  • Configuration checks and logic review

Proxy Tools: Burp Suite and ZAP

Burp Suite is the most common web proxy in pentesting due to its focused workflow and deep tooling. It includes Proxy, Repeater, Intruder, and Scanner modules and ships with an integrated Chromium browser. Many advanced features are limited to Burp Pro or Enterprise, but the Community edition is still strong for manual testing and learning.

OWASP Zed Attack Proxy (ZAP) is a free and open source alternative backed by OWASP. It offers similar core proxy features, integrated spidering, passive checks, and an active scanner. In practice, both tools can be used side by side depending on the task, license limits, and personal workflow.

Proxy Configuration

The fastest setup is using a preconfigured browser. In Burp, the Proxy tab lets you open the built-in browser, and all traffic is routed to the proxy automatically. ZAP offers a quick-launch browser icon that uses its proxy settings by default.

For a normal browser like Firefox, you need to set the proxy address and port manually or use an extension. FoxyProxy Standard lets you toggle proxy profiles without editing browser settings every time. A typical profile uses 127.0.0.1 and port 8080, but you can choose any free port your proxy listens on.

HTTPS traffic requires trusting the proxy certificate so the browser does not break TLS inspection. In Burp, you can visit http://burp and download the CA certificate. In ZAP, you can export the server certificate from Tools > Options > Network > Server Certificates. Then import the CA into Firefox under Settings > Privacy & Security > View Certificates > Authorities and trust it for websites and email.

Request Interception and Editing

Interception is enabled from the Proxy tab in Burp, and the Intercept toggle controls whether requests are paused. Once interception is on, load the target in the proxied browser and the request will appear in Burp for inspection. You can forward the request as-is or edit it to test how the server responds to changes.

Manual editing is often faster than tools for small changes, so get comfortable with modifying headers, cookies, and body fields directly. If you need repeated changes, you can send a request to Repeater with CTRL+R and experiment without stopping live traffic. These quick edits often reveal validation gaps or hidden functionality.

Response Interception and Auto-Replace

Sometimes you need to intercept server responses before they reach the browser. Response interception lets you modify HTML, JSON, or JavaScript on the fly to reveal hidden fields or bypass front-end restrictions. In Burp, enable it in Proxy > Proxy settings under response interception rules.

Automation helps when the same change must be applied repeatedly. Burp match-and-replace rules can modify headers or content automatically across all requests or responses. A common example is replacing the User-Agent header with a custom value to bypass naive filters.

User-Agent: HackTheBox Agent 1.0

Repeater and Proxy History

Proxy history is your record of every request that passed through the proxy. In Burp, Proxy > HTTP history lists traffic and lets you filter by type, status code, or tool. It is also a safe place to grab a clean baseline request for testing.

Repeater is designed for controlled, manual iteration. Select a request in history and send it to Repeater with CTRL+R, then edit and resend as many times as needed. It also reduces noise because only the Repeater traffic is sent when you click Send.

Encoding and Decoding Workflow

Encoding and decoding are required when you edit requests manually. URL encoding keeps special characters safe so servers parse requests correctly. Burp makes this easy with Convert Selection > URL > URL-encode key characters or CTRL+U.

The Decoder tab handles other transformations, especially base64 and hex. You can paste data into Decoder, choose a decoding scheme, and instantly view the plaintext. This is useful when APIs pass tokens or data as encoded blobs.

Terminal Proxies

Proxying command-line tools is useful when you want all traffic in your proxy history. ProxyChains routes any CLI tool through your proxy by editing /etc/proxychains.conf and placing a proxy entry at the bottom. Once configured, any command wrapped with ProxyChains will appear in Burp or ZAP.

#socks4         127.0.0.1 9050
http 127.0.0.1 8080

proxychains -q curl http://SERVER_IP:PORT

Metasploit can also use a proxy for its modules. Launch msfconsole, pick a module, and set PROXIES to your local HTTP proxy. Then run the module as normal, and the traffic will show up in your proxy history.

msfconsole

msf6 > use auxiliary/scanner/http/robots_txt
msf6 auxiliary(scanner/http/robots_txt) > set PROXIES HTTP:127.0.0.1:8080
msf6 auxiliary(scanner/http/robots_txt) > set RHOST SERVER_IP
msf6 auxiliary(scanner/http/robots_txt) > set RPORT PORT
msf6 auxiliary(scanner/http/robots_txt) > run

Web Fuzzing with Burp Intruder

Burp Intruder is a built-in fuzzer that can attack endpoints, directories, parameters, and headers. It is powerful for targeted fuzzing, but the Community edition is rate limited to one request per second. That makes CLI tools like ffuf faster for large scans, while Intruder stays useful for focused tests.

The core steps are choosing a request, sending it to Intruder, and setting payload positions. In the Positions tab, select the part of the request to fuzz, such as a directory name or parameter value. Then configure payloads in the Payloads tab with a list, a runtime file, or a character substitution pattern.

Use the Settings tab to tune results and filter by response details. Grep - Match lets you highlight responses that contain a string like 200 OK, making it easier to see valid hits. These adjustments turn Intruder into a precise discovery tool rather than a noisy brute force run.

Burp Scanner Workflow

Burp Scanner is a Pro feature that combines crawling, passive checks, and active testing. It starts by mapping the target, then analyzes responses for likely issues, and finally sends extra requests to validate findings. Scanner findings should guide human validation rather than replace it.

Scope configuration keeps scans focused and safe. In Target > Site map, you can add or remove URLs from scope, which controls what the crawler and scanner touch. From the Dashboard, you can launch a new scan with preset strategies like fast crawling or critical-only audits.

Reports are useful when you need to share findings or archive results. You can export issues for a specific host or an entire scope using the report options in the Site map. Always sanity check the output, especially when the scanner labels a result as low confidence.

ZAP Scanning Alternative

ZAP provides a free scanning workflow that parallels Burp. The Spider builds a site tree by following links and forms, and it can be started from the context menu or the HUD in the preconfigured browser. As the spider runs, ZAP performs passive checks on responses to flag missing headers or DOM-based issues.

Active scanning in ZAP runs after the spider has built a target tree. If you start an active scan without a site tree, ZAP will automatically spider first. The scanner then sends crafted requests to identify vulnerabilities and labels them by severity.

ZAP also supports reporting in multiple formats. You can generate an HTML report from the Report menu, or export to XML and Markdown when needed. Even with a report, always verify the results manually to avoid false positives.

Plugins and Extensions

Extensions expand Burp with scanners, decoders, and quality-of-life tools. The BApp Store is accessible in the Extender tab, and it lists extensions by popularity and category. Review documentation before enabling a new extension in a live test.

Useful examples include:

  • Active Scan++ for extra scanner checks
  • Retire.JS for vulnerable JavaScript libraries
  • Autorize for access control testing
  • Java Deserialization Scanner for common deserialization issues
  • Software Version Reporter for fingerprinting hints
  • CSP Auditor for content security policy analysis

Pick extensions that match your goals and avoid installing everything at once. Each extension adds complexity and may slow down your workflow. Start with a small, trusted set and expand when you need a specific capability.

Reference

This article is based on my personal study notes from the Information Security Foundations track.

Full repository: https://github.com/lameiro0x/pentesting-path-htb