HTTP Headers Check

What is HTTP Headers Check?

HTTP Headers Check fetches the response headers from any URL and displays them in a structured, categorized report. Use this tool to verify your caching configuration, confirm security headers are present, trace redirect chains, and check how your site appears to Googlebot — all without touching browser developer tools.

Why HTTP Headers Matter

Every HTTP response your server sends includes a block of metadata called headers — key-value pairs that arrive before the response body and instruct the client how to handle what follows. They control caching, security, indexing, and performance. If the HTML body is the package being delivered, then the headers are the shipping label and handling instructions.

Headers split into two directions: Request Headers (what the browser sends to the server) and Response Headers (what the server sends back). As a webmaster, you have total control over your response headers. A single missing Cache-Control header can cause your server to crumble under the weight of duplicate requests that should have been cached by a CDN. A misconfigured X-Robots-Tag can make your entire site disappear from Google overnight.

THE BI-DIRECTIONAL HEADER FLOW:
CLIENT: "GET /index.html HTTP/1.1"
  -> Request Headers (User-Agent, Accept, Cookie, Accept-Encoding)
SERVER: "HTTP/1.1 200 OK"
  -> Response Headers (Content-Type, Cache-Control, Server, X-Robots-Tag)

Auditing Headers: CLI vs. Browser Tools

There are two primary schools of header inspection. Browser Developer Tools are excellent for quick, one-off checks while developing — press F12, head to the Network tab, and click any resource to see its Headers panel.

However, browser tools can sometimes hide the truth. They may show the status of a resource after it has been manipulated by local browser settings or cache. For professional auditing, or for checking how crawlers like Googlebot see your site, the command line with cURL is the gold standard. cURL interacts directly with the server and returns raw metadata exactly as it arrives over the wire.

# The professional audit command — shows only headers, no body:
curl -sI https://www.yourdomain.com

# Follow redirect chains and show each hop:
curl -sIL https://yourdomain.com | grep -E "HTTP/|Location"

Use the -L flag to follow redirects. If you see three separate Location headers before your page loads, you have identified a performance bottleneck. Each hop in a redirect chain requires a new DNS lookup and a new TLS handshake, adding hundreds of milliseconds of latency for mobile users.

Content Headers: Type, Encoding, and Transfer

Every response must have a Content-Type header. This is the identity card of the resource. If your server sends a stylesheet but labels it text/plain, the browser will refuse to apply the styles. A misconfigured Content-Type is a very common cause of broken pages during site migrations.

Content-Encoding: br (Brotli) provides significantly better compression than gzip for text-based files like HTML and JavaScript. If your server is sending large uncompressed files, you are wasting bandwidth and increasing your Time to First Byte (TTFB), which is a core Web Vital for SEO rankings.

Caching Strategy: Cache-Control and ETags

The Cache-Control header is the steering wheel for your global infrastructure. It tells browsers and CDNs: "You can keep this file in memory so you do not have to ask me for it again."

The Golden Rule of caching is to separate Static Assets from Dynamic HTML. Your CSS and JavaScript files should usually be content-hashed (e.g., style.a1b2c3.css), allowing you to set a max-age of one year. Your HTML pages should typically use no-cache or a very short max-age, ensuring users see your latest content immediately.

; High-Performance Cache Configuration:
Cache-Control: max-age=31536000, public, immutable   ; For hashed CSS/JS
Cache-Control: no-cache, must-revalidate              ; For HTML pages

ETags handle scenarios where content might have changed. An ETag is a serial number for a file. When a browser has a cached copy, it sends that serial number back to the server. The server checks: "Has my version of the file changed?" If not, it sends back a 304 Not Modified header with no body — saving bandwidth and mobile data usage.

Path Management: Redirects and Location Headers

When you move a page, you use a 301 (Permanent) or 302 (Temporary) redirect. The destination of that move is carried in the Location header. Redirect hygiene is a major part of technical SEO — search engines have a crawl budget and will only follow a limited number of hops before giving up.

1. 301 Moved Permanently: Tells browsers and Google to forget the old URL and update their index. Best for permanent moves.
2. 302 Found: Tells them "I am here temporarily, keep the old URL." Best for maintenance or A/B testing.
3. Absolute URLs: Always use full URLs in your Location headers. Relative paths can cause loops in older browsers.

You should aim for one-hop redirects. Instead of chaining A → B → C, configure A → C and B → C directly. This minimizes latency and ensures link equity passes efficiently to your new pages.

Security Headers: Your Invisible Armor

Many servers (like Nginx, Apache, or PHP) will, by default, send headers like Server: nginx/1.24.0 or X-Powered-By: PHP/8.1. This is gold for an attacker — if they know exactly which version of PHP you are running, they can look up specific unpatched vulnerabilities (CVEs) for that version. Stripping these disclosure headers is a core part of security hardening.

# Check what security headers your site is missing:
curl -sI https://yourdomain.com | grep -iE "strict-transport|x-frame|x-content-type|referrer-policy"

Crawling and Indexing: X-Robots and Canonical Headers

The X-Robots-Tag header is the pro move for indexing control. It allows you to set noindex or nofollow directives on files that are not HTML — such as PDFs or API responses that should not appear in public search results.

A major audit disaster occurs when a developer accidentally leaves X-Robots-Tag: noindex in their production environment after testing. Because it is an HTTP header, it does not show up in View Source. To any normal user, the site looks fine. But to Google, you are waving a massive "go away" sign — this can cause total loss of organic traffic.

; Common Indexing Headers:
X-Robots-Tag: noindex, nofollow     ; Hide resource completely from search
Link: <https://site.com/p>; rel="canonical"  ; Set canonical for PDFs/images
Vary: Accept-Language, User-Agent   ; Help search engines understand variants

CDN and Proxy Headers: Decoding HIT, MISS, and Forwarded IDs

In the modern web, your server almost never talks directly to the user. There is usually a CDN or a load balancer in the middle. These middlemen add their own headers which are vital for debugging performance.

If you see a MISS on a static image every request, it usually means your Cache-Control header is set to no-store or private, which forbids the CDN from holding onto the file. The X-Forwarded-For header is critical for analytics and security logging — without it, your server sees the CDN's IP address rather than the user's real IP.

Systematic Audit Workflow

A professional header audit is a triple-pass process.

1. The Core Scan: Use curl -sI on your main URL and check for mandatory security headers.
2. The Chain Scan: Use curl -sIL to verify that your redirects are direct and efficient.
3. The Compression Scan: Use the Accept-Encoding flag to verify that Brotli is active.
4. The Caching Scan: Check a static .js file to ensure the max-age is sufficiently high.

# Verify Brotli compression is active:
curl -sI -H "Accept-Encoding: br" https://example.com/style.css | grep "content-encoding"
# Expected: content-encoding: br

# Check your redirect chain for extra hops:
curl -sIL https://yourdomain.com | grep -E "HTTP/|Location"
# Each "Location" line is one extra hop to eliminate

We recommend starting at the apex (your homepage) and then checking corner cases: a 404 error page, a deep product page, an image, and a JSON API endpoint. Headers are often set in different blocks of your server configuration, and it is common for the homepage to be perfect but for the API to be missing all security protections.

An HTTP Header audit should be part of your monthly health check. As you add new features, new third-party trackers, or new subdomains, your headers will slowly decline in quality. A regular audit rhythm ensures that your site remains fast, secure, and Google-friendly for the long term.