HTTP Status Codes and URL Checker logo

HTTP Status Codes and URL Checker

A HTTP Status Codes Crawler is a tool that scans a website and retrieves HTTP status codes for each page. This helps in diagnosing errors and optimizing technical SEO.

antonio_espresso
$0.001

Apify HTTPS Status Checker 🚀

An Apify Actor that crawls websites and retrieves their HTTP status codes to help monitor site availability, detect broken links, and analyze redirects.


📌 Features

Extracts URLs from sitemaps if available.
Crawls websites when no sitemap is found to collect URLs.
Retrieves HTTP status codes for each discovered URL.
Detects broken links (404 errors) and highlights them.
Provides structured JSON output with status summaries.
Ideal for SEO audits, website monitoring, and performance analysis.


⚙️ Input Parameters

The actor accepts the following input in JSON format:

1{
2  "url": "https://example.com",
3  "max_urls": 10,
4  "follow_links": true,
5  "mode": "auto"
6}
ParameterTypeDescription
urlStringThe target URL or domain (required)
max_urlsIntegerMaximum number of pages to analyze (default: 5)
follow_linksBooleanWhether to follow links in the crawl (default: false)
modeString"sitemap", "crawl", or "auto" (default: "auto")

📤 Output Format

The actor returns structured JSON output with two key sections:

1️⃣ URL Details

Each scanned URL along with its status code.

1{
2  "details": [
3    { "url": "https://example.com", "status": 200 },
4    { "url": "https://example.com/missing-page", "status": 404 }
5  ]
6}

2️⃣ Status Code Summary

Aggregates occurrences of each HTTP status code.

1{
2  "overview": [
3    { "Status Code": 200, "Count": 10 },
4    { "Status Code": 404, "Count": 2 }
5  ]
6}

🚀 How to Run

Option 1: Using Apify Console

  1. Go to the Apify Actor Page.
  2. Click Run and provide the input JSON.
  3. Retrieve results from the Dataset Storage.

Option 2: API Execution (cURL)

Run the actor directly via API:

1cat > input.json <<'EOF'
2{
3  "url": "https://example.com",
4  "max_urls": 10,
5  "follow_links": true,
6  "mode": "auto"
7}
8EOF
9
10curl "https://api.apify.com/v2/acts/YOUR_ACTOR_ID/runs?token=$API_TOKEN" \
11  -X POST \
12  -d @input.json \
13  -H 'Content-Type: application/json'

🔍 Example Use Cases

💡 SEO Audits → Identify broken links and redirects.
💡 Website Monitoring → Track site uptime & status changes.
💡 Data Validation → Ensure external links are working.
💡 Content Management → Find outdated or broken content.


📚 Resources


🛠️ Author & Support

Created by Antonio Blago antonioblago.de🛠️
💬 Need help? Reach out via Apify Community


Frequently Asked Questions

Is it legal to scrape job listings or public data?

Yes, if you're scraping publicly available data for personal or internal use. Always review Websute's Terms of Service before large-scale use or redistribution.

Do I need to code to use this scraper?

No. This is a no-code tool — just enter a job title, location, and run the scraper directly from your dashboard or Apify actor page.

What data does it extract?

It extracts job titles, companies, salaries (if available), descriptions, locations, and post dates. You can export all of it to Excel or JSON.

Can I scrape multiple pages or filter by location?

Yes, you can scrape multiple pages and refine by job title, location, keyword, or more depending on the input settings you use.

How do I get started?

You can use the Try Now button on this page to go to the scraper. You’ll be guided to input a search term and get structured results. No setup needed!