Article Scraper & News Content Extractor 📰🚀

Extract clean, structured data from news articles and blog posts with this powerful Apify Actor. Get article text, metadata, keywords, summaries, and more – perfect for content analysis, market research, news aggregation, and SEO monitoring. No coding required!

Features ✨

• Comprehensive Article Extraction 📰 Get the full article text, cleanly extracted from the webpage
• Key Metadata 📅 Retrieve publication date, author(s), and source URL
• SEO & Content Analysis 🔍 Extract keywords, meta descriptions, and automatically generated summaries
• Multimedia Extraction 🖼️ Get links to the main image, all images, and embedded videos
• Language Detection 🌐 Automatically identifies the language of the article
• Flexible Input 🔗 Use a list of URLs to scrape multiple articles
• Proxy Support ⚙️ Use Apify Proxy or custom proxy URLs for reliable scraping
• Customizable ⚙️ Set request timeout and user agent
• Analysis-Ready Data (JSON) 💾 Structured data output, perfect for analysis and integration
• Error Handling ✅ Robust error handling with informative messages

Why Use This Article Scraper? 🤔

This Actor is your one-stop solution for extracting valuable data from online articles. Whether you're a marketer tracking brand mentions, a researcher collecting data for analysis, or a developer building a news aggregation app, this tool saves you time and effort.

Designed for:

• Speed: Get data quickly and efficiently
• Accuracy: Reliable data extraction, even from complex websites
• Ease of Use: No coding required – just provide the URLs
• Scalability: Handles both small and large scraping tasks

Data Output 📦

The Actor returns a JSON dataset with the following fields for each article:

Field	Description
articleURL	The URL of the scraped article
sourceURL	The base URL of the website
articleLanguage	The language of the article (e.g., "en", "es")
articleTitle	The title of the article
articleAuthors	A comma-separated list of the article's authors
articlePublishDate	The publication date of the article (ISO 8601 format)
articleText	The full text content of the article
articleTopImage	The URL of the main image of the article
articleAllImages	A comma-separated list of URLs for all images found
articleVideos	A comma-separated list of URLs for embedded videos
articleKeywords	A comma-separated list of keywords extracted
articleSummary	A concise summary of the article
scrapedAt	The timestamp of when the article was scraped
scrapeSuccess	Boolean indicating scraping success
articleMetaDescription	The meta description of the article
articleMetaKeywords	A comma-separated list of the meta keywords
scrapeErrorMessage	An error message if scrapeSuccess is false

Example Output

1[
2  {
3    "articleURL": "https://www.example.com/news/article1",
4    "sourceURL": "https://www.example.com",
5    "articleLanguage": "en",
6    "articleTitle": "Example News Article",
7    "articleAuthors": "John Doe, Jane Smith",
8    "articlePublishDate": "2024-07-27T10:00:00Z",
9    "articleText": "This is the full text of the example news article...",
10    "articleTopImage": "https://www.example.com/images/article1.jpg",
11    "articleAllImages": "https://www.example.com/images/article1.jpg,https://www.example.com/images/article2.png",
12    "articleVideos": "",
13    "articleKeywords": "news, example, article",
14    "articleSummary": "A brief summary of the example news article.",
15    "scrapedAt": "2024-07-27T12:34:56Z",
16    "scrapeSuccess": true,
17    "articleMetaDescription": "An example article for demonstration.",
18    "articleMetaKeywords": "example, article, news, demo"
19  }
20]

Use Cases 💡

Content Marketing & SEO 📢

• Competitor Analysis: Track what your competitors are writing about
• Content Audits: Analyze your own website's content
• Keyword Research: Identify trending topics and keywords
• Backlink Monitoring: Find websites that are linking to your content
• Brand Monitoring: Get alerts for every mention

Market Research & Business Intelligence 📊

• News Aggregation: Build your own news feed
• Trend Analysis: Identify emerging trends and topics
• Sentiment Analysis: Analyze the tone and sentiment of articles
• Information Gathering: Collect data about specific niches

Academic Research 🎓

• Data Collection: Gather data for research papers
• Text Analysis: Analyze large volumes of text data

Other Applications 🌐

• Machine Learning: Train ML models with scraped article data
• Content Curation: Find and share relevant articles with your audience

Getting Started 🚀

1. Find the "Article Scraper & News Content Extractor" in the Apify Store

2. Configure the input:

   • startUrls: An array of URLs to scrape
   • language: (Optional) The expected language of the articles (default: "en")
   • requestTimeout: (Optional) The timeout for each request (default: 7 seconds)
   • fetchImages: (Optional) Whether to fetch images (default: true)
   • proxyConfiguration: Select a proxy configuration
   • browserUserAgent: (Optional) Custom User-Agent

3. Run the Actor

4. Access results in JSON, CSV, Excel, or other formats

5. Optional: Schedule automatic runs, set up webhooks, or integrate with other Apify Actors

Key Benefits 🏆

Data Quality

• ✅ Reliable & Accurate: Uses the robust newspaper3k library
• ✅ Clean Data: Extracts only the relevant information
• ✅ Structured Format: Easy to use and integrate

Platform Advantages

• ✅ Scalable & Serverless: Handles large scraping tasks without infrastructure management
• ✅ Cost-Effective: Pay only for what you use
• ✅ Full Apify Integration: Seamlessly connects with other Apify tools
• ✅ User-Friendly: No coding required
• ✅ Automated Updates: The Actor is maintained and updated regularly

---

Start extracting valuable data from articles today! ➡️