LLM Web Scraper
LLM Web Scraper API Documentation
Introduction
The LLM Web Scraper API combines web scraping with Large Language Model processing to extract specific information from web pages using natural language prompts. This version provides structured, intelligent data extraction.
Authentication
All API requests require authentication using an API key. Include your key in the x-api-key
header:
Base URL
Pricing
Each API call costs 1 credit.
Endpoint Details
LLM-Enhanced Web Scraping
Extract structured information from a webpage using natural language prompts.
HTTP Method: POST
Endpoint: /
Request Headers
x-api-key
YOUR_API_KEY_HERE
Your unique API authentication key
Content-Type
application/json
Specify JSON request body
Request Body Parameters
url
string
Yes
URL of the web page to scrape
prompt
string
Yes
Natural language prompt describing what to extract
use_llm
boolean
Yes
Must be set to true
for LLM processing
webhook
string
No
Optional webhook URL for receiving response
Example Request
Response Structure
Prompt Guidelines
Be specific about what information you want to extract
Specify the desired format for numerical data
Mention if you want specific sorting or filtering
Include any required unit conversions
Example prompts:
"Extract product specifications in a structured format"
"Find all prices and convert them to USD"
"List main article headings and their first paragraphs"
Error Handling
Common Error Codes:
400: Bad Request (invalid parameters)
401: Unauthorized (invalid API key)
422: Unprocessable Content (LLM processing failed)
429: Too Many Requests
Limitations
Maximum processing time: 20 seconds
Complex prompts may require status checking
Some websites may block automated access
LLM processing may not be 100% accurate
Support
For technical support or to report issues, contact: hey@manojlk.work
Last updated