> For the complete documentation index, see [llms.txt](https://docs.yetanotherapi.com/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.yetanotherapi.com/web-scrapper-deprecated/llm-web-scraper.md).

# LLM Web Scraper

## LLM Web Scraper API Documentation

### Introduction

The LLM Web Scraper API combines web scraping with Large Language Model processing to extract specific information from web pages using natural language prompts. This version provides structured, intelligent data extraction.

### Authentication

All API requests require authentication using an API key. Include your key in the `x-api-key` header:

```
x-api-key: YOUR_API_KEY_HERE
```

### Base URL

```
https://api.yetanotherapi.com/v1/llm-web-scrapper
```

### Pricing

Each API call costs 1 credit.

### Endpoint Details

#### LLM-Enhanced Web Scraping

Extract structured information from a webpage using natural language prompts.

**HTTP Method**: POST\
**Endpoint**: `/`

**Request Headers**

| Header       | Value                | Description                        |
| ------------ | -------------------- | ---------------------------------- |
| x-api-key    | YOUR\_API\_KEY\_HERE | Your unique API authentication key |
| Content-Type | application/json     | Specify JSON request body          |

**Request Body Parameters**

| Parameter | Type    | Required | Description                                        |
| --------- | ------- | -------- | -------------------------------------------------- |
| url       | string  | Yes      | URL of the web page to scrape                      |
| prompt    | string  | Yes      | Natural language prompt describing what to extract |
| use\_llm  | boolean | Yes      | Must be set to `true` for LLM processing           |
| webhook   | string  | No       | Optional webhook URL for receiving response        |

**Example Request**

```bash
curl --location 'https://api.yetanotherapi.com/v1/llm-web-scrapper' \
--header 'x-api-key: $API_KEY_HERE' \
--header 'Content-Type: application/json' \
--data '{
    "url": "https://www.amazon.in/AMVR-Controller-Compatible-Accessories-Adjustable/dp/B0CJRK7B8J",
    "prompt": "Extract the product name, price, rating, and top 3 features",
    "use_llm": true,
    "webhook": "https://your-webhook.com/endpoint"
}'
```

**Response Structure**

```json
{
    "request_id": "c03995eb-e117-4eca-85c8-e6d398a968d9",
    "llm_json_structure": {
        "product_name": "AMVR VR Controller Grip Cover",
        "price": 1499.00,
        "rating": 4.5,
        "top_features": [
            "Anti-slip surface",
            "Adjustable fit",
            "Compatible with Meta Quest 2"
        ]
    }
}
```

### Prompt Guidelines

* Be specific about what information you want to extract
* Specify the desired format for numerical data
* Mention if you want specific sorting or filtering
* Include any required unit conversions

Example prompts:

* "Extract product specifications in a structured format"
* "Find all prices and convert them to USD"
* "List main article headings and their first paragraphs"

### Error Handling

```json
{
    "error": "Invalid prompt format",
    "status_code": 400
}
```

Common Error Codes:

* 400: Bad Request (invalid parameters)
* 401: Unauthorized (invalid API key)
* 422: Unprocessable Content (LLM processing failed)
* 429: Too Many Requests

### Limitations

* Maximum processing time: 20 seconds
* Complex prompts may require status checking
* Some websites may block automated access
* LLM processing may not be 100% accurate

### Support

For technical support or to report issues, contact: <hey@manojlk.work>