scraperapi-php-sdk

ScraperAPI — PHP SDK 最佳实践

要求： PHP 7.0+、Composer、composer require scraperapi/sdk、SCRAPERAPI_API_KEY 环境变量。

设置

<?php
require __DIR__ . '/vendor/autoload.php';
use ScraperAPI\Client;

$client = new Client(getenv('SCRAPERAPI_API_KEY'));

切勿将 API 密钥硬编码。每次都从环境变量中读取。

基本用法

// Simple GET — returns HTML string via ->raw_body
$html = $client->get("https://example.com/")->raw_body;
echo $html;

// With a single parameter
$html = $client->get("https://example.com/", ["render" => true])->raw_body;

// With multiple parameters
$html = $client->get(
    "https://example.com/",
    [
        "render"       => true,
        "country_code" => "us",
    ]
)->raw_body;

参数以关联数组的形式传递。->raw_body 用于从响应对象中提取 HTML 字符串。

决策指南

情况	处理方式
单个 URL，同步请求	$client->get($url, $params)->raw_body
页面通过 JavaScript 加载内容	添加 "render" => true
网站屏蔽数据中心代理	添加 "premium" => true
最严格的反机器人防护	添加 "ultra_premium" => true
同一域名上的多步骤 / 分页流程	使用 "session_number"
向目标站点 POST 表单或 JSON 请求体	$client->post($url, $options)->raw_body
20 个以上的 URL 或批量任务	通过 cURL 或 Guzzle 使用异步端点
受支持的平台（Amazon、Google 等）	直接使用结构化数据端点

参数参考

渲染

// Render JavaScript before returning HTML
// Use when: page is a React/Vue/Angular SPA, or scrape returns empty/partial content
// Cost: +10 credits
$html = $client->get("https://spa-site.com/", ["render" => true])->raw_body;

// Wait for a specific DOM element (requires render: true)
$html = $client->get("https://spa-site.com/", [
    "render"            => true,
    "wait_for_selector" => ".product-list",
])->raw_body;

// Screenshot (auto-enables rendering)
$html = $client->get("https://example.com/", ["screenshot" => true])->raw_body;

先不使用 render。仅当响应缺少预期内容时才添加它，因为这会增加费用和延迟。

代理和地理定位

// Route through a country-specific proxy — no extra credit cost
$html = $client->get("https://example.com/", ["country_code" => "de"])->raw_body;

// Premium residential/mobile IPs — for sites that block datacenter proxies
// Cost: 10 credits (25 with render)
$html = $client->get("https://hard-site.com/", ["premium" => true])->raw_body;

// Ultra-premium — for the toughest anti-bot protections
// Cost: 30 credits (75 with render)
// Note: incompatible with custom headers — keep_headers is ignored
$html = $client->get("https://hardest-site.com/", ["ultra_premium" => true])->raw_body;

premium 和 ultra_premium 互斥，切勿同时设置。 升级顺序：标准（1 个 cr）→ render（10 个 cr）→ premium（10 个 cr）→ ultra_premium（30 个 cr）。

会话（粘性代理）

// Reuse the same proxy IP across requests — useful for pagination and multi-step flows
// Sessions expire 15 minutes after last use; any integer is a valid session ID
$html1 = $client->get("https://example.com/page1", ["session_number" => 42])->raw_body;
$html2 = $client->get("https://example.com/page2", ["session_number" => 42])->raw_body;

请求头和设备类型

// Forward custom headers to the target site
// Note: keep_headers is ignored when ultra_premium is true
$html = $client->get("https://example.com/", [
    "keep_headers" => true,
])->raw_body;
// Pass headers in the request options array alongside ScraperAPI params

// Emulate a mobile or desktop browser user-agent
$html = $client->get("https://example.com/", ["device_type" => "mobile"])->raw_body;

自动解析和响应格式

// Return structured JSON instead of HTML for supported sites (Amazon, Google, etc.)
$json = $client->get("https://amazon.com/dp/B09V3KXJPB", ["autoparse" => true])->raw_body;
$data = json_decode($json, true);

// Markdown output — useful for text pipelines
$md = $client->get("https://docs.example.com/", ["output_format" => "markdown"])->raw_body;

POST 请求

// POST a JSON body to the target site through ScraperAPI's proxy
$options = [
    "body"    => json_encode(["key" => "value"]),
    "headers" => ["Content-Type" => "application/json"],
];
$result = $client->post("https://example.com/api", $options)->raw_body;

升级阶梯

始终从成本最低的选项开始，仅在遇到屏蔽时升级。

function scrapeWithEscalation(Client $client, string $url): ?string
{
    $tiers = [
        [],
        ["render"       => true],
        ["premium"      => true],
        ["premium"      => true, "render" => true],
        ["ultra_premium" => true],
    ];

    foreach ($tiers as $params) {
        $html = $client->get($url, $params)->raw_body;
        if ($html && stripos($html, '<html') !== false) {
            return $html;
        }
    }
    return null;
}

异步任务（用于批量处理）

SDK 是同步的——每次 ->get() 调用都会阻塞，直到收到响应（最长 70 秒）。对于 20 个以上的 URL，请使用异步 REST 端点。

$apiKey = getenv('SCRAPERAPI_API_KEY');

function submitJob(string $url, array $apiParams = []): array
{
    global $apiKey;
    $ch = curl_init('https://async.scraperapi.com/jobs');
    curl_setopt_array($ch, [
        CURLOPT_POST           => true,
        CURLOPT_POSTFIELDS     => json_encode(['apiKey' => $apiKey, 'url' => $url, 'apiParams' => $apiParams]),
        CURLOPT_HTTPHEADER     => ['Content-Type: application/json'],
        CURLOPT_RETURNTRANSFER => true,
    ]);
    $response = curl_exec($ch);
    curl_close($ch);
    return json_decode($response, true); // ["id" => "...", "statusUrl" => "..."]
}

function pollJob(array $job, int $maxWait = 120, int $interval = 5): string
{
    $deadline = time() + $maxWait;
    while (time() < $deadline) {
        $data = json_decode(file_get_contents($job['statusUrl']), true);
        if ($data['status'] === 'finished') return $data['response']['body'];
        if ($data['status'] === 'failed')   throw new \RuntimeException("Job {$job['id']} failed");
        sleep($interval);
    }
    throw new \RuntimeException("Job {$job['id']} timed out");
}

// Submit and collect
$urls = ['https://example.com/p1', 'https://example.com/p2'];
$jobs = array_map('submitJob', $urls);
$results = array_map('pollJob', $jobs);

结构化数据端点

对于受支持的平台，请使用结构化端点，而不是原始 HTML——它们会返回干净的 JSON，无需编写解析逻辑。

function structuredGet(string $vertical, array $params = []): array
{
    global $apiKey;
    $query = http_build_query(array_merge(['api_key' => $apiKey], $params));
    $url   = "https://api.scraperapi.com/structured/{$vertical}?{$query}";
    $body  = file_get_contents($url);
    if ($body === false) throw new \RuntimeException("Request failed for {$vertical}");
    return json_decode($body, true);
}

// Google SERP
$results = structuredGet('google/search', ['query' => 'PHP web scraping']);

// Amazon product details
$product = structuredGet('amazon/product', ['asin' => 'B09V3KXJPB']);

// Walmart search
$items = structuredGet('walmart/search', ['query' => 'standing desk', 'tld' => 'com']);

所有垂直领域及必填字段，请参阅结构化数据文档。

错误处理

function safeScrape(Client $client, string $url, array $params = []): ?string
{
    try {
        return $client->get($url, $params)->raw_body;
    } catch (\Exception $e) {
        $status = method_exists($e, 'getCode') ? (int) $e->getCode() : 0;
        switch ($status) {
            case 401: throw new \RuntimeException('Invalid API key — check SCRAPERAPI_API_KEY');
            case 403: throw new \RuntimeException('Blocked or out of credits — try premium or ultra_premium');
            case 429: throw new \RuntimeException('Rate limit — reduce concurrency or switch to async');
            case 500:
            case 503: throw new \RuntimeException('Transient error — retry with exponential backoff');
            default:  throw $e;
        }
    }
}

状态码参考：200 表示成功，401 表示密钥无效，403 表示请求被屏蔽 / 额度不足，404 表示未找到目标， 429 表示请求频率超限，500/503 表示暂时性错误（不会扣费，可安全重试）。

另请参阅重试文档。

额度费用参考

请求类型	额度
标准	1
"render" => true	10
"premium" => true	10
"premium" => true, "render" => true	25
"ultra_premium" => true	30
"ultra_premium" => true, "render" => true	75

在任何请求中添加 "max_cost" => N 可限制额度消耗——如果请求费用超过 N 个额度，则返回 403。

文档

• PHP SDK 入门指南
• SDK 方法参考
• 重试失败的请求
• API 状态码
• 控制面板和额度