异步发送多个goutte请求

问题描述

这是我正在使用的代码

require_once 'goutte.phar';
use Goutte\Client;
$client = new Client();
for($i=0;$i<10;$i++){
     $crawler = $client->request('GET', 'http://website.com');
     echo '<p>'.$crawler->filterXpath('//meta[@property="og:description"]')->attr('content').'</p>';
     echo '<p>'.$crawler->filter('title')->text().'</p>';
}

这可行，但是需要很多时间来处理?有什么方法可以更快地做到这一点.

This works but takes a lot of time to process? Is there any way to do it faster.

推荐答案

对于初学者来说，您的代码示例没有任何异步之处.这意味着您的应用程序将顺序执行，执行get请求，等待响应，解析响应然后循环返回.

For starters, there is nothing asynchronous about your code sample. Which means that your application will sequentially, perform a get request, wait for the response, parse the response and then loop back.

虽然Goutte在内部使用Guzzle，但并未使用Guzzles异步功能.

While Goutte uses Guzzle internally, it does not make use of Guzzles asynchronous capabilities.

要真正使您的代码异步，您需要在以下位置参考Guzzle文档:

To truly make your code asynchronous you will want to refer to the Guzzle Documentation on:

• 在池中发送请求
• 异步响应处理

• Sending Requests within a Pool
• Asynchronous Response Handling

您上面的代码示例将产生以下结果:

Your code sample above would result in something like:

require 'vendor/autoload.php' //assuming composer package management.

$client = new GuzzleHttp\Client();

$requests = [
    $client->createRequest('GET', $url1),
    $client->createRequest('GET', $url2),
    $client->createRequest('GET', $url3),
    $client->createRequest('GET', $url4),
    $client->createRequest('GET', $url5),
    $client->createRequest('GET', $url6),
    $client->createRequest('GET', $url7),
    $client->createRequest('GET', $url8),
    $client->createRequest('GET', $url9),
    $client->createRequest('GET', $url10),
];

$options = [
    'complete' => [
        [
            'fn' => function (CompleteEvent $event) {
                $crawler = new Symfony\Component\DomCrawler\Crawler(null, $event->getRequest()->getUrl());
                $crawler->addContent($event->getResponse->getBody(), $event->getResponse()->getHeader('Content-Type'));
                echo '<p>'.$crawler->filterXpath('//meta[@property="og:description"]')->attr('content').'</p>';
                echo '<p>'.$crawler->filter('title')->text().'</p>';
            },
            'priority' => 0,    // Optional
            'once'     => false // Optional
        ]
    ]
];

$pool = new GuzzleHttp\Pool($client, $requests, $options);

$pool->wait();

这篇关于异步发送多个goutte请求的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持！