如何从我的 FastAPI 应用程序向另一个站点 (API) 发送 HTTP 请求?

问题描述

我正在尝试使用以下代码片段一次向服务器 http://httpbin.org/uuid 发送 100 个请求

I am trying to send 100 requests at a time to a server http://httpbin.org/uuid using the following code snippet

from fastapi import FastAPI
from time import sleep
from time import time
import requests
import asyncio

app = FastAPI()

URL= "http://httpbin.org/uuid"


# @app.get("/")
async def main():
    r = requests.get(URL)
    # print(r.text)

    return r.text

async def task():
    tasks = [main(),main(),main(),main(),main(),main(),main(),main(),main(),main(),main(),main(),main(),main(),main(),main(),main(),main(),main(),main(),main(),main(),main(),main(),main(),main(),main(),main(),main(),main(),main(),main(),main(),main(),main(),main(),main(),main(),main(),main(),main(),main(),main(),main(),main(),main(),main(),main(),main(),main(),main(),main(),main(),main(),main(),main(),main(),main(),main(),main(),main(),main(),main(),main(),main(),main(),main(),main(),main(),main(),main(),main(),main(),main(),main(),main(),main(),main(),main(),main(),main(),main(),main(),main(),main(),main(),main(),main(),main(),main(),main(),main(),main(),main(),main(),main(),main(),main(),main(),main()]
    # print(tasks)
    # input("stop")
    result = await asyncio.gather(*tasks)
    print (result)

@app.get('/')
def f():
    start = time()
    asyncio.run(task())
    print("time: ",time()-start)

我将 FastAPI 与 Asyncio 结合使用以实现尽可能短的大约 3 秒或更短的时间，但使用上述方法我得到的总时间为 66 秒，超过一分钟.我还想保留 main 函数以对 r.text 进行附加操作.我知道要实现如此短的时间，需要并发性，但我不确定我在这里犯了什么错误.

I am using FastAPI with Asyncio to achieve the lowest time possible around 3 seconds or less but using the above method I am getting an overall time of 66 seconds that is more than a minute. I also want to keep the main function for additional operations on r.text. I understand that to achieve such low time, concurrency is required but I am not sure what mistake I'm doing here.

推荐答案

requests 是一个同步库.您需要使用基于 asyncio 的库来异步发出数百个请求.

requests is a synchronous library. You need to use an asyncio-based library to make hundreds of requests asynchronously.

httpx.AsyncClient 通常用于 FastAPI 应用程序来请求外部服务.它还用于应用程序的异步测试.默认使用它.

httpx.AsyncClient is typically used in FastAPI applications to request external services. It is also used for asynchronous tests of application. Use it by default.

from fastapi import FastAPI
from time import time
import httpx
import asyncio

app = FastAPI()

URL = "http://httpbin.org/uuid"


async def request(client):
    response = await client.get(URL)
    return response.text


async def task():
    async with httpx.AsyncClient() as client:
        tasks = [request(client) for i in range(100)]
        result = await asyncio.gather(*tasks)
        print(result)


@app.get('/')
async def f():
    start = time()
    await task()
    print("time: ", time() - start)

输出

['{
  "uuid": "65c454bf-9b12-4ba8-98e1-de636bffeed3"
}
', '{
  "uuid": "03a48e56-2a44-48e3-bd43-a0b605bef359"
}
',...
time:  0.5911855697631836

aiohttp

aiohttp 也可以在 FastAPI 应用程序中使用，但如果您确实需要，请这样做.

aiohttp

aiohttp can also be used in FastAPI applications, but do so if you really need it.

from fastapi import FastAPI
from time import time
import aiohttp
import asyncio

app = FastAPI()

URL = "http://httpbin.org/uuid"


async def request(session):
    async with session.get(URL) as response:
        return await response.text()


async def task():
    async with aiohttp.ClientSession() as session:
        tasks = [request(session) for i in range(100)]
        result = await asyncio.gather(*tasks)
        print(result)


@app.get('/')
async def f():
    start = time()
    await task()
    print("time: ", time() - start)

---

如果你想限制并行执行的请求数量，你可以像这样使用asyncio.semaphore:

MAX_IN_PARALLEL = 10
limit_sem = asyncio.Semaphore(MAX_IN_PARALLEL)


async def request(client):
    async with limit_sem:
        response = await client.get(URL)
        return response.text

这篇关于如何从我的 FastAPI 应用程序向另一个站点 (API) 发送 HTTP 请求?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持！