Skip to content

http_error_middleware

from pulsefire.middlewares import http_error_middleware

HTTP error middleware.

Should be positioned as late as possible and before rate limiter middlewares (if any) in the client middlewares list.

Responses are handled differently based on their HTTP status:

Status Measures
2XX Return response.
3XX Raise aiohttp.ClientResponseError.
4XX Raise aiohttp.ClientResponseError.
429 Exponential retries (2^n).
5XX Exponential retries (2^n).
Conn Exponential retries (2^n).

Example: 

http_error_middleware(3)

Parameters:

Name Type Description Default
max_retries int

Number of retries to perform before giving up.

 3

Raises:

Type Description
ClientResponseError

When retries have exhausted.
Source code in pulsefire/middlewares.py 
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
def http_error_middleware(max_retries: int = 3):
    """HTTP error middleware.

    Should be positioned as late as possible and before rate limiter middlewares
    (if any) in the client middlewares list.

    Responses are handled differently based on their HTTP status:

    | Status | Measures                              |
    | ------ | ------------------------------------- |
    | 2XX    | Return response.                      |
    | 3XX    | Raise `aiohttp.ClientResponseError`.  |
    | 4XX    | Raise `aiohttp.ClientResponseError`.  |
    | 429    | Exponential retries (2^n).            |
    | 5XX    | Exponential retries (2^n).            |
    | Conn   | Exponential retries (2^n).            |

    Example:
    ```python
    http_error_middleware(3)
    ```

    Parameters:
        max_retries: Number of retries to perform before giving up.

    Raises:
        aiohttp.ClientResponseError: When retries have exhausted.
    """

    def constructor(next: MiddlewareCallable):

        async def middleware(invocation: Invocation):
            last_response: aiohttp.ClientResponse | None = None
            last_connexc: aiohttp.ClientConnectionError | asyncio.TimeoutError = asyncio.TimeoutError()
            for attempt in range(max_retries + 1):
                if attempt:
                    await asyncio.sleep(2 ** attempt)
                try:
                    response: aiohttp.ClientResponse = await next(invocation)
                except (asyncio.TimeoutError, aiohttp.ClientConnectionError) as connexc:
                    last_connexc = connexc
                    continue
                last_response = response
                if 300 > response.status >= 200:
                    return response
                if not (response.status == 429 or response.status >= 500):
                    response.raise_for_status()
            else:
                if last_response:
                    last_response.raise_for_status()
                raise last_connexc

        return middleware

    return constructor