Proxy middleware
Proxy middleware automatically sets proxy arguments for all HTTP requests.
Configuration of the middleware is defined in ProxyMiddlewareConfig
.
How to configure middleware for the SneakpeekServer
(will be used globally for all requests):
from aiohttp import BasicAuth
from sneakpeek.middleware.proxy_middleware import ProxyMiddleware, ProxyMiddlewareConfig
server = SneakpeekServer.create(
...
middleware=[
ProxyMiddleware(
ProxyMiddlewareConfig(
proxy = "http://example.proxy.com:3128",
proxy_auth = BasicAuth(login="mylogin", password="securepassword"),
)
)
],
)
How to override middleware settings for a given scraper:
from aiohttp import BasicAuth
from sneakpeek.middleware.proxy_middleware import ProxyMiddlewareConfig
scraper = Scraper(
...
config=ScraperConfig(
...
middleware={
"proxy": ProxyMiddlewareConfig(
proxy = "http://example.proxy.com:3128",
proxy_auth = BasicAuth(login="mylogin", password="securepassword"),
)
}
),
)