Authentication

Crawlingo provides built-in authentication helpers for common web authentication schemes including Basic, Bearer, API Keys, Cookies, and Dynamic OAuth2 tokens.

4 min read Updated July 2026

Basic Auth

basic_auth.py
python
1from crawlingo.auth import BasicAuth
2
3auth = BasicAuth("username", "password")
4session.headers(auth.headers())
5# Automatically sets: Authorization: Basic base64(username:password)

Bearer Token

bearer_token.py
python
1from crawlingo.auth import BearerAuth
2
3auth = BearerAuth("your-secret-token")
4session.headers(auth.headers())
5# Automatically sets: Authorization: Bearer your-secret-token

Custom Header Auth

header_auth.py
python
1from crawlingo.auth import HeaderAuth
2
3auth = HeaderAuth("X-API-Key", "abc123456")
4session.headers(auth.headers())
5# Sets: X-API-Key: abc123456

API Key Query Parameter

query_auth.py
python
1from crawlingo.auth import ApiKeyQueryAuth
2
3# Appends ?api_key=xyz to every request URL
4auth = ApiKeyQueryAuth("api_key", "xyz123")
5# Session automatically appends query params on dispatch

Dynamic Auth (OAuth2 / Token Refresh)

dynamic_auth.py
python
1from crawlingo.auth import DynamicAuth
2import requests
3
4def refresh_token():
5 response = requests.post("https://auth.example.com/oauth/token", json={
6 "grant_type": "client_credentials",
7 "client_id": "my_client_id",
8 "client_secret": "my_client_secret"
9 })
10 return response.json()["access_token"]
11
12# Refreshes automatically 60s before token expiration
13auth = DynamicAuth(refresh_token, min_validity_secs=60)
14session.headers(auth.headers())