Flat-File Access with boto3

Use boto3 when you want direct Python access to the bucket. Then you do not use the REST API or the download wrapper of the Python SDK.

List Objects

list_objects.py

1import os
2import boto3
3
4session = boto3.session.Session(
5    aws_access_key_id=os.environ["AWS_ACCESS_KEY_ID"],
6    aws_secret_access_key=os.environ["AWS_SECRET_ACCESS_KEY"],
7    aws_session_token=os.environ["AWS_SESSION_TOKEN"],
8    region_name="auto",
9)
10
11s3 = session.client(
12    "s3",
13    endpoint_url=os.environ["CHD_ENDPOINT"],
14)
15
16response = s3.list_objects_v2(
17    Bucket="cryptohftdata-processed",
18    Prefix="binance_spot/2026-09-02/12/",
19)
20
21for entry in response.get("Contents", []):
22    print(entry["Key"], entry["Size"])

Download A File

download_file.py

1import boto3
2
3s3 = boto3.client(
4    "s3",
5    endpoint_url="https://e5de051a8af1332b58a11bd5b2e40258.r2.cloudflarestorage.com",
6    aws_access_key_id="<TEMP_ACCESS_KEY_ID>",
7    aws_secret_access_key="<TEMP_SECRET_ACCESS_KEY>",
8    aws_session_token="<TEMP_SESSION_TOKEN>",
9    region_name="auto",
10)
11
12s3.download_file(
13    "cryptohftdata-processed",
14    "binance_spot/2026-09-02/12/BTCUSDT_trades.parquet",
15    "BTCUSDT_trades.parquet",
16)

Security Notes

Do not write the temporary credentials directly into production code. Load them from environment variables or from a different short-lived secret source. When the credentials expire, replace them with new credentials.