Pagination

Start with page=1. Increment the page while hasNextPage is true.

items = []

for page in range(1, 4):  # Explicit three-page safety bound.
    response = requests.get(
        "https://api.compsniper.com/v1/scrape",
        headers={"Authorization": f"Bearer {os.environ['COMPSNIPER_API_KEY']}"},
        params={"keyword": "sony wh-1000xm5", "page": page, "count": 240},
        timeout=75,
    )
    response.raise_for_status()
    data = response.json()
    items.extend(data["items"])

    if not data["hasNextPage"]:
        break

Each successful page consumes one request. Production jobs should define both a maximum page count and a maximum execution time.

Count fields

• totalItems: listings returned on the current page, up to 240.
• totalResults: eBay's reported match count as a string, or null when unavailable.
• scrapedCount: upstream rows examined when date filters are applied.
• hasNextPage: whether eBay indicates another result page.

For deeper server-side pagination and CSV export, use Max Mode.