Skip to content

Travel price intelligence

Package-holiday price benchmarking for a travel agency, EU market · Built from scratch, solo: scraper, admin console and AI hook-up

Updated

Before

Comparing package prices with competitors by hand did not scale, and collection jobs hung after every outage.

After

~5×

faster collection on one competitor site once it dropped the browser; the browser pool refills itself after outages.

How it works
  1. 1Person:Staff

    Pick a package to compare: departure city, date, length of stay, occupancy.

  2. 2Software:Scheduler

    Starts the benchmark job.

  3. 3Software:Browsers

    Collect competitor prices behind rotating proxies.

    Software:HTTP client

    Reads the one source that needs no browser.

  4. 4Person:Staff

    See the price comparison in the admin console.

Always on

  • After an outage the browser pool refills itself.
  • The client's AI agent can start a benchmark too.

Problem

The agency needed its packages compared against competitor sites by departure city, date, length and occupancy.

Diagnosis

The sites block bots, so stability and proxy cost were the hard parts, not parsing. Jobs hung on an empty browser pool.

Options I weighed

  1. After an outage, jobs hung on an empty browser pool. The fix?

    Chosen

    A pool that refills itself after an outage

    A proxy traffic limit once drained the pool from 40 browsers to 0; now it refills every 60 seconds. Cost: a refill loop to maintain.

    • Rejected

      A fixed-size pool

      After an outage it shrank and stayed small.

  2. Browsers and proxies cost money. Does every source need one?

    Chosen

    Plain HTTP requests for the source that allows them

    One site went from 8 to 43 dates a minute, with no browser to keep alive.

    • Rejected

      A full browser for every source

      Pays browser and proxy costs where none are needed.

Solution

A scheduler runs benchmark jobs on a browser pool with rotating proxies. Staff use an admin console, and the client's AI agent can start benchmarks too.

Collection
  • Python
  • Camoufox
  • Playwright
  • curl_cffi
  • Rotating proxies
  • Apify
API and data
  • FastAPI
  • Supabase
  • PostgreSQL
Admin consoles
  • Next.js
  • React
  • Vite
  • TanStack Query
  • Chart.js
AI agent link
  • MCP server
Quality
  • GitHub Actions
  • gitleaks
  • Vitest
  • pytest

Related services

Figures: Commit log