Infrastructure

Deployment & Operations

Containerized deployment using Docker and Docker Compose for environment consistency across development and production.

Service Architecture

gumtree-monitor

Active scraping component with Python, Selenium, and notifier logic

PostgreSQL (Neon)

External managed database for persistence and reliability

Admin Dashboard

Deployed to Vercel/Netlify for edge performance

Deployment Workflow

1

Clone & Configure

git clone <repo> && cp .env.example .env

Clone repository and set up environment variables

2

Build Images

docker-compose build

Build Docker images for all services

3

Start Services

docker-compose up -d

Launch all containers in detached mode

4

Monitor Logs

docker-compose logs -f gumtree-monitor

Follow container logs for debugging

Environment Configuration

.env
VariableDescriptionDefault
DATABASE_URL*PostgreSQL Connection String—
TWILIO_ACCOUNT_SID*Twilio Credentials—
TWILIO_AUTH_TOKEN*Twilio Auth Token—
TWILIO_WHATSAPP_NUMBER*Sender Number (whatsapp:+...)—
HEADLESSRun Chrome headlesstrue
SCRAPE_INTERVAL_MINMin loop sleep (seconds)300
SCRAPE_INTERVAL_MAXMax loop sleep (seconds)600

* Required variables

Troubleshooting Guide

Selenium Crashes

Increase shm_size to '2gb' in docker-compose.yml. Chrome is memory-hungry.

No Notifications

Check user filters in recipients.json. Ensure lead_quality matches allowed levels.

Database Connection

Verify DATABASE_URL is correct and Neon database is active.