Redis vs PostgreSQL Benchmark

Load 125 million phone number records and benchmark read performance between Redis and PostgreSQL

Prerequisites

pip install psycopg2-binary redis locust

PostgreSQL Setup

sudo -u postgres psql

CREATE DATABASE numbers_db; CREATE USER myuser WITH PASSWORD 'mypassword'; GRANT ALL PRIVILEGES ON DATABASE numbers_db TO myuser; \q

Redis Setup

sudo systemctl restart redis-server

Load Data into PostgreSQL

python load_postgres.py

Load Data into Redis

python load_redis.py

Run Benchmark

locust -f main.py

Then open in browser: http://localhost:8089

Default Settings

Total records: 125,000,000 Phone number range: 100,000,000 to 224,999,999 PostgreSQL workers: 12 Redis workers: 8

Expected Results

Redis:

• Average latency: 0.5 to 2 ms
• P99 latency: 3 to 8 ms
• Throughput: 40,000 to 60,000 req/s
• Load time: 15 to 20 minutes

PostgreSQL:

• Average latency: 2 to 10 ms
• P99 latency: 15 to 40 ms
• Throughput: 10,000 to 20,000 req/s
• Load time: 20 to 30 minutes

File Structure

load_postgres.py - PostgreSQL data loader load_redis.py - Redis data loader main.py - Locust benchmark script

Advanced Settings

In load_postgres.py: NUM_WORKERS = 12 BATCH_SIZE = 2_000_000

In load_redis.py: NUM_WORKERS = 8 BATCH_SIZE = 100_000

Troubleshooting

PostgreSQL connection issue: sudo systemctl status postgresql sudo ufw allow 5432

Redis connection issue: sudo systemctl status redis-server redis-cli ping

Out of memory: Reduce NUM_WORKERS to 4 in loader files

Important Notes

PostgreSQL table is created as UNLOGGED (faster speed) Each Locust user has a dedicated database connection Response time is pure query time (excluding connection) Phone numbers are sequential starting from 100 million

Cleanup

Remove PostgreSQL database: DROP DATABASE numbers_db; DROP USER myuser;

Clear Redis: redis-cli FLUSHALL