1.8 KiB
Redis vs PostgreSQL Benchmark
Load 125 million phone number records and benchmark read performance between Redis and PostgreSQL
Prerequisites
pip install psycopg2-binary redis locust
PostgreSQL Setup
sudo -u postgres psql
CREATE DATABASE numbers_db; CREATE USER myuser WITH PASSWORD 'mypassword'; GRANT ALL PRIVILEGES ON DATABASE numbers_db TO myuser; \q
Redis Setup
sudo systemctl restart redis-server
Load Data into PostgreSQL
python load_postgres.py
Load Data into Redis
python load_redis.py
Run Benchmark
locust -f main.py
Then open in browser: http://localhost:8089
Default Settings
Total records: 125,000,000 Phone number range: 100,000,000 to 224,999,999 PostgreSQL workers: 12 Redis workers: 8
Expected Results
Redis:
- Average latency: 0.5 to 2 ms
- P99 latency: 3 to 8 ms
- Throughput: 40,000 to 60,000 req/s
- Load time: 15 to 20 minutes
PostgreSQL:
- Average latency: 2 to 10 ms
- P99 latency: 15 to 40 ms
- Throughput: 10,000 to 20,000 req/s
- Load time: 20 to 30 minutes
File Structure
load_postgres.py - PostgreSQL data loader load_redis.py - Redis data loader main.py - Locust benchmark script
Advanced Settings
In load_postgres.py: NUM_WORKERS = 12 BATCH_SIZE = 2_000_000
In load_redis.py: NUM_WORKERS = 8 BATCH_SIZE = 100_000
Troubleshooting
PostgreSQL connection issue: sudo systemctl status postgresql sudo ufw allow 5432
Redis connection issue: sudo systemctl status redis-server redis-cli ping
Out of memory: Reduce NUM_WORKERS to 4 in loader files
Important Notes
PostgreSQL table is created as UNLOGGED (faster speed) Each Locust user has a dedicated database connection Response time is pure query time (excluding connection) Phone numbers are sequential starting from 100 million
Cleanup
Remove PostgreSQL database: DROP DATABASE numbers_db; DROP USER myuser;
Clear Redis: redis-cli FLUSHALL