SQLite3 performance, Python ORM, MongoDB, speed

Post date: Sep 27, 2014 5:23:02 AM

I'm using SQLite for most of my projects, in production and for hobby. When Python ORM is required, I'm using peewee. One of most important features of SQLite is that it's included in Python base libs. One very important factor is that it's also faster than many other databases out there. What? Faster? Yes, because it runs in the process, and doesn't require context change to proceed. So if I make 100k queries looking up information, it's probably fastest using SQLite than other databases, which run in their own process.

Timing 100k reads from database: MongoDB: 43.3 s SQLite: 19.4 s

Same test with 4 parallel threads MongoDB: 29.9 s SQLite: 25.1 s

So as we can see, SQLite is much faster for single thread batch processing that most of other databases. With 4 concurrent threads SQLite is still faster than MongoDB.

About web sites, when using WAL mode, I can handle easily over 200 write transactions / second using very light single core VPS server with SSD. This means that it should be trivial to handle at least a few million hits / day, each with a few write transactions. Basically other things start to block at least with that server, before the pure database lock, write, release cycle becomes the bottle neck.

Also check out this great link: SQLite: Small. Fast. Reliable. Choose any three.