SSD LBA, SQLite4, Cache contention, Tails mirrors, Chosen key encryption, Stale locks

Post date: May 25, 2014 7:39:45 AM

• Article about SSD LBA, no it doesn't mean same thing as the traditonal LBA with hard drives. This new technology should in some cases boost SSD (write) performance up to 300%, in cases when drive is quite full. It's done simply by writing data using different write pattern than earlier. Based on the paper it's nothing special. They just increase fragmentation and decrease amount of data written to disk. These are very traditional trade offs especially with database WAL and LSFS. They didn't tell if it decreases read speed, because even if SSD 'seek times' are low, most drives read random blocks from disk much slower than linear data. Which means that fragmenting data especially in very small blocks will probably cause reduced read speeds. Just as fastest database for writes is ordinary log file, but it's not a database at all. Only problem is that reading from that file is very slow because you have to scan whole file to find what you're looking for or in worst case, several of these files.
  • Reminded my self about SQLite4 design principles. Only thing which makes me bit worried is that they're preferring single key space. Because with SQLite3 it's usually preferable to split data in separate key spaces to improve performance. Instead of putting billion records in one table, it's much better to split those chronologically so that the data from three years is split into monthly tables. Now we got average of 27,7 million keys per table and it's much more manageable amount than having a billion keys. Binary trees tend to grow when you put a lot of stuff in same table, so this is clearly beneficial approach. Total and final performance death hits you hard at the point when the primary key index doesn't fit into system memory anymore. After that everything is going to be extremely slow.
  • Excellent post about cache contention, those are things that most of engineers never even consider thinking about.
  • Actually Tails project was asking for people to setup mirrors. This is one of the reasons you should ALWAYS check download signature, because nothing prevents mirror maintainer delivering you something else than what you were expecting to get.
  • Simple stuff, but a way nice to actually apply it. How to encrypt data using chosen key to get chosen result when decrypting.
• When I said that I wanted to use my own locking library, because I'm sick'n'tired of stale locks, I wasn't kidding. A few days ago I started to wonder why my Deluge isn't working anymore. After rebooting and checking logs, I found out that they had stale lock file in the app path. Reboot or restarting app didn't fix the problem. It required manual intervention and that's exactly what I hate so much. It might not sound bad, but when you have tons of servers and just a few of those fail with some kind of random stuff daily, you're getting really quickly tired of that. Everything should work, and if it doesn't work, reboot should fix it. If it doesn't fix it, it's absolutely totally broken. I don't want to use it.