FB-Confidentially, Optimistic concurrency control, Durus, My mail server, Pizza Worm, etc.

Post date: Sep 9, 2012 4:18:14 PM

So I closed FB-Confidentially project. What was the application about? Well, it was modified version of the off-the-record application for Facebook. It allowed users to send encrypted messages between users, so that Facebook couldn't track the communication and all data was stored on servers without encryption key. The encryption key was sent in link to message received as facebook notification message. There were also option to use additional password for the message. This password was also hashed and added to original message encryption key. I personally think that this solution was quite nice. It also made for me totally impossible to monitor / read messages, and therefore I couldn't reveal the message content even if i would like to. These kind of encryption solutions are also beneficial to service providers, because they can't be held responsible for the communication, because they can't know message contents. But I already had off-the-record, this FB version was simply made so it would be easier to communicate with my friends privately and I also wanted to learn how to make required Facebook integrations and what steps were actually required to public a service with FB.
Studied: Optimistic concurrency control, Obstruction free, lock free, wait free, cas, ll/sc, atomic commit, database isolation modes, software transactional memory, Durus Python STM library.
Played with Durus Python STM library for a day. I wrote tests app, and got not so great results. One of the reasons must be that I have only quad core cpu and another reason is that the test apps I wrote were really small and processing same data too concurrently causing many of the transactions to be aborted / retried. These are issues that are really complex and task, algoritm etc related. Main point was that I found out Durus to be technically working. When I modified code to use traditional Python locking it did run faster. Yes, I did use multiprocessing of course, not threads even if I'm talking about multithreading here because it's generic term. Most important part of here was that I found out Durus to be technically working as expected. Most important part is that STM does code writing simpler than using traditional (fine grained) locking solutions. Google App Engine's database uses optimistic concurrency control and I have been happy with it. Now I can do just the same with system memory, if I see it to be required. Normally I prefer process pool and allocate tasks so that it's basically shared (write) nothing app. Results from separate processes can be joined after all processes have completed processing the data. It's nice to get tons of ConfilctError exceptions (Durus) and TransactionFailedError (Google App Engine / Datastore), but that's exactly how it's supposed to work. Using Durus might be nice with server (at least 8 cores) and a single task that has to be processed as fast as possible. Tasks should be such that it can't be easily completed with locks or using shared nothing principle.
Configured postfix, dovecot and roundcube again for my server. Still some testing required so I'm sure it's reliable enough, after that server starts to handle my email traffic. I'm sick'n'tired with Yahoo and it's non-reliable services. Btw. Namecheap email forwarding isn't perfect either. Everyone recommeded using MySQL (InnoDB) for message storage, but that's overkill for personal mail server. I decided to use maildir instead. Another reason was that I'm not running MySQL on my server and I'm not planning to. If this all turns out to work well, I'll write super simple UI for my own TrashMail / Spamgourmet kind of service, which I can use reliably with my own domain.
Open Data Center Alliance (ODCA) Library - Quite light stuff, but good if you don't have earlier experience in this field I liked Accelerating Cloud Adoption with Standard Security Measures.
Studied more about Intel's Ivy Bridge and Haswell. - Let's see if Haswell already integrates whole motherboard chipset in CPU. Support for three monitors is also quite nice. I don't play games, so CPU integrated display adapter is just great for me.
Told Daniel how to cheat in Pizza Worm. I think I haven't told too many people how to do that. There are two cheats, another is called huge pizza cheat and another is 10% score bonus. How to activate those? Well, I think podcast Bitar till Kaffet might reveal it in near future. - I was actually writing clone of that "bouncing lines" windows screen saver just for fun. But then it turned into Pizza Worm game. Bouncing lines required that you maintain history of lines drawn on screen and information about the direction where the line ends were moving to. I simply changed line to dot and added keyboard control to replace bouncing and that's it. After a while I added collision detection. First versions of Pizza Worm didn't have music or background image, those were added later. I also used moving buffer instead of circular buffer with first versions, but it was too heavy for my 286/12MHz computer, so I had to modify PW to use circular buffer to avoid doing slow (and pointelss) memory copy instructions.

Sigh, my to be blogged backlog just keeps growing.