Credentials, Security, Superintelligence, AI, Python 3.6 what's new

Post date: Jun 25, 2017 6:53:44 AM

  • Even more wonderful discussion about using default credentials. People just say, that setting device / customer specific credentials is way too hard and complex to maintain. It's just better if everyone is using admin / admin or admin / password. - Sigh. But I'm sure this is nothing new for anyone. It's totally ok to use default credentials everywhere. It makes life much better and simpler.
  • Quite nice article about AI and superintelligence - Collection of approaches to super intelligence aspect from multiple advantage points. It's totally natural to wonder what will be the 'true motivation' behind super intelligence, and how well it can be controlled. Most of people seem to care way too much about things which do not actually make sense in pure logic form. Sometimes it would be good idea to check what's the root cause when you say you've gotta do something. I'm personally having hard time figuring out motivations for some of the stuff I do. Like this blog, why? What's the actual point and so on.

Rest is jargon about Python 3.6 new features.

  • Python 3.6 - What's new? - Yes, I do like f-strings. Yet it seems that people are still looking solutions to do lazy deferred execution, so f-strings could be used as templates. Type hints are also awesome and very welcomed feature. Asynchronous Generators sound awesome. But as I don't need to usually write very high performance code, I'm still very confused about how are these async things actually improve performance, as long as there's the GIL present. I've seen actually some projects to run in huge problems, writing everything in way cool asynchronous way and then finding that it doesn't actually allow ANY parallel execution. It's hard to see how this would bring any better performance than bunch of traditional block threads. But as said, these things can be done in at least hundreds of different ways. Maybe I should write some experimental code to experiment with async code, to see the real benefits. But probably won't do it before, I'll see any practical use for these in my own projects. Pathlib is nice. Simplifies path handling. Using UTF-8 on Windows as default, finally! It has been constant struggle on Windows compared to Linux. New faster, memory conserving and order-preserving dict is also awesome improvement. New secrets module is awesome, no need to use os.urandom for that anymore. Deque pickle is nice. No need to convert those to standard lists anymore before saving. Decimal.as_integer_ratio for fractional (rational) number representation is also very neat, finally. Neat, hashlib now packs BLAKE2 and SHA-3 as well as scrypt. Yahoo, ChaCha20 Poly1305 support added to SSL library. Lot of generic optimizations, very nice.

Dictionary vs list vs deque vs order preserving dictionaries. Lot of speculation.

  • New 3.6 order preserving dictionary (dict). I actually missed one thing. Because dict is preserves order, there should be a way to access items in dict also using index. In pyclock pro some items are tracked using list and dict, because index positions are important. If dict would allow access using index, it would mean that there's no need to maintain parallel list. Which would naturally improve performance quite a lot.. It is possible to pop from dictionary as well as get items or even get keys or iter(dict). But any of these methods do not allow direct index access without using list presentation as proxy. Hmm... I wonder why they haven't thought about this.
  • Just my social media post about the topic: Python 3.6 dictionaries are order-preserving. I'm just wondering why there's no way to access dictionary items by index directly. When dictionary retains order, I would love to see it to be accessible just like a list. It would be quite logical.
    • Of course you can get keys() from dict, or items(), but I would think that's inefficient way, if I know I want to access 5th item in dictionary.
  • Compact dictionary - Again classic reasons. Less memory used -> more cache friendly code. These fundamental changes in Python are awesome. This clearly shows how the construction of basic building blocks, which everything else builds on is extremely important.
  • The PyClockPro - Is one example which seems to be one of the worst use cases for the new dictionaries. Because it does repeatedly remove and insert new keys to keep the cache dictionary at the maximum allowed size.
  • Art of optimization, sigh! At least on rather small cache data sets Like 100 items, it seems that deque is slower than traditional list. Let's try same, but with larger cache. Got so deep, that next post will contain the details.
  • Some continuation to my blog post on social media:
    • Maybe I've missed some essential Python feature or syntax. But afaik, there's no good way to access dictionary by index right now. - That's exactly why I'm asking. Also Collections.OrderedDict doesn't provide by index access. Also keys, items or iter are all very slow methods of accessing specific index. With lists accessing by index is fast.
  • With PyClockPro project I'm now maintaining a list of keys for index access and dictionary for fast key access. But if ordered dictionary would allow direct index access, there wouldn't be a need to maintain separate list for index access. This is something I have to reconsider later, when Python 3.6 is mainstream.
  • Any potential pro-tips about this would be appreciated. This is just the way I figured out to be the fastest so far for this use case. But I'm always looking for improvement.
    • I'm just asking if there's a better way. I'm not proposing any syntax. Of course obvious would be new method to access / get by index.
  • As bonus, it might also be that this compact dictionary would provide better 'random deletion' performance than list or deque, which also provide bad random delete performance. Or is there existing doubly linked list with good random delete / insert performance available in Python standard library? After quick testing, it seems that deque provides also good performance when removing items when item being removed is not first (left) or last (right) item in queue. Yet, if accessing deque list with large index number, it's very slow. Probably due to linked list walk happening behind the scenes.
  • If this wasn't enough. Here's even more: Python 3 - List vs Deque - When to use and why