IPv6, vCPU vs CPU, Duplicati, VAT, Profiling, Cold Storage, Mobile Retail, Customer Service, Collections

Post date: May 10, 2015 7:55:47 AM

DNA announced that it's going to provide IPv6, it's the first major operator in Finland to do that. Sonera has been offering limited 6rd so far. DNA IPv6 info.
Reminded my self about Finnish Communications Regulatory Authority's (FICORA) IPv6 regulations and recommendations (200/20014 S, TR-177, RIPE-554, 28 H/2010 M, MPS 28, 13 B/2011, RFC 7084, RFC 4192, RFC 7010, SLAAC, DHCPv6, RA, DHCPv6-PD, Prefix Delegation, DNS, NTP, MRO flags, Managed Configuration, Other Configuration, RFC 4890, ICMPv6, 3GPP R 8, 9, 10, LTE, UMTS, RFC 5969)
In Finland it's recommended that everybody gets at least /56 IPv6 network prefix. It's also recommended that users receive "long term addresses" meaning that customer will be using the same addressing for extended periods (years) and recommended valid address duration is 30 days, which allows address to be refreshed at any point during that time. Long term addressing also makes privacy addressing more or less useful when you can track users based on /56 subnets level alone.
IPv6 is important part of Net Neutrality. Because without IPv6 there's clear discrimination towards users and operators who do not happen to have IPv4 addresses.
Duplicati claims that remote directory does not exist. Without even connecting the server. Classical trap. Error message is 100% misleading and contains pure lies. As well as it doesn't have ANYTHING at all to do with the problem what they're reporting. It's just so annoying to see programs to do stuff like this. Yet on the other hand, when you know how to create absolutely misleading error messages you can place those whenever you want in case it's required. Investigating the problem which has nothing to do with the underlying issue will surely waste a lot of time for the whole team. After having a lunch and thinking about it again, I found the problem. Problem? The IP address isn't white listed on FTP server level. This causes FTP server to completely reject binding attempt -> No log entry about rejected connection. As well as Duplicati gives totally misleading message that remote directory doesn't exist. It should say connection refused. Yet, it just doesn't do that. Giving false information and misleading error messages just wastes everybody's time like it did in this case.
Another favorite type of error is that if you get error 666 after 1 seconds of starting, it means something else than getting it after 5 seconds of starting and so on. When you run the program with unfamiliar environment and get the error 666 after 3 seconds. Which of the 666 errors it is? Have fun!
How to make bullet proof Python programs which never crash? Just user totally generic catch all error handler with nice descriptive error messages and your program won't ever crash. Btw. This awesome error handling module is so generic, that it works for every application. Intuitive and informative error messages also make all other documentation of the project unnecessary saving huge amount of time when developing new applications and software. Don't ever waste your time writing man pages or command line help. Even source code comments are now past.
- Just add to your program:
- try:
- code_here()
- except:
- print('Error')
- # Now you can handle all exceptions and your program never crashes! It's always ehh, controlled termination.
This is funny, people often say more cores is better. But that's not the way. Like in case where there is an option to choose 100% of one Xeon Core or there's option to choose 8x vCPU which shows as 8 cores to virtual machine. But it might be so that even all of those 8 vCPUs combined doesn't produce same performance as the one real Xeon Core from what ever model is being used. So don't get confused about "getting so many cores". It means nothing. And having more cores can be significantly worse than having fewer. Especially in cases where the applications won't utilize the cores fully. Of course this helps in situations which I've described earlier, where really bad code is run on high priority and then "everything is jammed" and adding more cores solved the problem. But uuh. How about just fixing that crappy code or running it on lower priority? Having more slower CPUs can also bring up many kind of locking issues which you wouldn't have encountered otherwise. Race conditions and stuff like that. It's so common that code isn't perfect, there are flaws. But with a few fast cores, it's much less likely to hit these than with tons of really slow CPUs. With systems using traditional locking this also leads to worse lock contention. Because time spent processing data while lock is being held is longer.
Had interesting meeting about mobile payments integration. It's interesting to see what will follow. I know there are projects which are full of hype, lot of resources are spent and nothing practical comes out. But let's see and hope that in this case there would be some kind of useful end product. At least they got good prospects how to get this stuff to work. But isn't that what everybody is saying?
There's so much to do with server subcontracts. Security issues, invoicing procedures and modes. How invoices get handler correctly and correct VAT paid and so on. Some service providers charge all service individually, which is bad, because it produces so many invoices. Then there are service providers which charge like monthly or when there's certain amount of services to be charged used. When that happens they can provide VAT calculation for whole sum at once. This is of course very efficient, but requires a credit account. Then there's the same model which is just pre-paid, so you pay something before hand, then when invoices come, they charge it from that account. But now it's a good question if that account is with or without VAT. In some cases, it's with vat, and that can be a problem. Because account charges do not separate charge sum from VAT. But the VAT should be individually picked from all services which are charged from that account. It's quite problematic way do deal with that. Because if I pay lets say 10000€ including vat, there's no VAT being paid yet. The vat separation happens later from the invoices paid from that account. It's horror for bookkeeping. Some other providers do so that when you charge account for 10000€ they actually invoice 12400€ and the VAT separation from payment happens at that stage. I think personally that it's better way to deal with VAT because it makes things much simpler than the solution where VAT is separated later. Only case where that won't work, is of course cases where services being charged from account can contain multiple different VAT classes. That, if something is going to be interesting. So four different methods. Direct invoicing, batch invoicing based on invoices (via credit), paying from account, where VAT separated when charging and paying from account, where VAT is separated when using the account. It's just wonderful how much mess and discussion you can get from simple topics with accounting department.
Why performance profiling is so important? Well, fixing just a few methods / functions from whole program can make it easily 90% faster. This was the case once again with one project. Over 90% of CPU and memory and database access was caused by single 42 rows / lines long function. Fixing that made everything run just so much faster and lighter.
Got small headache with JavaScript, but via trial and error, reading, and retrying, you'll learn finally. Yes, JavaScript isn't Pythonic, but I'll survive. It just takes bit more time.
Excellent analysis about Aerospike database. This is exactly why you need to know the implementation details well enough. If you just go and assume things, you're going to fail badly.
Was Python 3.X a mistake? I personally think it certainly wasn't. Here's a nice comparison what's the situation with 'new' versions of PERL and PHP.
How to connect IPv4 systems to IPv6 network. That's bad one. Because every system should support IPv6. Yeah, I know. We're all going to deal with that pain at some point in future. Simplest solution would be TCP relay proxy afaik. - Rest of my thoughts at LinkedIn: That's totally application dependent. Operating systems and networks also affect that. If there are some heavy legacy reasons why this can't be done, I would use separate proxy. Of course this case assumes you can freely control the connection points. So instead of connecting to IPv6 only server, connect to IPv4 server which can relay the connection to IPv6 server. That's the easiest thing which I would use in case I would encounter such requirements with legacy hardware which can't use IPv6 at all. Of course there are several ways to do this. But simplest setup would be using normal workstation with proxy / relay software, which is pretty trivial to setup. When connection X comes in, connect to Y and relay all data back and forth. Been there, done that, several times. (Just like ssh port forwarding, or Tor hidden service, nothing special there). You aren't providing any useful information with your questions, so I don't really know what would be the preferred solution. Technically this has nothing to do with NAT. Run one box which connects both networks (v4 / v6). Then you make clients to connect this relay on port X which then in turn connects using TCPv6 the server and port X forwarding all traffic. Potential problems? Security, authentication and of course if the server limits connections by IP or so, now all IPv4 connections seem to be coming from one IP. But I assume this is industrial or something setup, so I'm sure you can work around both of those limits in limited / private network(s). But these are really hard to judge when you don't really know what the connection(s) are being used for.
Under the hood - Facebooks cold storage system. Really nice article. I have to say that they sure did choose no frills approach. Because there was nothing particularly interesting or new. That's just how data is being stored. I've also blogged about Erasure Coding and Reed-Solomon error correction several times. And as they say. It's ages old and well known stuff. So I was bit disappointed after reading the article and finding out that there really wasn't anything new. Only the 1 exabyte scale they're doing it is of course something amazing. But that's just scaling up the concept.
Retailer Mobile Strategies Start With Driving In-Store Sales - What I might like? After all these years and QR code hype. I might be ready to see QR codes on shelve labels which would offer extensive product information on customers own mobile device. As well as stores should provide tables which you can use to get the same information if you don't want to use your own device. It's amazing how often product information which is a key factor to sell, is bad or nearly non-existent. One Finnish on-line store which is "leading the market" is selling backpacks online. Yet they fail to tell the size of the backpack. They just mention the length of the person which the backpack is designed for. What kind of ... Yeah, that's just so much fail. This same chain is especially saying that they prefer to provide perfect product information to customers, because it makes the sales transaction efficient and quick, because there's no need to talk with customer service. Yet they leave key details out of their descriptions.
The customer service approach related to so many aspects on on-line trade. I often hear people saying that they got so good customer service. But my personal opinion is that the company is doing something really wrong if they need customer service and customers need to contact it. I've encountered so many customer service especially with larger service providers which are extremely, nice, polite, what ever. But there' just one fail. How about getting the thing just done, and not waste time talking about it? So companies boasting about their customer services, probably suck. If things work out well without ever contacting the customers service, that's the way things should work out.
Checked out some RIPE IPv6 address allocation stuff. It seems that smallest IPv6 block they're willing to allocate is /32 address space. If you need anything smaller than that, you just should contact your ISP to get that.
My thoughts about new Google+ Collections feature. "Google+ Collections? Well, that's a nice way to group posts. Because I've always hated the way you need to follow a "person" or "page" instead of "topic". This helps a lot. Let's say someone posts 99% of cat pictures and funny theme stuff. But that 1% that is left out is absolutely amazing original technology related posts which I really don't want to miss. Should I see all that 99%? No thank you, please. So now if the poster uses collections correctly, this will fix the major issue I were experiencing. I've got a few people in mind, which I really hope would start using collections properly asap. I also hope that collection could be applied to posts made to communities. Aka traditional cross-posting or labeling."
Nice basic TED talk about AI, what happens when it get's smarter than us? Nothing new there. It's just good to remind your self about this topic every now and then.