Repsonsivity, ShadowCopy, AsyncIO, Free Antivirus

• It's amazing to see how different responsivity levels helpdesks can have. Some helpdesks literally handle issues in almost seconds, often in minutes. But some helpdesks take days, weeks or even months for the same stuff, which other service centers took care of the issue in a few minutes. - This is one thing where customer service automation can help, it makes it possible to cover matters super fast. But in this case I did still mean actual human intervention.
• Configured bunch of systems to use VSS NTFS Volume Shadow Copy feature, to maintain some version history on the server disk, incase quick restoration of some files is required. This is much faster way to recover, from oops, I failed. Than restoring from remote off-site backup system.
• Finally found a while to play with asyncio and tasks and coroutines in Python 3.6.2. Now I think I've got good enough basic understanding to use coroutines in production with cases where the IO is expected to be exceptionally slow. This is great addition to processes, process pools, threads and threadpools. I've been using earlier. - This will also improve performance with one project, where there's high number of slow I/O requests complete. I'm also using aiohttp. Anyway, I'll be using asyncio very sparingly, and only in situations where it's obviously beneficial. I've seen projects where everything is async and code is absolutely full of callback handlers and it makes the project and data flow very hard to follow. But something which is very useful for many of my use cases is the concurrent.futures. It allows you to shortly fan out running multiple slow things (I/O) in parallel and after that's done, continuing with basic single threaded work flow. Best part of it, is the ultra simple ThreadPoolExecutor. This is very Pythonic and nice way, compared to the asyncio librarys complexity. The ThreadPoolExecutor can be used very nicely with with syntax or using executor syntax where tasks are submitted to the pool. Because the returned values can be processed directly from submit, it also creates automatic join and results collection in simplistic easy to follow way. For CPU intensive tasks the ProcessPoolExecutor is the way to go, because it doesn't suffer from GIL limitations. For tasks which might run for longer, and require the control code to continue running, I often used to use ThreadPool and apply_async with callback. Which works very nicely. But this ThreadPoolExecutor / Executor and related Futures are much simpler syntax to implement and follow. Problem with the app. At some point I have to convert one project using ProcessPool and apply_async to ProcessPoolExecutor with Futures just for fun to see how it works in detail. So far I've only made some simple testing code. And used ThreadPoolExecutor for short fan outs. It's also worth of noting that depending from task type, it's much more efficient to use Threads than Processes, because Threads launch over 10x faster than Processes. Even if on Linux process forking is quite efficient. After a while of playing, it's trivial. Now I got all the options working well and similarly. ThreadPool, concurrent.futures and asyncio with callbacks. For the code I did choose the concurrent.futures.ThreadpoolExecutor. It can be used with submit format, which allows pushing tasks asynchronously and getting results with result call. And also the map method works well, which just maps tasks to different threads. I did choose to use the map, because it's the simplest and cleanest form of getting the job done efficiently.
• Tested with a friend bunch of free antivirus software quickly. Immunet, 360 Total Security, Panda Antivirus, Baidu Antivirus. Well, without good test data set / malware collection, tests are pretty useless. But at least those did detect EICAR test file. So all worked at least on some technical level as supposed.

2019-01-27