Discrimination, Statistics, Bayesian, Data, Analytics

Post date: Mar 26, 2017 4:28:53 PM

Warning, this is n/n (nothing / nothing) post. This doesn't actually provide any value for me, nor it provides any value for anyone else. So only read if you're really bored and want to think complex issues where there's no right answer.

Just some random bits of deeper thoughts, or actually mostly questions. I don't have answer or 'right' opinion to the any of the issues(?) mentioned in this post.
Final question is that how it can be discriminatory if it's based on data? It's not something that would be inherently designed to be discriminatory, it's just something which reflects reality. Afaik, I wouldn't define that as discrimination. So many things are related and work as proxy for something else. So even if the results might look like discriminatory, it's not. It's just based on how things are. I don't personally like when people say that you can't say that, it's not right thing, or it's wrong or something else. No, it's not wrong if it's based on real facts and data. Even if it would be wrong on some ethical incriminatory values or hurt someones feelings. It's still fact based and true and therefore not discrimination or whatever you want to call that in different situations. Yet all this technology allows people to be processed (hah, what a horrible term) as individuals. Instead of larger aggregate groups. Yet, as said, people still often are profiled very much like their aggregate groups. But of course it's possible to be individual without typical grouping and current big data technology makes it possible. It's another question if the default grouping when there isn't enough data can discriminate someone. Probably, but isn't this the current case in US with the credit score stuff. So what if I don't have any credit score. It doesn't mean I wouldn't be credit worthy. Is that discrimination?
It's also funny if there's group A and B. B is the group which commits 90% of crimes. It doesn't mean everyone in group B would be criminal. But still, if there are "random" checks on street to see if they've got stolen goods. I would say it's totally fair and purely logical to focus 90% of the search effort on people belonging to group B. That A / B grouping could be anything, sex, religion, wearing a rabbi hat, having specific brand of jeans, having a gang symbol, specific country's passport etc. It's all just Bayesian math. Naturally all of these features can be combined in overall estimate. Is that discrimination?
Sometimes it sounds like 'anti-discrimination' laws are more laws to discriminate others in favor of some small minorities. I don't know? But this is the feeling I get at times. Also lot of effort might be 'wasted?' to fit in some minorities to some situations, etc. This of course totally in sense of efficiency and generalization. When making business decisions, there's usually careful consideration. It's not based on discrimination or not, but it's consideration to get the best end result. It might seem like discrimination at times, but it's not. That's not the key factor. Is it discrimination against operating system / vendor? If other operating system, database, server platform, cloud service, etc providers more cost efficient and over all better solution? Of course they claim that they've got generic market share of % in this market, and we should have quota of % in our servers to utilize their technology? Wouldn't that sound crazy? Sometimes vendors seem to think like this. But I can assure, I don't care about the vendor, I assure you care about the overall package. Also unions seem to have sometimes interesting views about salaries etc.Same job, same salary. But what if one employee is 5x more productive than others? Etc. These are complex and generally hard topics to talk about. I don't have any kind of 'right' or fixed view for this. But it all comes to fair overall consideration what's the best way to deal with the situation.
Similar questions go about countries and communities. Should you have freedom of speech? What if you're speaking against the 'organization' you belong to? Yes, it could be a country as well as company or some other community. Would it be just better to leave and than trying to undermine the organization? If Singapore is kind of police state so what, nobody can deny they're being highly successful. - These are all extremely complex questions, but still worth of asking. This is also relevant question according to the minorities and ethnicity. "When in Rome, do as the Romans do" What if they come here, and want us to do as they please? Is that right then?
Yet we've seen some stories where the input data for this statistical analysis (sometimes hyped as machine learning) can be highly misleading. In one story which I couldn't find right now, they had system to do medical analysis. The dataset they used was from single stage of the process, analysis made by doctor. But the results were highly skewed because in very serious cases the paramedics already took the patient to the next step passing by the doctor. Therefore the machine learning process didn't recognize the very serious cases, because the data set created from the doctor analyses lacked the samples. Unfortunately this is totally normal 'process' failure, which I've described happening over and over again in this blog. Then nerds claim, it's working, even if in reality it's killing people. - But that's just matter of perspective.
I think I've actually written about every topic mentioned in tis post earlier, so nothing new. No great conclusions to be made, just questions without any answers. Also all of these things are very fungible depending on perspective and position being viewed from.
Just as intentionally provocative setup. Everyone's got right for food. But is it right, that poor people get it free, and rich people have to pay for it? Is that fair then? - Yes, I know, this is intentionally provocative setup. But provides one of the complex questions which arise when reading election related questions and answers. - I'm also aware how socialist some European countries are. - Is that a bad thing? Can't say, too complex situation.