Better Interpretability Leads to Better Adoption Is your highly-trained model easy to understand? A sophisticated machine learning algorithm usually can produce accurate predictions, but its notorious “black box” nature does not help adoption at all. Think about this: If you ask me to swallow a black pill without telling me…

Image classification is the task to recognize an image. It is also called image recognition. Computer scientists have been innovative in extracting meaning from images. Its history is fascinating, though most people don’t know much about it. For this reason, I am going to tell you the stories of innovation…

Many machine learning algorithms are complicated and not easy to understand, even though they have rendered an impressive level of accuracy. As humans, we must be able to fully understand how decisions are being made so that we can trust the decisions of AI systems. We need ML models to…

You receive a data science interview opportunity from your dream company. You have surveyed many the-top-50-question types of articles but still feel uncertain. Since there are already many similar articles, why do I dare to add an article to this crowded topic? In this article, I re-write many ordinary answers…

Machine learning uses a lot of discrete distributions such as the Bernoulli, Binomial, and Multinomial distributions to solve problems. Two related discrete distributions, the Beta and Dirichlet distributions are less known but are widely used in data science. The Dirichlet distribution is especially important in Natural Language Processing (NLP). The…

In Chapter 1, we talked about supervised learning can target better for known outliers, and unsupervised learning can explore new types of outliers. Can we take the advantage of both supervised and unsupervised learning? Specifically, since supervised learning achieves better precision in general, and the outlier scores from unsupervised learning…

Classification problems are often solved using supervised learning algorithms such as Random Forest, Support Vector Machine, Logistic Regressor, and so on. Supervised learning algorithms require a known target to build a model. However, it is often the case that we only see normal data patterns but not rare events. The…

After introducing the Local Outlier Factor (LOF), we can introduce the Cluster-Based Local Outlier Factor (CBLOF). It defines anomalies as a combination of local distances to nearby clusters, and the size of the clusters to which the data point belongs. It first clusters data points into large or small clusters…

The Local Outlier Factor (LOF) is another effective unsupervised learning method for outlier detection. Since its invention in the early 2000s (Breunig et al., 2000 [1]), it has been applied widely to different types of problems. It is a density-based technique that uses the nearest-neighbor search to identify anomalous points…