Better Interpretability Leads to Better Adoption Is your highly-trained model easy to understand? A sophisticated machine learning algorithm usually can produce accurate predictions, but its notorious “black box” nature does not help adoption at all. Think about this: If you ask me to swallow a black pill without telling me…

Image classification is the task to recognize an image. It is also called image recognition. Computer scientists have been innovative in extracting meaning from images. Its history is fascinating, though most people don’t know much about it. For this reason, I am going to tell you the stories of innovation…

Many machine learning algorithms are complicated and not easy to understand, even though they have rendered an impressive level of accuracy. As humans, we must be able to fully understand how decisions are being made so that we can trust the decisions of AI systems. We need ML models to…

You receive a data science interview opportunity from your dream company. You have surveyed many the-top-50-question types of articles but still feel uncertain. Since there are already many similar articles, why do I dare to add an article to this crowded topic? In this article, I re-write many ordinary answers…

Anomaly detection is the detection of any rare events that deviate significantly from the majority of the data. Those rare events do not conform to a well-defined behavior. They are also called Outliers, noises, novelties, or exceptions. Rare events can detrimentally impact the business operation and result in a significant…

In Chapter 1, we talked about supervised learning can target better for known outliers, and unsupervised learning can explore new types of outliers. Can we take the advantage of both supervised and unsupervised learning? …

Classification problems are often solved using supervised learning algorithms such as Random Forest, Support Vector Machine, Logistic Regressor, and so on. Supervised learning algorithms require a known target to build a model. However, it is often the case that we only see normal data patterns but not rare events. The…

After introducing the Local Outlier Factor (LOF), we can introduce the Cluster-Based Local Outlier Factor (CBLOF). It defines anomalies as a combination of local distances to nearby clusters, and the size of the clusters to which the data point belongs. It first clusters data points into large or small clusters…

The Local Outlier Factor (LOF) is another effective unsupervised learning method for outlier detection. Since its invention in the early 2000s (Breunig et al., 2000 [1]), it has been applied widely to different types of problems. It is a density-based technique that uses the nearest-neighbor search to identify anomalous points…