For visual anomaly detection, there are two main approaches: 1) the image reconstruction method and 2) the feature modeling method.
In practical application, the image reconstruction method is difficult to use due to 1) iterative generation process and 2) large errors of regenerating complex texture structures for images.
Up to 2021, PatchCore has shown SOTA performance in visual industrial anomaly detection with more than 99% AUROC score of detecting anomaly regions on the MVTec AD dataset.
The working principle of PatchCore is shown in Figure 1.
In the training stage, PacthCore models the patch features from nomial samples images by utilizing a deep neural network backbone. The author applied WideResNet50-backbone as a Pretrained Encoder/feature extractor to get locally aware patch features.
Because the number of deep features becomes larger when the number of training nominal images increases, Corest-Sampling was applied to construct an efficient memory bank M of these nomial deep features.
In the testing/inference stage, patch features of abnormal images are extracted from the same feature extractor and compared with the features from the memory bank M to get the anomaly score of image patches.
Considering the Pytorch Implementation of PatchCore, please take the reference of GitHub repo: https://github.com/rvorias/ind_knn_ad
- Feature extractor is predefined in the class KNN_extractor() in models.py with timm.create_model() function and True ‘feature_only’ setting.
2. Corest-Sampling is done by the function ‘get_corest_idx_randomp’ from utils.py.
3. To compare the features of test images with features from the memory bank, the code implementation is defined in the predict() of PathCore method in the models.py.
In the end, there are some interesting questions that need to be answered why PatchCore is a successful industrial visual anomaly detector?
Q1. Why deep features generated from the pre-trained encoder with ImageNet are useful?
Q2. Why memory bank and Corest-sampling is better than other KNN methods?
