Computer Science › Artificial Intelligence

Anomaly Detection Techniques and Applications

Works Count: 60588

Wikipedia

Description

This cluster of papers focuses on the detection of anomalies in high-dimensional data, particularly in the context of video analysis, surveillance, and time series data. It covers a wide range of techniques including unsupervised learning, outlier detection, deep learning, and novelty detection for identifying abnormal patterns and events.

Keywords

Anomaly Detection; Unsupervised; Outlier Detection; Deep Learning; High-Dimensional Data; Video Analysis; Neural Networks; Novelty Detection; Surveillance; Time Series

Algorithms for Mining Distance-Based Outliers in Large Datasets

1998-08-24

Edwin M. Knorr , Raymond T. Ng

This paper deals with finding outliers (exceptions) in large, multidimensional datasets. The identification of outliers can lead to the discovery of truly unexpected knowledge in areas such as electronic commerce, … This paper deals with finding outliers (exceptions) in large, multidimensional datasets. The identification of outliers can lead to the discovery of truly unexpected knowledge in areas such as electronic commerce, credit card fraud, and even the analysis of performance statistics of professional athletes. Existing methods that we have seen for finding outliers in large datasets can only deal efficiently with two dimensions/attributes of a dataset. Here, we study the notion of DB- (DistanceBased) outliers. While we provide formal and empirical evidence showing the usefulness of DB-outliers, we focus on the development of algorithms for computing such outliers. First, we present two simple algorithms, both having a complexity of O(k N’), k being the dimensionality and N being the number of objects in the dataset. These algorithms readily support datasets with many more than two attributes. Second, we present an optimized cell-based algorithm that has a complexity that is linear wrt N, but exponential wrt k. Third, for datasets that are mainly disk-resident, we present another version of the cell-based algorithm that guarantees at most 3 passes over a dataset. We provide

Support Vector Machines

2020-01-30

Mohammed J. Zaki , Wagner Meira

The fundamental algorithms in data mining and machine learning form the basis of data science, utilizing automated methods to analyze patterns and models for all kinds of data in applications … The fundamental algorithms in data mining and machine learning form the basis of data science, utilizing automated methods to analyze patterns and models for all kinds of data in applications ranging from scientific discovery to business analytics. This textbook for senior undergraduate and graduate courses provides a comprehensive, in-depth overview of data mining, machine learning and statistics, offering solid guidance for students, researchers, and practitioners. The book lays the foundations of data analysis, pattern mining, clustering, classification and regression, with a focus on the algorithms and the underlying algebraic, geometric, and probabilistic concepts. New to this second edition is an entire part devoted to regression methods, including neural networks and deep learning.

Machine learning: Trends, perspectives, and prospects

2015-07-16

Michael I. Jordan , Tom M. Mitchell

Machine learning addresses the question of how to build computers that improve automatically through experience. It is one of today’s most rapidly growing technical fields, lying at the intersection of … Machine learning addresses the question of how to build computers that improve automatically through experience. It is one of today’s most rapidly growing technical fields, lying at the intersection of computer science and statistics, and at the core of artificial intelligence and data science. Recent progress in machine learning has been driven both by the development of new learning algorithms and theory and by the ongoing explosion in the availability of online data and low-cost computation. The adoption of data-intensive machine-learning methods can be found throughout science, technology and commerce, leading to more evidence-based decision-making across many walks of life, including health care, manufacturing, education, financial modeling, policing, and marketing.

Support Vector Data Description

2003-12-16

David M. J. Tax , Robert P. W. Duin

View PDF Chat

Big Data Deep Learning: Challenges and Perspectives

2014-01-01

Xuewen Chen , Xiaotong Lin

Deep learning is currently an extremely active research area in machine learning and pattern recognition society. It has gained huge successes in a broad area of applications such as speech … Deep learning is currently an extremely active research area in machine learning and pattern recognition society. It has gained huge successes in a broad area of applications such as speech recognition, computer vision, and natural language processing. With the sheer size of data available today, big data brings big opportunities and transformative potential for various sectors; on the other hand, it also presents unprecedented challenges to harnessing data and information. As the data keeps getting bigger, deep learning is coming to play a key role in providing big data predictive analytics solutions. In this paper, we provide a brief overview of deep learning, and highlight current research efforts and the challenges to big data, as well as the future trends.

View PDF Chat

Isolation-Based Anomaly Detection

2012-03-01

Fei Tony Liu , Kai Ming Ting , Zhi‐Hua Zhou

Anomalies are data points that are few and different. As a result of these properties, we show that, anomalies are susceptible to a mechanism called isolation . This article proposes … Anomalies are data points that are few and different. As a result of these properties, we show that, anomalies are susceptible to a mechanism called isolation . This article proposes a method called Isolation Forest ( i Forest), which detects anomalies purely based on the concept of isolation without employing any distance or density measure---fundamentally different from all existing methods. As a result, i Forest is able to exploit subsampling (i) to achieve a low linear time-complexity and a small memory-requirement and (ii) to deal with the effects of swamping and masking effectively. Our empirical evaluation shows that i Forest outperforms ORCA, one-class SVM, LOF and Random Forests in terms of AUC, processing time, and it is robust against masking and swamping effects. i Forest also works well in high dimensional problems containing a large number of irrelevant attributes, and when anomalies are not available in training sample.

Distance-based outliers: algorithms and applications

2000-02-01

Edwin M. Knorr , Raymond T. Ng , Vladimir Tucakov

Graph based anomaly detection and description: a survey

2014-07-04

Leman Akoglu , Hanghang Tong , Danai Koutra

View PDF Chat

Novelty detection: a review—part 1: statistical approaches

2003-09-17

M. Markou , Sameer Singh

Support vector domain description

1999-11-01

David M. J. Tax , Robert P. W. Duin

Support Vector Method for Novelty Detection

1999-11-29

Bernhard Schölkopf , Robert C. Williamson , Alex Smola +2 more

Suppose you are given some dataset drawn from an underlying probability distribution P and you want to estimate a simple subset S of input space such that the probability that … Suppose you are given some dataset drawn from an underlying probability distribution P and you want to estimate a simple subset S of input space such that the probability that a test point drawn from P lies outside of S equals some a priori specified ν between 0 and 1. We propose a method to approach this problem by trying to estimate a function f which is positive on S and negative on the complement. The functional form of f is given by a kernel expansion in terms of a potentially small subset of the training data; it is regularized by controlling the length of the weight vector in an associated feature space. We provide a theoretical analysis of the statistical performance of our algorithm. The algorithm is a natural extension of the support vector algorithm to the case of unlabelled data.

A Bayesian computer vision system for modeling human interactions

2000-01-01

Nuria Oliver , Barbara Rosario , Alex Pentland

We describe a real-time computer vision and machine learning system for modeling and recognizing human behaviors in a visual surveillance task. The system deals in particularly with detecting when interactions … We describe a real-time computer vision and machine learning system for modeling and recognizing human behaviors in a visual surveillance task. The system deals in particularly with detecting when interactions between people occur and classifying the type of interaction. Examples of interesting interaction behaviors include following another person, altering one's path to meet another, and so forth. Our system combines top-down with bottom-up information in a closed feedback loop, with both components employing a statistical Bayesian approach. We propose and compare two different state-based learning architectures, namely, HMMs and CHMMs for modeling behaviors and interactions. Finally, a synthetic "Alife-style" training system is used to develop flexible prior models for recognizing human interactions. We demonstrate the ability to use these a priori models to accurately classify real human behaviors and interactions with no additional tuning or training.

A review of novelty detection

2014-01-03

Marco A. F. Pimentel , David A. Clifton , Lei Clifton +1 more

Anomaly detection

2009-07-01

Varun Chandola , Arindam Banerjee , Vipin Kumar

Anomaly detection is an important problem that has been researched within diverse research areas and application domains. Many anomaly detection techniques have been specifically developed for certain application domains, while … Anomaly detection is an important problem that has been researched within diverse research areas and application domains. Many anomaly detection techniques have been specifically developed for certain application domains, while others are more generic. This survey tries to provide a structured and comprehensive overview of the research on anomaly detection. We have grouped existing techniques into different categories based on the underlying approach adopted by each technique. For each category we have identified key assumptions, which are used by the techniques to differentiate between normal and anomalous behavior. When applying a given technique to a particular domain, these assumptions can be used as guidelines to assess the effectiveness of the technique in that domain. For each category, we provide a basic anomaly detection technique, and then show how the different existing techniques in that category are variants of the basic technique. This template provides an easier and more succinct understanding of the techniques belonging to each category. Further, for each category, we identify the advantages and disadvantages of the techniques in that category. We also provide a discussion on the computational complexity of the techniques since it is an important issue in real application domains. We hope that this survey will provide a better understanding of the different directions in which research has been done on this topic, and how techniques developed in one area can be applied in domains for which they were not intended to begin with.

Efficient algorithms for mining outliers from large data sets

2000-05-16

Sridhar Ramaswamy , Rajeev Rastogi , Kyuseok Shim

In this paper, we propose a novel formulation for distance-based outliers that is based on the distance of a point from its k th nearest neighbor. We rank each point … In this paper, we propose a novel formulation for distance-based outliers that is based on the distance of a point from its k th nearest neighbor. We rank each point on the basis of its distance to its k th nearest neighbor and declare the top n points in this ranking to be outliers. In addition to developing relatively straightforward solutions to finding such outliers based on the classical nested-loop join and index join algorithms, we develop a highly efficient partition-based algorithm for mining outliers. This algorithm first partitions the input data set into disjoint subsets, and then prunes entire partitions as soon as it is determined that they cannot contain outliers. This results in substantial savings in computation. We present the results of an extensive experimental study on real-life and synthetic data sets. The results from a real-life NBA database highlight and reveal several expected and unexpected aspects of the database. The results from a study on synthetic data sets demonstrate that the partition-based algorithm scales well with respect to both data set size and data set dimensionality.

One-class svms for document classification

2002-03-01

Larry M. Manevitz , Malik Yousef

We implemented versions of the SVM appropriate for one-class classification in the context of information retrieval. The experiments were conducted on the standard Reuters data set. For the SVM implementation … We implemented versions of the SVM appropriate for one-class classification in the context of information retrieval. The experiments were conducted on the standard Reuters data set. For the SVM implementation we used both a version of Schoelkopf et al. and a somewhat different version of one-class SVM based on identifying outlier data as representative of the second-class. We report on experiments with different kernels for both of these implementations and with different representations of the data, including binary vectors, tf-idf representation and a modification called Hadamard representation. Then we compared it with one-class versions of the algorithms prototype (Rocchio), nearest neighbor, naive Bayes, and finally a natural one-class neural network classification method based on bottleneck compression generated filters.The SVM approach as represented by Schoelkopf was superior to all the methods except the neural network one, where it was, although occasionally worse, essentially comparable. However, the SVM methods turned out to be quite sensitive to the choice of representation and kernel in ways which are not well understood; therefore, for the time being leaving the neural network approach as the most robust.

A Survey of Outlier Detection Methodologies

2004-10-01

Joanna Hodge , Jim Austin

View PDF Chat

LOF

2000-05-16

Markus Breunig , Hans‐Peter Kriegel , Raymond T. Ng +1 more

For many KDD applications, such as detecting criminal activities in E-commerce, finding the rare instances or the outliers, can be more interesting than finding the common patterns. Existing work in … For many KDD applications, such as detecting criminal activities in E-commerce, finding the rare instances or the outliers, can be more interesting than finding the common patterns. Existing work in outlier detection regards being an outlier as a binary property. In this paper, we contend that for many scenarios, it is more meaningful to assign to each object a degree of being an outlier. This degree is called the local outlier factor (LOF) of an object. It is local in that the degree depends on how isolated the object is with respect to the surrounding neighborhood. We give a detailed formal analysis showing that LOF enjoys many desirable properties. Using real-world datasets, we demonstrate that LOF can be used to find outliers which appear to be meaningful, but can otherwise not be identified with existing approaches. Finally, a careful performance evaluation of our algorithm confirms we show that our approach of finding local outliers can be practical.

View PDF Chat

Learning Temporal Regularity in Video Sequences

2016-06-01

Mahmudul Hasan , Jonghyun Choi , Jan Neumann +2 more

Perceiving meaningful activities in a long video sequence is a challenging problem due to ambiguous definition of 'meaningfulness' as well as clutters in the scene. We approach this problem by … Perceiving meaningful activities in a long video sequence is a challenging problem due to ambiguous definition of 'meaningfulness' as well as clutters in the scene. We approach this problem by learning a generative model for regular motion patterns (termed as regularity) using multiple sources with very limited supervision. Specifically, we propose two methods that are built upon the autoencoders for their ability to work with little to no supervision. We first leverage the conventional handcrafted spatio-temporal local features and learn a fully connected autoencoder on them. Second, we build a fully convolutional feed-forward autoencoder to learn both the local features and the classifiers as an end-to-end learning framework. Our model can capture the regularities from multiple datasets. We evaluate our methods in both qualitative and quantitative ways - showing the learned regularity of videos in various aspects and demonstrating competitive performance on anomaly detection datasets as an application.

View PDF Chat

A survey of deep neural network architectures and their applications

2016-12-18

Weibo Liu , Zidong Wang , Xiaohui Liu +3 more

Unsupervised Anomaly Detection with Generative Adversarial Networks to Guide Marker Discovery

2017-01-01

Thomas Schlegl , Philipp Seeböck , Sebastian M. Waldstein +2 more

View PDF Chat

hdbscan: Hierarchical density based clustering

2017-03-21

Leland McInnes , John Healy , Steve Astels

Efficient kNN Classification With Different Numbers of Nearest Neighbors

2017-04-12

Shichao Zhang , Xuelong Li , Ming Zong +2 more

nearest neighbor (kNN) method is a popular classification method in data mining and statistics because of its simple implementation and significant classification performance. However, it is impractical for traditional kNN … nearest neighbor (kNN) method is a popular classification method in data mining and statistics because of its simple implementation and significant classification performance. However, it is impractical for traditional kNN methods to assign a fixed value (even though set by experts) to all test samples. Previous solutions assign different values to different test samples by the cross validation method but are usually time-consuming. This paper proposes a kTree method to learn different optimal values for different test/new samples, by involving a training stage in the kNN classification. Specifically, in the training stage, kTree method first learns optimal values for all training samples by a new sparse reconstruction model, and then constructs a decision tree (namely, kTree) using training samples and the learned optimal values. In the test stage, the kTree fast outputs the optimal value for each test sample, and then, the kNN classification can be conducted using the learned optimal value and all training samples. As a result, the proposed kTree method has a similar running cost but higher classification accuracy, compared with traditional kNN methods, which assign a fixed value to all test samples. Moreover, the proposed kTree method needs less running cost but achieves similar classification accuracy, compared with the newly kNN methods, which assign different values to different test samples. This paper further proposes an improvement version of kTree method (namely, k*Tree method) to speed its test stage by extra storing the information of the training samples in the leaf nodes of kTree, such as the training samples located in the leaf nodes, their kNNs, and the nearest neighbor of these kNNs. We call the resulting decision tree as k*Tree, which enables to conduct kNN classification using a subset of the training samples in the leaf nodes rather than all training samples used in the newly kNN methods. This actually reduces running cost of test stage. Finally, the experimental results on 20 real data sets showed that our proposed methods (i.e., kTree and k*Tree) are much more efficient than the compared methods in terms of classification tasks.

A mathematical theory of evidence

1977-06-01

Gian‐Carlo Rota

On Calibration of Modern Neural Networks

2017-01-01

Chuan Guo , Geoff Pleiss , Yu Sun +1 more

Confidence calibration -- the problem of predicting probability estimates representative of the true correctness likelihood -- is important for classification models in many applications. We discover that modern neural networks, … Confidence calibration -- the problem of predicting probability estimates representative of the true correctness likelihood -- is important for classification models in many applications. We discover that modern neural networks, unlike those from a decade ago, are poorly calibrated. Through extensive experiments, we observe that depth, width, weight decay, and Batch Normalization are important factors influencing calibration. We evaluate the performance of various post-processing calibration methods on state-of-the-art architectures with image and document classification datasets. Our analysis and experiments not only offer insights into neural network learning, but also provide a simple and straightforward recipe for practical settings: on most datasets, temperature scaling -- a single-parameter variant of Platt Scaling -- is surprisingly effective at calibrating predictions.

Deep learning and its applications to machine health monitoring

2018-06-14

Rui Zhao , Ruqiang Yan , Zhenghua Chen +3 more

A Survey on Deep Learning

2018-09-18

Samira Pouyanfar , Saad Sadiq , Yilin Yan +6 more

The field of machine learning is witnessing its golden era as deep learning slowly becomes the leader in this domain. Deep learning uses multiple layers to represent the abstractions of … The field of machine learning is witnessing its golden era as deep learning slowly becomes the leader in this domain. Deep learning uses multiple layers to represent the abstractions of data to build computational models. Some key enabler deep learning algorithms such as generative adversarial networks, convolutional neural networks, and model transfers have completely changed our perception of information processing. However, there exists an aperture of understanding behind this tremendously fast-paced domain, because it was never previously represented from a multiscope perspective. The lack of core understanding renders these powerful methods as black-box machines that inhibit development at a fundamental level. Moreover, deep learning has repeatedly been perceived as a silver bullet to all stumbling blocks in machine learning, which is far from the truth. This article presents a comprehensive review of historical and recent state-of-the-art approaches in visual, audio, and text processing; social network analysis; and natural language processing, followed by the in-depth analysis on pivoting and groundbreaking advances in deep learning applications. It was also undertaken to review the issues faced in deep learning such as unsupervised learning, black-box models, and online learning and to illustrate how these challenges can be transformed into prolific future research avenues.

Gaussian Error Linear Units (GELUs)

2016-01-01

Dan Hendrycks , Kevin Gimpel

We propose the Gaussian Error Linear Unit (GELU), a high-performing neural network activation function. The GELU activation function is $x\Phi(x)$, where $\Phi(x)$ the standard Gaussian cumulative distribution function. The GELU … We propose the Gaussian Error Linear Unit (GELU), a high-performing neural network activation function. The GELU activation function is $x\Phi(x)$, where $\Phi(x)$ the standard Gaussian cumulative distribution function. The GELU nonlinearity weights inputs by their value, rather than gates inputs by their sign as in ReLUs ($x\mathbf{1}_{x>0}$). We perform an empirical evaluation of the GELU nonlinearity against the ReLU and ELU activations and find performance improvements across all considered computer vision, natural language processing, and speech tasks.

View PDF Chat

Deep Learning for Anomaly Detection: A Survey

2019-01-01

Raghavendra Chalapathy , Sanjay Chawla

Anomaly detection is an important problem that has been well-studied within diverse research areas and application domains. The aim of this survey is two-fold, firstly we present a structured and … Anomaly detection is an important problem that has been well-studied within diverse research areas and application domains. The aim of this survey is two-fold, firstly we present a structured and comprehensive overview of research methods in deep learning-based anomaly detection. Furthermore, we review the adoption of these methods for anomaly across various application domains and assess their effectiveness. We have grouped state-of-the-art research techniques into different categories based on the underlying assumptions and approach adopted. Within each category we outline the basic anomaly detection technique, along with its variants and present key assumptions, to differentiate between normal and anomalous behavior. For each category, we present we also present the advantages and limitations and discuss the computational complexity of the techniques in real application domains. Finally, we outline open issues in research and challenges faced while adopting these techniques.

A State-of-the-Art Survey on Deep Learning Theory and Architectures

2019-03-05

Md Zahangir Alom , Tarek M. Taha , Chris Yakopcic +7 more

In recent years, deep learning has garnered tremendous success in a variety of application domains. This new field of machine learning has been growing rapidly and has been applied to … In recent years, deep learning has garnered tremendous success in a variety of application domains. This new field of machine learning has been growing rapidly and has been applied to most traditional application domains, as well as some new areas that present more opportunities. Different methods have been proposed based on different categories of learning, including supervised, semi-supervised, and un-supervised learning. Experimental results show state-of-the-art performance using deep learning when compared to traditional machine learning approaches in the fields of image processing, computer vision, speech recognition, machine translation, art, medical imaging, medical information processing, robotics and control, bioinformatics, natural language processing, cybersecurity, and many others. This survey presents a brief survey on the advances that have occurred in the area of Deep Learning (DL), starting with the Deep Neural Network (DNN). The survey goes on to cover Convolutional Neural Network (CNN), Recurrent Neural Network (RNN), including Long Short-Term Memory (LSTM) and Gated Recurrent Units (GRU), Auto-Encoder (AE), Deep Belief Network (DBN), Generative Adversarial Network (GAN), and Deep Reinforcement Learning (DRL). Additionally, we have discussed recent developments, such as advanced variant DL techniques based on these DL approaches. This work considers most of the papers published after 2012 from when the history of deep learning began. Furthermore, DL approaches that have been explored and evaluated in different application domains are also included in this survey. We also included recently developed frameworks, SDKs, and benchmark datasets that are used for implementing and evaluating deep learning approaches. There are some surveys that have been published on DL using neural networks and a survey on Reinforcement Learning (RL). However, those papers have not discussed individual advanced techniques for training large-scale deep learning models and the recently developed method of generative models.

View PDF Chat

GANomaly: Semi-supervised Anomaly Detection via Adversarial Training

2019-01-01

Samet Akçay , Amir Atapour–Abarghouei , Toby P. Breckon

View PDF Chat

Future Frame Prediction for Anomaly Detection - A New Baseline

2018-06-01

Wen Liu , Weixin Luo , Dongze Lian +1 more

Anomaly detection in videos refers to the identification of events that do not conform to expected behavior. However, almost all existing methods tackle the problem by minimizing the reconstruction errors … Anomaly detection in videos refers to the identification of events that do not conform to expected behavior. However, almost all existing methods tackle the problem by minimizing the reconstruction errors of training data, which cannot guarantee a larger reconstruction error for an abnormal event. In this paper, we propose to tackle the anomaly detection problem within a video prediction framework. To the best of our knowledge, this is the first work that leverages the difference between a predicted future frame and its ground truth to detect an abnormal event. To predict a future frame with higher quality for normal events, other than the commonly used appearance (spatial) constraints on intensity and gradient, we also introduce a motion (temporal) constraint in video prediction by enforcing the optical flow between predicted frames and ground truth frames to be consistent, and this is the first work that introduces a temporal constraint into the video prediction task. Such spatial and motion constraints facilitate the future frame prediction for normal events, and consequently facilitate to identify those abnormal events that do not conform the expectation. Extensive experiments on both a toy dataset and some publicly available datasets validate the effectiveness of our method in terms of robustness to the uncertainty in normal events and the sensitivity to abnormal events. All codes are released in https://github.com/StevenLiuWen/ano_pred_cvpr2018.

View PDF Chat

Real-World Anomaly Detection in Surveillance Videos

2018-06-01

Waqas Sultani , Chen Chen , Mubarak Shah

Surveillance videos are able to capture a variety of realistic anomalies. In this paper, we propose to learn anomalies by exploiting both normal and anomalous videos. To avoid annotating the … Surveillance videos are able to capture a variety of realistic anomalies. In this paper, we propose to learn anomalies by exploiting both normal and anomalous videos. To avoid annotating the anomalous segments or clips in training videos, which is very time consuming, we propose to learn anomaly through the deep multiple instance ranking framework by leveraging weakly labeled training videos, i.e. the training labels (anomalous or normal) are at video-level instead of clip-level. In our approach, we consider normal and anomalous videos as bags and video segments as instances in multiple instance learning (MIL), and automatically learn a deep anomaly ranking model that predicts high anomaly scores for anomalous video segments. Furthermore, we introduce sparsity and temporal smoothness constraints in the ranking loss function to better localize anomaly during training. We also introduce a new large-scale first of its kind dataset of 128 hours of videos. It consists of 1900 long and untrimmed real-world surveillance videos, with 13 realistic anomalies such as fighting, road accident, burglary, robbery, etc. as well as normal activities. This dataset can be used for two tasks. First, general anomaly detection considering all anomalies in one group and all normal activities in another group. Second, for recognizing each of 13 anomalous activities. Our experimental results show that our MIL method for anomaly detection achieves significant improvement on anomaly detection performance as compared to the state-of-the-art approaches. We provide the results of several recent deep learning baselines on anomalous activity recognition. The low recognition performance of these baselines reveals that our dataset is very challenging and opens more opportunities for future work. The dataset is available at: http://crcv.ucf.edu/projects/real-world.

View PDF Chat

On calibration of modern neural networks

2017-07-17

Chuan Guo , Geoff Pleiss , Yu Sun +1 more

Memorizing Normality to Detect Anomaly: Memory-Augmented Deep Autoencoder for Unsupervised Anomaly Detection

2019-10-01

Dong Gong , Lingqiao Liu , Vuong Le +4 more

Deep autoencoder has been extensively used for anomaly detection. Training on the normal data, the autoencoder is expected to produce higher reconstruction error for the abnormal inputs than the normal … Deep autoencoder has been extensively used for anomaly detection. Training on the normal data, the autoencoder is expected to produce higher reconstruction error for the abnormal inputs than the normal ones, which is adopted as a criterion for identifying anomalies. However, this assumption does not always hold in practice. It has been observed that sometimes the autoencoder "generalizes" so well that it can also reconstruct anomalies well, leading to the miss detection of anomalies. To mitigate this drawback for autoencoder based anomaly detector, we propose to augment the autoencoder with a memory module and develop an improved autoencoder called memory-augmented autoencoder, i.e. MemAE. Given an input, MemAE firstly obtains the encoding from the encoder and then uses it as a query to retrieve the most relevant memory items for reconstruction. At the training stage, the memory contents are updated and are encouraged to represent the prototypical elements of the normal data. At the test stage, the learned memory will be fixed, and the reconstruction is obtained from a few selected memory records of the normal data. The reconstruction will thus tend to be close to a normal sample. Thus the reconstructed errors on anomalies will be strengthened for anomaly detection. MemAE is free of assumptions on the data type and thus general to be applied to different tasks. Experiments on various datasets prove the excellent generalization and high effectiveness of the proposed MemAE.

View PDF Chat

Machine Learning: Algorithms, Real-World Applications and Research Directions

2021-03-22

Iqbal H. Sarker

View PDF Chat

Deep Learning for Anomaly Detection

2021-03-05

Guansong Pang , Chunhua Shen , Longbing Cao +1 more

Anomaly detection, a.k.a. outlier detection or novelty detection, has been a lasting yet active research area in various research communities for several decades. There are still some unique problem complexities … Anomaly detection, a.k.a. outlier detection or novelty detection, has been a lasting yet active research area in various research communities for several decades. There are still some unique problem complexities and challenges that require advanced approaches. In recent years, deep learning enabled anomaly detection, i.e., deep anomaly detection , has emerged as a critical direction. This article surveys the research of deep anomaly detection with a comprehensive taxonomy, covering advancements in 3 high-level categories and 11 fine-grained categories of the methods. We review their key intuitions, objective functions, underlying assumptions, advantages, and disadvantages and discuss how they address the aforementioned challenges. We further discuss a set of possible future opportunities and new perspectives on addressing the challenges.

View PDF Chat

Introduction to Machine Learning

2019-01-01

F. Richard Yu , Ying He

Ensemble deep learning: A review

2022-07-30

M. A. Ganaie , Minghui Hu , A. K. Malik +2 more

View PDF Chat

Deep Learning: A Comprehensive Overview on Techniques, Taxonomy, Applications and Research Directions

2021-08-18

Iqbal H. Sarker

View PDF Chat

Learning Deep Architectures for AI

2009-01-01

Yoshua Bengio

LOF

2000-05-16

Markus Breunig , Hans‐Peter Kriegel , Raymond T. Ng +1 more

View PDF Chat

Is AI powering Big Brother? Surveillance research is on the rise

2025-06-25

Nick Petrić Howe | Nature

Seconds Count – A QIP Empowering OMFS Staff in Emergency Response

2025-06-24

Ameena Jassim | International Journal of Oral and Maxillofacial Surgery

Blockchain Abnormal Transaction Identification Method Based on Multi-Dimensional Feature Fusion and Edge Computing

2025-06-24

Wentao Jiang , Wenbao Jiang , Jin Ding | International Journal of High Speed Electronics and Systems

With the widespread adoption of blockchain technology, various types of cybercrimes have emerged on the Ethereum network, including phishing scams, money laundering, and Ponzi schemes. These illegal activities not only … With the widespread adoption of blockchain technology, various types of cybercrimes have emerged on the Ethereum network, including phishing scams, money laundering, and Ponzi schemes. These illegal activities not only threaten the security of the Ethereum ecosystem but also result in significant economic losses. Therefore, establishing efficient methods for anomalous transaction detection is crucial for blockchain security. However, existing approaches are limited by the use of single-type features and locally optimal feature selection, which restricts the comprehensiveness and performance of the models, making it difficult to effectively handle the diversity and heterogeneity of the data. To address this problem, this paper proposes a blockchain abnormal transaction identification method that combines multi-dimensional feature fusion, edge computing and future networks, which integrates three different types of features: statistical features, network structure features, and motif features. This method provides a comprehensive and multi-angle characterization of node features, fully exploiting the latent information in the data and enhancing the model’s performance and generalization ability. Experimental results demonstrate that the proposed method achieves excellent performance on the dataset, significantly improving the accuracy of anomalous transaction detection and showing its potential for enhancing blockchain transaction security, thereby providing new technical support for blockchain security. In addition, combining with edge computing and future network technologies, this method can also better cope with the complex environment in large-scale blockchain networks and provide new technical support for blockchain security.

Leveraging Edge Intelligence and LLMs to Advance 6G-Enabled Internet of Automated Defense Vehicles

2025-06-24

Murat Arda Önsü , Poonam Lohan , Burak Kantarcı | IEEE Internet of Things Magazine

Developing a Deep Learning-Based Framework for Real-Time Anomaly Detection and Alerting Mechanism Using Embedded Systems

2025-06-24

Vijayalakshmi G. V. Mahesh , Richesh Gaurav , M A Nisarga +2 more | Advances in computational intelligence and robotics book series

The increase in surveillance and monitoring devices at various locations over the last decades has provided new ways to use the collected data in a variety of useful applications. This … The increase in surveillance and monitoring devices at various locations over the last decades has provided new ways to use the collected data in a variety of useful applications. This includes event management forecasting, live data monitoring, traffic analysis, crowd behavior, home security, public safety and targeted analysis, especially in the areas of anomaly and intrusion detection. Artificial intelligence methods with machine learning and deep learning architectures have emerged as a valuable anomaly detection tool that is highly effective in modern society. This chapter aims to present an approach based on deep learning methods, specially designed to detect anomaly activities through video surveillance cameras for smart city applications to aid law enforcement.

Embedding Models for Multivariate Time Series Anomaly Detection in Industry 5.0

2025-06-24

Lorenzo Colombi , Michela Vespa , Nicolas Belletti +7 more | Data Science and Engineering

Abstract Industrial processes often involve the generation—and the analysis—of multivariate time series data, which poses several challenges from the anomaly detection perspective. In addition to the need to detect previously … Abstract Industrial processes often involve the generation—and the analysis—of multivariate time series data, which poses several challenges from the anomaly detection perspective. In addition to the need to detect previously unseen anomalies, the high dimensionality of industrial datasets introduces the complexity of simultaneously analyzing multiple features and their interactions. Finally, industrial datasets are typically highly imbalanced, with minimal information on anomalous processes. To address these issues, we propose a novel anomaly detection framework that introduces two embedding models, based on Time2Vec and Discrete Wavelet Transforms, leveraging their capabilities to represent multivariate time series as vectors while capturing and preserving temporal dependencies and combining them with several classifiers to enhance the overall performance of anomaly detection. We tested our solution using a publicly available benchmark dataset and a real industrial use case, particularly data collected from a Bonfiglioli gear manufacturing plant. The results demonstrate that, unlike traditional reconstruction-based autoencoders, which often struggle with sporadic noise, our embedding-based solutions maintain high performance across various noise conditions.

Spatio-temporal graph convolutional network with domain generalization: A novel rotating machinery RUL prediction method in small samples

2025-06-24

Rui Bai , Yongbo Li , Jiancheng Yin +1 more | Advanced Engineering Informatics

A gated convolutional neural network-based tool remaining useful life prediction method

2025-06-24

Shilong Zhou , Yaonan Cheng , Jing Xue +3 more | Engineering Applications of Artificial Intelligence

Disasters classification in a compound event perspective: Insights from existing databases

2025-06-23

Carlo De Michele , Fabiola Banfi , Maria Pia Russomando | Environmental Research Letters

Abstract Natural disasters often result from compound event dynamics, in which multiple interacting drivers converge across spatial and temporal scales, significantly amplifying their impacts. The concept of compound events has … Abstract Natural disasters often result from compound event dynamics, in which multiple interacting drivers converge across spatial and temporal scales, significantly amplifying their impacts. The concept of compound events has gained increasing attention in recent literature, offering opportunities to enhance disaster understanding, while also presenting challenges and open issues for modern risk assessment frameworks. &#xD;This study investigates the capability of existing disasters/extreme events databases (Emergency Events Database EM-DAT, Severe Weather Data Inventory SWDI, and Canadian Disaster Database CDD) to capture compound event dynamics, and assess the accuracy of reported impacts.&#xD;We found that SWDI, a national dataset for the USA, reports a high number of compound events versus single events, always higher than 50\%, except for wildfires, and its structure allows for accurately identify spatially compounding events. &#xD; This percentage in EM-DAT, a global dataset, is always lower than 50\%, except for storms. A good match in events occurrences can be observed between the three databases, however the agreement in terms of deaths and injures varies depending on the databases compared.&#xD; Finally, the work highlights the limitations of existing databases in representing the multidimensional nature of risks, and the cascading impacts that emerge from compound hazards. Reclassifying disasters from a compound event perspective not only enriches our knowledge of hazard dynamics, but also provides actionable pathways for improving risk assessment, informing adaptive policies, and enhancing resilience to the growing complexity of environmental challenges.

Detecting anomalies in space-object light curves using temporal-attention LSTM

2025-06-23

W. D. Li , Yu Zhang , Jianfeng Wang +6 more | Space habitation.

TNeXt: A convolutional neural network for remote harbor classification

2025-06-23

Mert Gürtürk , Şengül Doğan , Türker Tuncer | Ain Shams Engineering Journal

Multi-path feature fusion and constrained boundary optimization for open intent detection

2025-06-22

Jinjie Huang , Wenhe Li | The Journal of Supercomputing

Advancements in Mental Health Diagnosis: Leveraging Delta Feature Extraction Framework and PWSA Ensemble for Motion Data Analysis

2025-06-22

A. Lakshmi Muddana , S. Annapoorani | International Journal of Data Mining Modelling and Management

Enhancing real-time video surveillance: a circular queue-guided stacked parallel convolution network for efficient suspicious activity detection

2025-06-21

Sanathana Konugolu Venkata Sekar , Srini Narayanan , P. Sirenjeevi +1 more | Earth Science Informatics

Source-Free Model Transferability Assessment for Smart Surveillance via Randomly Initialized Networks

2025-06-20

Wei-Cheng Wang , Sam Leroux , Pieter Simoens | Sensors

Smart surveillance cameras are increasingly employed for automated tasks such as event and anomaly detection within smart city infrastructures. However, the heterogeneity of deployment environments, ranging from densely populated urban … Smart surveillance cameras are increasingly employed for automated tasks such as event and anomaly detection within smart city infrastructures. However, the heterogeneity of deployment environments, ranging from densely populated urban intersections to quiet residential neighborhoods, renders the use of a single, universal model suboptimal. To address this, we propose the construction of a model zoo comprising models trained for diverse environmental contexts. We introduce an automated transferability assessment framework that identifies the most suitable model for a new deployment site. This task is particularly challenging in smart surveillance settings, where both source data and labeled target data are typically unavailable. Existing approaches often depend on pretrained embeddings or assumptions about model uncertainty, which may not hold reliably in real-world scenarios. In contrast, our method leverages embeddings generated by randomly initialized neural networks (RINNs) to construct task-agnostic reference embeddings without relying on pretraining. By comparing feature representations of the target data extracted using both pretrained models and RINNs, this method eliminates the need for labeled data. Structural similarity between embeddings is quantified using minibatch-Centered Kernel Alignment (CKA), enabling efficient and scalable model ranking. We evaluate our method on realistic surveillance datasets across multiple downstream tasks, including object tagging, anomaly detection, and event classification. Our embedding-level score achieves high correlations with ground-truth model rankings (relative to fine-tuned baselines), attaining Kendall’s τ values of 0.95, 0.94, and 0.89 on these tasks, respectively. These results demonstrate that our framework consistently selects the most transferable model, even when the specific downstream task or objective is unknown. This confirms the practicality of our approach as a robust, low-cost precursor to model adaptation or retraining.

Efficient slice anomaly detection network for 3D brain MRI Volume

2025-06-20

Zeduo Zhang , Yalda Mohsenzadeh | PLOS Digital Health

Current anomaly detection methods excel with benchmark industrial data but struggle with natural images and medical data due to varying definitions of ‘normal’ and ‘abnormal.’ This makes accurate identification of … Current anomaly detection methods excel with benchmark industrial data but struggle with natural images and medical data due to varying definitions of ‘normal’ and ‘abnormal.’ This makes accurate identification of deviations in these fields particularly challenging. Especially for 3D brain MRI data, all the state-of-the-art models are reconstruction-based with 3D convolutional neural networks which are memory-intensive, time-consuming and producing noisy outputs that require further post-processing. We propose a framework called Simple Slice-based Network (SimpleSliceNet), which utilizes a model pre-trained on ImageNet and fine-tuned on a separate MRI dataset as a 2D slice feature extractor to reduce computational cost. We aggregate the extracted features to perform anomaly detection tasks on 3D brain MRI volumes. Our model integrates a conditional normalizing flow to calculate log likelihood of features and employs the contrastive loss to enhance anomaly detection accuracy. The results indicate improved performance, showcasing our model’s remarkable adaptability and effectiveness when addressing the challenges exists in brain MRI data. In addition, for the large-scale 3D brain volumes, our model SimpleSliceNet outperforms the state-of-the-art 2D and 3D models in terms of accuracy, memory usage and time consumption. Code is available at: https://github.com/Jarvisarmy/SimpleSliceNet .

View PDF Chat

Toward a new dataset: Mexican Traffic Signs ReWaIn-MTS for detection using deep learning

2025-06-20

Daniela Bolaños-Flores , Tania A. Ramírez-delReal , Hamurabi Gamboa-Rosales +1 more | IEEE Latin America Transactions

The Application of Generative Adversarial Network on Data Augmentation for Target Selection

2025-06-20

Yang Liu , Zihan Wang , Qing Xu +1 more | International Journal of Automotive Technology

Impact of Machine Learning on Intrusion Detection Systems for the Protection of Critical Infrastructure

2025-06-20

Avinash Kumar , Jairo Gutiérrez | Information

In the realm of critical infrastructure protection, robust intrusion detection systems (IDSs) are essential for securing essential services. This paper investigates the efficacy of various machine learning algorithms for anomaly … In the realm of critical infrastructure protection, robust intrusion detection systems (IDSs) are essential for securing essential services. This paper investigates the efficacy of various machine learning algorithms for anomaly detection within critical infrastructure, using the Secure Water Treatment (SWaT) dataset, a comprehensive collection of time-series data from a water treatment testbed, to experiment upon and analyze the findings. The study evaluates supervised learning algorithms alongside unsupervised learning algorithms. The analysis reveals that supervised learning algorithms exhibit exceptional performance with high accuracy and reliability, making them well-suited for handling the diverse and complex nature of anomalies in critical infrastructure. They demonstrate significant capabilities in capturing spatial and temporal variables. Among the unsupervised approaches, valuable insights into anomaly detection are provided without the necessity for labeled data, although they face challenges with higher rates of false positives and negatives. By outlining the benefits and drawbacks of these machine learning algorithms in relation to critical infrastructure, this research advances the field of cybersecurity. It emphasizes the importance of integrating supervised and unsupervised techniques to enhance the resilience of IDSs, ensuring the timely detection and mitigation of potential threats. The findings offer practical guidance for industry professionals on selecting and deploying effective machine learning algorithms in critical infrastructure environments.

Unsupervised Approaches to Finding Outliers in Caption-Represented Images

2025-06-20

Jakub Zaprzałka , Magdalena Topczewska | Entropy

Both supervised and unsupervised machine learning algorithms are often based on regression to the mean. However, the mean can easily be biased by unevenly distributed data, i.e., outlier records. Batch … Both supervised and unsupervised machine learning algorithms are often based on regression to the mean. However, the mean can easily be biased by unevenly distributed data, i.e., outlier records. Batch normalization methods address this problem to some extent, but they also influence the data. In text-based data, the problem is even more pronounced, as distance distinctions between outlier records diminish with increasing dimensionality. The ultimate solution to achieving unbiased data is identifying the outliers. To address this issue, multidimensional scaling (MDS) and agglomerative-based techniques are proposed for detecting outlier records in text-based data. For both methods, two of the most common distance metrics are applied: Euclidean distance and cosine distance. Furthermore, in the MDS approach, both metric and non-metric versions of the algorithm are used, whereas in the agglomerative approach, the last-p and level cutoff techniques are applied. The methods are also compared with a raw-data-based method, which selects the most distant element from the others based on a given distance metric. Experiments were conducted on overlapping subsets of a dataset containing roughly 2000 records of descriptive image captions. The algorithms were also compared in terms of efficiency with a proposed algorithm and evaluated through human judgment based on the described images. Unsurprisingly, the cosine distance turned out to be the most effective distance metric. The metric-MDS-based algorithm appeared to outperform the others based on human evaluation. The presented algorithms successfully identified outlier records.

Enhanced Video Anomaly Detection Through Dual Triplet Contrastive Loss for Hard Sample Discrimination

2025-06-20

Chunxiang Niu , Siyu Meng , Rong Wang | Entropy

Learning discriminative features between abnormal and normal instances is crucial for video anomaly detection within the multiple instance learning framework. Existing methods primarily focus on instances with the highest anomaly … Learning discriminative features between abnormal and normal instances is crucial for video anomaly detection within the multiple instance learning framework. Existing methods primarily focus on instances with the highest anomaly scores, neglecting the identification and differentiation of hard samples, leading to misjudgments and high false alarm rates. To address these challenges, we propose a dual triplet contrastive loss strategy. This approach employs dual memory units to extract four key feature categories: hard samples, negative samples, positive samples, and anchor samples. Contrastive loss is utilized to constrain the distance between hard samples and other samples, enabling accurate identification of hard samples and enhancing the discriminative ability of hard samples and abnormal features. Additionally, a multi-scale feature perception module is designed to capture feature information at different levels, while an adaptive global–local feature fusion module constructs complementary feature enhancement through feature fusion. Experimental results demonstrate the effectiveness of our method, achieving AUC scores of 87.16% on the UCF-Crime dataset and AP scores of 83.47% on the XD-Violence dataset.

Resilient Anomaly Detection in Fiber-Optic Networks: A Machine Learning Framework for Multi-Threat Identification Using State-of-Polarization Monitoring

2025-06-20

Gulmina Malik , Imran Chowdhury Dipto , Muhammad Umar Masood +7 more | AI

We present a thorough machine-learning framework based on real-time state-of-polarization (SOP) monitoring for robust anomaly identification in optical fiber networks. We exploit SOP data under three different threat scenarios: (i) … We present a thorough machine-learning framework based on real-time state-of-polarization (SOP) monitoring for robust anomaly identification in optical fiber networks. We exploit SOP data under three different threat scenarios: (i) malicious or critical vibration events, (ii) overlapping mechanical disturbances, and (iii) malicious fiber tapping (eavesdropping). We used various supervised machine learning techniques like k-Nearest Neighbor (k-NN), random forest, extreme gradient boosting (XGBoost), and decision trees to classify different vibration events. We also assessed the framework’s resilience to background interference by superimposing sinusoidal noise at different frequencies and examining its effects on the polarization signatures. This analysis provides insight into how subsurface installations, subject to ambient vibrations, affect detection fidelity. This highlights the sensitivity to which external interference affects polarization fingerprints. Crucially, it demonstrates the system’s capacity to discern and alert on malicious vibration events even in the presence of environmental noise. However, we focus on the necessity of noise-mitigation techniques in real-world implementations while providing a potent, real-time mechanism for multi-threat recognition in the fiber networks.

Quantile Residual Lifetime Regression for Multivariate Failure Time Data

2025-06-19

Tonghui Yu , Liming Xiang , Jong‐Hyeon Jeong | Statistica Sinica

A Hybrid Spiking Model for Anomaly Detection in Multivariate Time Series

2025-06-19

Wei Zhang , Ping He , Shengrui Wang +2 more | Expert Systems

ABSTRACT Deep neural networks have exhibited preeminent performance in anomaly detection, but they struggle to effectively capture changes over time in multivariate time‐series data and suffer from resource consumption issues. … ABSTRACT Deep neural networks have exhibited preeminent performance in anomaly detection, but they struggle to effectively capture changes over time in multivariate time‐series data and suffer from resource consumption issues. Spiking neural networks address these limitations by capturing the change in time‐varying signals and decreasing resource consumption, but they sacrifice performance. This paper develops a novel spiking‐based hybrid model incorporated a temporal prediction network and a reconstruction network. It integrates a unique first‐spike frequency encoding scheme and a firing rate based anomaly score method. The encoding scheme enhances the event representation ability, while the anomaly score enables efficient anomaly identification. Our proposed model not only maintains low resource consumption but also improves the ability of anomaly detection. Experiments on publicly real‐world datasets confirmed that the proposed model acquires state‐of‐the‐art performance superior to existing approaches. Remarkably, it costs 5.04× lower energy consumption compared with the artificial neural network version.

Integrated Deep Learning Framework for Weapon—Behavior Analysis

2025-06-19

P. Shanthi , V. Manjula | CRC Press eBooks

Dynamic dual mining framework for long-tailed out-of-distribution detection

2025-06-19

Bin Sheng , Dengye Pan , Xiaoqiang Li | Applied Intelligence

A novel enhanced Bayesian classifier with multiple smoothing parameters

2025-06-19

Yanli Zhao , Guang Yang , Hong Wei | Knowledge and Information Systems

Focal weighting strategy with multi-label multi-scale granularity-aware for out-of-distribution detection

2025-06-19

Xin Peng , Yifei Cheng , Yusheng Cheng | Applied Intelligence

MTAD-Kanformer: multivariate time-series anomaly detection via kan and transformer

2025-06-19

Xin Xie , Wenbin Zheng , Shenping Xiong +1 more | Applied Intelligence

Res2coder: A two-stage residual autoencoder for unsupervised time series anomaly detection

2025-06-19

Hao Wang , Yingjian Liu , Haoyu Yin +4 more | Applied Intelligence

Optical flow and deep learning-based anomaly detection for hajj pilgrimage crowd monitoring

2025-06-19

Md Roman Bhuiyan , Junaidi Abdullah , Noramiza Hashim +7 more | Signal Image and Video Processing

Time series data cleaning based on sliding window and linear fitting method for vibrating wire strain gauges

2025-06-19

Hui Chen , Jianxun Chen , Yanbin Luo +3 more | Measurement Science and Technology

Abstract Vibrating wire strain gauges (VWSG) are extensively utilized in civil engineering, yet conventional data cleaning methods inadequately address trend-associated anomalies such as bias and gain. To address these challenges, … Abstract Vibrating wire strain gauges (VWSG) are extensively utilized in civil engineering, yet conventional data cleaning methods inadequately address trend-associated anomalies such as bias and gain. To address these challenges, this study proposes a novel data cleaning method based on particle swarm optimization enhanced sliding window and linear regression (PSO-SW-LR) feature extraction. First, the overall data is divided into segments using a sliding window, with linear regression applied to each segment. Next, slope and intercept coefficients are extracted to generate slope and intercept streams, with slope streams reflecting trend information for each segment.. Then, by eliminating abnormal slope values and corresponding intercept values, abnormal data with trend change characteristics can be effectively removed. Finally, regression calculations and the median method are applied to reconstruct the data. Validated on real VWSG monitoring data, SW-LR demonstrates superior performance over eight existing techniques, achieving an 88% RMSE reduction and 79% MAE improvement in composite anomaly scenarios. The proposed approach offers a robust and efficient solution for cleaning long-term time series data.&#xD;

A novel lightweight dual-stream recurrent transformer model for anomaly detection in driving dashcam videos

2025-06-18

Aya Abdelsalam Ismail , Doaa Aboshady | Neural Computing and Applications

Inferring change points in unlabelled time series data collected from the network diagnosis tool

2025-06-18

Cleiton M. de Almeida , Rosa M. M. Leão , Edmundo de Souza e Silva | Annals of Telecommunications

A new approach of anomaly detection in shopping center surveillance videos for theft prevention based on RLCNN model

2025-06-18

Muhammad Sajid , Ali Haider Khan , Kaleem Razzaq Malik +2 more | PeerJ Computer Science

The amount of video data produced daily by today's surveillance systems is enormous, making analysis difficult for computer vision specialists. It is challenging to continuously search these massive video streams … The amount of video data produced daily by today's surveillance systems is enormous, making analysis difficult for computer vision specialists. It is challenging to continuously search these massive video streams for unexpected accidents because they occur seldom and have little chance of being observed. Contrarily, deep learning-based anomaly detection decreases the need for human labor and has comparably trustworthy decision-making capabilities, hence promoting public safety. In this article, we introduce a system for efficient anomaly detection that can function in surveillance networks with a modest level of complexity. The proposed method starts by obtaining spatiotemporal features from a group of frames. The multi-layer extended short-term memory model can precisely identify continuing unusual activity in complicated video scenarios of a busy shopping mall once we transmit the in-depth features extracted. We conducted in-depth tests on numerous benchmark datasets for anomaly detection to confirm the proposed framework's functionality in challenging surveillance scenarios. Compared to state-of-the-art techniques, our datasets, UCF50, UCF101, UCFYouTube, and UCFCustomized, provided better training and increased accuracy. Our model was trained for more classes than usual, and when the proposed model, RLCNN, was tested for those classes, the results were encouraging. All of our datasets worked admirably. However, when we used the UCFCustomized and UCFYouTube datasets compared to other UCF datasets, we achieved greater accuracy of 96 and 97, respectively.

Multi-block local outlier factor anomaly detection of complex industrial systems

2025-06-18

Alexandre Gaffet , Pauline Ribot , Élodie Chanthery +1 more | Applied Intelligence

Abstract Anomaly detection is critical in industrial systems for ensuring equipment reliability and improving product quality, especially with the increasing complexity of electronic board production. However, traditional anomaly detection approaches … Abstract Anomaly detection is critical in industrial systems for ensuring equipment reliability and improving product quality, especially with the increasing complexity of electronic board production. However, traditional anomaly detection approaches often fail when dealing with high-dimensional data and limited system knowledge. To address this gap, this article aims to develop an effective unsupervised method for anomaly detection suitable for large-scale industrial contexts with minimal prior knowledge. The proposed Multi-block Local Outlier Factor (MLOF) method combines a variable decomposition technique based on Mutual Information and spectral clustering with a local anomaly detection algorithm using the Local Outlier Factor. The method was validated on the Tennessee Eastman Process and real-world industrial cases from Surface Mount Technology production lines, notably by comparing its results with 5 other methods in the literature. Results demonstrate a 15% improvement in anomaly detection performance compared to classical LOF on benchmark data and effective application in detecting anomalies in real production scenarios. The MLOF method represents a significant step forward in anomaly detection for complex systems, offering robust, scalable, and accurate solutions even in data-intensive and knowledge-scarce environments.

A Convolutional Neural Network-LSTM Based Physical Sensor Anomaly Detector for Interdependent SCADA Controllers

2025-06-18

Rishabh Das , Aaron Werth , Thomas H. Morris | AI Computer Science and Robotics Technology

This study outlines a novel intrusion detection system (IDS) to detect compromised sensor data anomalies in interdependent industrial processes. The IDS used a peer-to-peer communication framework which allowed multiple programmable … This study outlines a novel intrusion detection system (IDS) to detect compromised sensor data anomalies in interdependent industrial processes. The IDS used a peer-to-peer communication framework which allowed multiple programmable logic controllers (PLCs) to communicate and share sensor data. Utilizing the shared sensor data, state estimators used a long short-term memory (LSTM) machine learning algorithm to identify anomalous sensor readings connected to neighboring PLCs controlling an interdependent physical process. This study evaluated the performance of the IDS on three industrial operations aligning to a midstream oil terminal. The framework successfully detected several multi-sensor compromises during mid-stream oil terminal operations. A set of performance evaluations also showed no impact on the real-time operations of the PLC and outlined the prediction latencies of the framework.

Detection of Data Manipulation in Datasets Using Machine Learning

2025-06-18

RIMSHA ARFEEN | INTERANTIONAL JOURNAL OF SCIENTIFIC RESEARCH IN ENGINEERING AND MANAGEMENT

Data integrity is pivotal for achieving model performance and delicacy and for making believable opinions in moment's data wisdom and analytics environment. This design enforced and estimated a machine literacy- … Data integrity is pivotal for achieving model performance and delicacy and for making believable opinions in moment's data wisdom and analytics environment. This design enforced and estimated a machine literacy- driven frame that can descry data tampering through a generative analysis of a structured dataset in its original and acclimated countries. By transubstantiating both datasets to match their structure, and calculating a point-full difference vector, the system estimated and linked possible tampering in the acclimated dataset through statistical analysis on named ordered features, similar as Interquartile Range( IQR), entropy analysis, and Original Outlier Factor( LOF). These named features were also drafted into a Random Forest classifier that directly labelled each record as either tampered or not tampered. The end product showed significant pledge in landing anomalies, similar as outliers, null inserts, mismatching types and subtle shifts in value. The results indicated high perfection and recall on a range of manipulated datasets. Through successive trial, the system is promising for data confirmation and examination and the expansion of forensic auditing systems. This result is modular and scalable, which gives the added benefit of sound data integrity in critical means like finance, healthcare, and defense. Key Words: Data manipulation detection, data quality, anomaly detection, Interquartile Range (IQR), Local Outlier Factor (LOF), entropy analysis, skewness imputation, Shannon entropy, outlier detection, Random Forest, supervised classification, feature engineering, descriptive feature extraction, difference vectors, machine learning pipeline, validate data, data forensics, structured data comparison, ETL validation, automated dataset checking, and classification accuracy.

An explainable artificial intelligence driven fall system for sensor data analysis enhanced by butterworth filtering

2025-06-18

J. Shalini , L. Ashok Kumar | Engineering Applications of Artificial Intelligence