Computer Science › Artificial Intelligence

Neural Networks and Applications

Works Count: 205089

Wikipedia

Description

This cluster of papers covers a wide range of topics related to neural networks, including backpropagation learning, self-organizing maps, radial basis function networks, deep learning, and applications such as pattern classification and function approximation.

Keywords

Neural Networks; Self-Organizing Maps; Backpropagation Learning; Radial Basis Function Networks; Deep Learning; Artificial Neural Networks; Recurrent Neural Networks; Feedforward Neural Networks; Pattern Classification; Function Approximation

Learning representations by back-propagating errors

1986-10-01

David E. Rumelhart , Geoffrey E. Hinton , Ronald J. Williams

Understanding the difficulty of training deep feedforward neural networks

2010-03-31

Xavier Glorot , Yoshua Bengio

Whereas before 2006 it appears that deep multilayer neural networks were not successfully trained, since then several algorithms have been shown to successfully train them, with experimental results showing the … Whereas before 2006 it appears that deep multilayer neural networks were not successfully trained, since then several algorithms have been shown to successfully train them, with experimental results showing the superiority of deeper vs less deep architectures. All these experimental results were obtained with new initialization or training mechanisms. Our objective here is to understand better why standard gradient descent from random initialization is doing so poorly with deep neural networks, to better understand these recent relative successes and help design better algorithms in the future. We first observe the influence of the non-linear activations functions. We find that the logistic sigmoid activation is unsuited for deep networks with random initialization because of its mean value, which can drive especially the top hidden layer into saturation. Surprisingly, we find that saturated units can move out of saturation by themselves, albeit slowly, and explaining the plateaus sometimes seen when training neural networks. We find that a new non-linearity that saturates less can often be beneficial. Finally, we study how activations and gradients vary across layers and during training, with the idea that training may be more difficult when the singular values of the Jacobian associated with each layer are far from 1. Based on these considerations, we propose a new initialization scheme that brings substantially faster convergence. 1 Deep Neural Networks Deep learning methods aim at learning feature hierarchies with features from higher levels of the hierarchy formed by the composition of lower level features. They include Appearing in Proceedings of the 13 International Conference on Artificial Intelligence and Statistics (AISTATS) 2010, Chia Laguna Resort, Sardinia, Italy. Volume 9 of JMLR: WC Weston et al., 2008). Much attention has recently been devoted to them (see (Bengio, 2009) for a review), because of their theoretical appeal, inspiration from biology and human cognition, and because of empirical success in vision (Ranzato et al., 2007; Larochelle et al., 2007; Vincent et al., 2008) and natural language processing (NLP) (Collobert & Weston, 2008; Mnih & Hinton, 2009). Theoretical results reviewed and discussed by Bengio (2009), suggest that in order to learn the kind of complicated functions that can represent high-level abstractions (e.g. in vision, language, and other AI-level tasks), one may need deep architectures. Most of the recent experimental results with deep architecture are obtained with models that can be turned into deep supervised neural networks, but with initialization or training schemes different from the classical feedforward neural networks (Rumelhart et al., 1986). Why are these new algorithms working so much better than the standard random initialization and gradient-based optimization of a supervised training criterion? Part of the answer may be found in recent analyses of the effect of unsupervised pretraining (Erhan et al., 2009), showing that it acts as a regularizer that initializes the parameters in a “better” basin of attraction of the optimization procedure, corresponding to an apparent local minimum associated with better generalization. But earlier work (Bengio et al., 2007) had shown that even a purely supervised but greedy layer-wise procedure would give better results. So here instead of focusing on what unsupervised pre-training or semi-supervised criteria bring to deep architectures, we focus on analyzing what may be going wrong with good old (but deep) multilayer neural networks. Our analysis is driven by investigative experiments to monitor activations (watching for saturation of hidden units) and gradients, across layers and across training iterations. We also evaluate the effects on these of choices of activation function (with the idea that it might affect saturation) and initialization procedure (since unsupervised pretraining is a particular form of initialization and it has a drastic impact).

Neural networks for pattern recognition

1994-06-01

Chris Bishop

From the Publisher: This is the first comprehensive treatment of feed-forward neural networks from the perspective of statistical pattern recognition. After introducing the basic concepts, the book examines techniques for … From the Publisher: This is the first comprehensive treatment of feed-forward neural networks from the perspective of statistical pattern recognition. After introducing the basic concepts, the book examines techniques for modelling probability density functions and the properties and merits of the multi-layer perceptron and radial basis function network models. Also covered are various forms of error functions, principal algorithms for error function minimalization, learning and generalization in neural networks, and Bayesian techniques and their applications. Designed as a text, with over 100 exercises, this fully up-to-date work will benefit anyone involved in the fields of neural computation and pattern recognition.

View PDF Chat

Greedy function approximation: A gradient boosting machine.

2001-10-01

Jerome H. Friedman

Function estimation/approximation is viewed from the perspective of numerical optimization in function space, rather than parameter space. A connection is made between stagewise additive expansions and steepest-descent minimization. A general … Function estimation/approximation is viewed from the perspective of numerical optimization in function space, rather than parameter space. A connection is made between stagewise additive expansions and steepest-descent minimization. A general gradient descent “boosting” paradigm is developed for additive expansions based on any fitting criterion.Specific algorithms are presented for least-squares, least absolute deviation, and Huber-M loss functions for regression, and multiclass logistic likelihood for classification. Special enhancements are derived for the particular case where the individual additive components are regression trees, and tools for interpreting such “TreeBoost” models are presented. Gradient boosting of regression trees produces competitive, highly robust, interpretable procedures for both regression and classification, especially appropriate for mining less than clean data. Connections between this approach and the boosting methods of Freund and Shapire and Friedman, Hastie and Tibshirani are discussed.

View PDF Chat

Distilling the Knowledge in a Neural Network

2015-01-01

Geoffrey E. Hinton , Oriol Vinyals , Jay B. Dean

A very simple way to improve the performance of almost any machine learning algorithm is to train many different models on the same data and then to average their predictions. … A very simple way to improve the performance of almost any machine learning algorithm is to train many different models on the same data and then to average their predictions. Unfortunately, making predictions using a whole ensemble of models is cumbersome and may be too computationally expensive to allow deployment to a large number of users, especially if the individual models are large neural nets. Caruana and his collaborators have shown that it is possible to compress the knowledge in an ensemble into a single model which is much easier to deploy and we develop this approach further using a different compression technique. We achieve some surprising results on MNIST and we show that we can significantly improve the acoustic model of a heavily used commercial system by distilling the knowledge in an ensemble of models into a single model. We also introduce a new type of ensemble composed of one or more full models and many specialist models which learn to distinguish fine-grained classes that the full models confuse. Unlike a mixture of experts, these specialist models can be trained rapidly and in parallel.

Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift

2015-01-01

Sergey Ioffe , Christian Szegedy

Training Deep Neural Networks is complicated by the fact that the distribution of each layer's inputs changes during training, as the parameters of the previous layers change. This slows down … Training Deep Neural Networks is complicated by the fact that the distribution of each layer's inputs changes during training, as the parameters of the previous layers change. This slows down the training by requiring lower learning rates and careful parameter initialization, and makes it notoriously hard to train models with saturating nonlinearities. We refer to this phenomenon as internal covariate shift, and address the problem by normalizing layer inputs. Our method draws its strength from making normalization a part of the model architecture and performing the normalization for each training mini-batch. Batch Normalization allows us to use much higher learning rates and be less careful about initialization. It also acts as a regularizer, in some cases eliminating the need for Dropout. Applied to a state-of-the-art image classification model, Batch Normalization achieves the same accuracy with 14 times fewer training steps, and beats the original model by a significant margin. Using an ensemble of batch-normalized networks, we improve upon the best published result on ImageNet classification: reaching 4.9% top-5 validation error (and 4.8% test error), exceeding the accuracy of human raters.

Learning the parts of objects by non-negative matrix factorization

1999-10-01

Daniel D. Lee , H. Sebastian Seung

Model Selection and Multimodel Inference: A Practical Information-Theoretic Approach

2003-07-01

Fred S. Guthery , Kenneth P. Burnham , David Anderson

Introduction * Information and Likelihood Theory: A Basis for Model Selection and Inference * Basic Use of the Information-Theoretic Approach * Formal Inference From More Than One Model: Multi-Model Inference … Introduction * Information and Likelihood Theory: A Basis for Model Selection and Inference * Basic Use of the Information-Theoretic Approach * Formal Inference From More Than One Model: Multi-Model Inference (MMI) * Monte Carlo Insights and Extended Examples * Statistical Theory and Numerical Results * Summary

View PDF Chat

The perceptron: A probabilistic model for information storage and organization in the brain.

1958-01-01

Frank F. Rosenblatt

Long Short-Term Memory

1997-11-01

Sepp Hochreiter , Jürgen Schmidhuber

Learning to store information over extended time intervals by recurrent backpropagation takes a very long time, mostly because of insufficient, decaying error backflow. We briefly review Hochreiter's (1991) analysis of … Learning to store information over extended time intervals by recurrent backpropagation takes a very long time, mostly because of insufficient, decaying error backflow. We briefly review Hochreiter's (1991) analysis of this problem, then address it by introducing a novel, efficient, gradient based method called long short-term memory (LSTM). Truncating the gradient where this does not do harm, LSTM can learn to bridge minimal time lags in excess of 1000 discrete-time steps by enforcing constant error flow through constant error carousels within special units. Multiplicative gate units learn to open and close access to the constant error flow. LSTM is local in space and time; its computational complexity per time step and weight is O. 1. Our experiments with artificial data involve local, distributed, real-valued, and noisy pattern representations. In comparisons with real-time recurrent learning, back propagation through time, recurrent cascade correlation, Elman nets, and neural sequence chunking, LSTM leads to many more successful runs, and learns much faster. LSTM also solves complex, artificial long-time-lag tasks that have never been solved by previous recurrent network algorithms.

Deep learning in neural networks: An overview

2014-10-13

Jürgen Schmidhuber

View PDF Chat

A training algorithm for optimal margin classifiers

1992-07-01

Bernhard E. Boser , Isabelle Guyon , Vladimir Vapnik

A training algorithm that maximizes the margin between the training patterns and the decision boundary is presented. The technique is applicable to a wide variety of the classification functions, including … A training algorithm that maximizes the margin between the training patterns and the decision boundary is presented. The technique is applicable to a wide variety of the classification functions, including Perceptrons, polynomials, and Radial Basis Functions. The effective number of parameters is adjusted automatically to match the complexity of the problem. The solution is expressed as a linear combination of supporting patterns. These are the subset of training patterns that are closest to the decision boundary. Bounds on the generalization performance based on the leave-one-out method and the VC-dimension are given. Experimental results on optical character recognition problems demonstrate the good generalization obtained when compared with other learning algorithms.

View PDF Chat

Reducing the Dimensionality of Data with Neural Networks

2006-07-27

Geoffrey E. Hinton , Ruslan Salakhutdinov

High-dimensional data can be converted to low-dimensional codes by training a multilayer neural network with a small central layer to reconstruct high-dimensional input vectors. Gradient descent can be used for … High-dimensional data can be converted to low-dimensional codes by training a multilayer neural network with a small central layer to reconstruct high-dimensional input vectors. Gradient descent can be used for fine-tuning the weights in such “autoencoder” networks, but this works well only if the initial weights are close to a good solution. We describe an effective way of initializing the weights that allows deep autoencoder networks to learn low-dimensional codes that work much better than principal components analysis as a tool to reduce the dimensionality of data.

Approximation by superpositions of a sigmoidal function

1989-12-01

George Cybenko

View PDF Chat

A Wavelet Tour of Signal Processing

1999-01-01

Stéphane Mallat

Regularization and Variable Selection Via the Elastic Net

2005-03-09

Hui Zou , Trevor Hastie

We propose the elastic net, a new regularization and variable selection method. Real world data and a simulation study show that the elastic net often outperforms the lasso, while enjoying … We propose the elastic net, a new regularization and variable selection method. Real world data and a simulation study show that the elastic net often outperforms the lasso, while enjoying a similar sparsity of representation. In addition, the elastic net encourages a grouping effect, where strongly correlated predictors tend to be in or out of the model together. The elastic net is particularly useful when the number of predictors (p) is much bigger than the number of observations (n). By contrast, the lasso is not a very satisfactory variable selection method in the p≫n case. An algorithm called LARS-EN is proposed for computing elastic net regularization paths efficiently, much like algorithm LARS does for the lasso.

View PDF Chat

Density Estimation for Statistics and Data Analysis

2018-02-19

B.W. Silverman

Introduction. Survey of Existing Methods. The Kernel Method for Univariate Data. The Kernel Method for Multivariate Data. Three Important Methods. Density Estimation in Action. Introduction. Survey of Existing Methods. The Kernel Method for Univariate Data. The Kernel Method for Multivariate Data. Three Important Methods. Density Estimation in Action.

Introduction to Statistical Pattern Recognition

1990-01-01

Keinosuke Fukunaga

Multilayer feedforward networks are universal approximators

1989-01-01

Kurt Hornik , Maxwell B. Stinchcombe , Halbert White

SExtractor: Software for source extraction

1996-06-01

E. Bertin , S. Arnouts

We present the automated techniques we have developed for new software that optimally detects, deblends, measures and classifies sources from astronomical images: SExtractor (Source Extractor ). We show that a … We present the automated techniques we have developed for new software that optimally detects, deblends, measures and classifies sources from astronomical images: SExtractor (Source Extractor ). We show that a very reliable star/galaxy separation can be achieved on most images using a neural network trained with simulated images. Salient features of SExtractor include its ability to work on very large images, with minimal human intervention, and to deal with a wide variety of object shapes and magnitudes. It is therefore particularly suited to the analysis of large extragalactic surveys.

View PDF Chat

The Nature of Statistical Learning Theory

1995-01-01

Vladimir Vapnik

The Elements of Statistical Learning: Data Mining, Inference, and Prediction

2010-06-17

Zigang Lu

Investigating Causal Relations by Econometric Models and Cross-spectral Methods

1969-08-01

Clive W. J. Granger

There occurs on some occasions a difficulty in deciding the direction of causality between two related variables and also whether or not feedback is occurring. Testable definitions of causality and … There occurs on some occasions a difficulty in deciding the direction of causality between two related variables and also whether or not feedback is occurring. Testable definitions of causality and feedback are proposed and illustrated by use of simple two-variable models. The important problem of apparent instantaneous causality is discussed and it is suggested that the problem often arises due to slowness in recording information or because a sufficiently wide class of possible causal variables has not been used. It can be shown that the cross spectrum between two variables can be decomposed into two parts, each relating to a single causal arm of a feedback situation. Measures of causal lag and causal strength can then be constructed. A generalisation of this result with the partial cross spectrum is suggested.

Deep Learning

2016-11-18

Ian Goodfellow , Yoshua Bengio , Aaron Courville

Deep learning is a form of machine learning that enables computers to learn from experience and understand the world in terms of a hierarchy of concepts. Because the computer gathers … Deep learning is a form of machine learning that enables computers to learn from experience and understand the world in terms of a hierarchy of concepts. Because the computer gathers knowledge from experience, there is no need for a human computer operator to formally specify all the knowledge that the computer needs. The hierarchy of concepts allows the computer to learn complicated concepts by building them out of simpler ones; a graph of these hierarchies would be many layers deep. This book introduces a broad range of topics in deep learning. The text offers mathematical and conceptual background, covering relevant concepts in linear algebra, probability theory and information theory, numerical computation, and machine learning. It describes deep learning techniques used by practitioners in industry, including deep feedforward networks, regularization, optimization algorithms, convolutional networks, sequence modeling, and practical methodology; and it surveys such applications as natural language processing, speech recognition, computer vision, online recommendation systems, bioinformatics, and videogames. Finally, the book offers research perspectives, covering such theoretical topics as linear factor models, autoencoders, representation learning, structured probabilistic models, Monte Carlo methods, the partition function, approximate inference, and deep generative models. Deep Learning can be used by undergraduate or graduate students planning careers in either industry or research, and by software engineers who want to begin using deep learning in their products or platforms. A website offers supplementary material for both readers and instructors.

Advances in Neural Information Processing Systems 25

2012-01-01

Yichuan Zhang , Charles Sutton , Amos Storkey +1 more

Advances in Neural Information Processing Systems 19

2007-09-07

Papers from the 2006 flagship meeting on neural computation, with contributions from physicists, neuroscientists, mathematicians, statisticians, and computer scientists. The annual Neural Information Processing Systems (NIPS) conference is the flagship … Papers from the 2006 flagship meeting on neural computation, with contributions from physicists, neuroscientists, mathematicians, statisticians, and computer scientists. The annual Neural Information Processing Systems (NIPS) conference is the flagship meeting on neural computation and machine learning. It draws a diverse group of attendees—physicists, neuroscientists, mathematicians, statisticians, and computer scientists—interested in theoretical and applied aspects of modeling, simulating, and building neural-like or intelligent systems. The presentations are interdisciplinary, with contributions in algorithms, learning theory, cognitive science, neuroscience, brain imaging, vision, speech and signal processing, reinforcement learning, and applications. Only twenty-five percent of the papers submitted are accepted for presentation at NIPS, so the quality is exceptionally high. This volume contains the papers presented at the December 2006 meeting, held in Vancouver. Bradford Books imprint

Advances in Neural Information Processing Systems 14

2002-11-08

The proceedings of the 2001 Neural Information Processing Systems (NIPS) Conference. The annual conference on Neural Information Processing Systems (NIPS) is the flagship conference on neural computation. The conference is … The proceedings of the 2001 Neural Information Processing Systems (NIPS) Conference. The annual conference on Neural Information Processing Systems (NIPS) is the flagship conference on neural computation. The conference is interdisciplinary, with contributions in algorithms, learning theory, cognitive science, neuroscience, vision, speech and signal processing, reinforcement learning and control, implementations, and diverse applications. Only about 30 percent of the papers submitted are accepted for presentation at NIPS, so the quality is exceptionally high. These proceedings contain all of the papers that were presented at the 2001 conference. Bradford Books imprint

Classification and Regression Trees.

1986-03-01

John Van Ryzin , Leo Breiman , Jerome H. Friedman +2 more

Multilayer feedforward networks are universal approximators

1989-07-01

HornikK. , StinchcombeM. , WhiteH.

Advances in neural information processing systems 7

1996-04-01

Bagging predictors

1996-08-01

Leo Breiman

ggplot2

2009-01-01

Hadley Wickham

The Nature of Statistical Learning Theory

2000-01-01

Vladimir Vapnik

Latent Dirichlet Allocation

2002-11-08

David M. Blei , Andrew Y. Ng , Michael I. Jordan

Support-vector networks

1995-09-01

Corinna Cortes , Vladimir Vapnik

View PDF Chat

Distributed Optimization and Statistical Learning via the Alternating Direction Method of Multipliers

2010-01-01

Stephen Boyd

View PDF Chat

Neural Networks for Pattern Recognition

1995-11-23

Chris Bishop

Abstract This book provides the first comprehensive treatment of feed-forward neural networks from the perspective of statistical pattern recognition. After introducing the basic concepts of pattern recognition, the book describes … Abstract This book provides the first comprehensive treatment of feed-forward neural networks from the perspective of statistical pattern recognition. After introducing the basic concepts of pattern recognition, the book describes techniques for modelling probability density functions, and discusses the properties and relative merits of the multi-layer perceptron and radial basis function network models. It also motivates the use of various forms of error functions, and reviews the principal algorithms for error function minimization. As well as providing a detailed discussion of learning and generalization in neural networks, the book also covers the important topics of data processing, feature extraction, and prior knowledge. The book concludes with an extensive treatment of Bayesian techniques and their applications to neural networks.

View PDF Chat

Experimental Study and ANN-Based Predictions

2025-06-25

Vikas Bargah , Taraprasad Mohapatra , Hillol Joardar +2 more | CRC Press eBooks

Prediction System Algorithm Using QoS Values

2025-06-25

Pallavi R. Kathar | International Journal for Research in Applied Science and Engineering Technology

Correction: Paired competing neurons improving STDP supervised local learning in spiking neural networks

2025-06-25

Gaspard Goupy , Pierre Tirilly , Ioan Marius Bilasco | Frontiers in Neuroscience

AFMD: attention fusion-based multiscale decomposition network

2025-06-24

Ruidi Yang , Yuxing Mao , Yan Haowen +3 more | The Journal of Supercomputing

MuRAt-CAP-Net: A novel multi-input residual attention network for automated detection of A-phases and subtypes in cyclic alternating patterns

2025-06-24

Süleyman Yaman , Hasan Güler , Abdulkadir Şengür +2 more | Biomedical Signal Processing and Control

Implementing the Fourier Transform in a Sensor: A Benchmark Application for Neuromorphic Acoustic Sensing

2025-06-24

Hermann Folke Johann Rolf , Petro Feketa , Alexander Schaum +1 more | Neuromorphic Computing and Engineering

Abstract In changing environments, the mammalian hearing perception so far outperforms technical speech processing. This is enabled by the nonlinear dynamics of the cochlea. Inside of it, the processing is … Abstract In changing environments, the mammalian hearing perception so far outperforms technical speech processing. This is enabled by the nonlinear dynamics of the cochlea. Inside of it, the processing is based on a frequency decomposition and a sophisticated feedback loop, so that the amplification of spectrum of an external signal can vary from linear to compressive. This behavior can be mimicked by implementing a controllable Andronov-Hopf bifurcation into neuromorphic oscillators, which enables a compressive and frequency-selective response. However, the frequency decomposition with these neuromorphic oscillators has not been investigated yet. Here, we show that any oscillator, which exhibits an Andronov-Hopf bifurcation, has a unique response to external stimuli, if its bifurcation parameter is in a neighborhood of the critical point. In addition, we propose three different algorithms to enable the frequency decomposition by implementing the Fourier transform in an acoustic sensor. We found that this Fourier transform can be done by applying amplitude demodulation to the output of any oscillator exhibiting at least one Andronov-Hopf bifurcation and investigated the convergence time of the different algorithms. Our results demonstrate that the Fourier transform can be utilized for either a single oscillator, which is simple to implement, or an array of oscillators, which has a fast convergence time. Thereby, it is shown that the neuromorphic acoustic sensor consisting of these oscillators can both mimic the processing of the cochlea and be implemented into the technical unit. Finally, a potential experimental implementation using MEMS sensors is proposed.

Time-series modeling with neural flow maps

2025-06-24

Bingxian Xu , Zoey E. Ho , Yitong Huang

Abstract Constructing mathematical models from data is fundamental for understanding complex systems across scientific disciplines. However, real-world data often pose challenges such as irregular sampling, sparsity, and noise, that hinder … Abstract Constructing mathematical models from data is fundamental for understanding complex systems across scientific disciplines. However, real-world data often pose challenges such as irregular sampling, sparsity, and noise, that hinder the development of accurate, mechanistic models. In this work, we present a deep learning framework that directly reconstruct flow maps from data, assuming only that the observed patterns arise from an autonomous dynamical system. We demonstrate that our method accurately captures system dynamics across diverse settings, even with limited and irregularly sampled training data. When applied to the circadian transcriptomic time series data, it generates biologically valid predictions by integrating information across multiple organs. By parameterizing the full dynamical system, our proposed approach enables efficient computation of time derivatives and Jacobians directly from data, offering a powerful tool for analyzing and interpreting high-dimensional biological systems.

On Statistical Independence and Density Independence

2025-06-24

Milan Paštéka | Tatra Mountains Mathematical Publications

Abstract In the first part we recall the notion of statistical independence. The second part is devoted to the definition of selective density and its connection with the distribution of … Abstract In the first part we recall the notion of statistical independence. The second part is devoted to the definition of selective density and its connection with the distribution of sequences. Then we define the independence of sequences with respect to selective density. Finally, we prove that these two types of independence are equivalent.

Quantum neural networks can be normal

2025-06-24

Jayne Thompson | Nature Physics

Generating long-term stable squeezed states via multiple pump noise suppression

2025-06-23

Lijun Gao , Shaoping Shi , Bo Lü +5 more | Optics Letters

Design of structure of deep memristive reservoir and application to time series analysis

2025-06-23

Yan Zhang , Liai Pang , Xuesong Yang +2 more | Physica Scripta

Abstract Combining the characteristics of memristors with the advantages of reservoir computing, this paper proposes a novel model architecture—Deep Memristive Reservoir Computing (DMRC). This architecture integrates the dynamic behavior of … Abstract Combining the characteristics of memristors with the advantages of reservoir computing, this paper proposes a novel model architecture—Deep Memristive Reservoir Computing (DMRC). This architecture integrates the dynamic behavior of memristors into a deep reservoir computing framework. The core of this study lies in utilizing memristors to construct a random memristor array, which serves as the foundation for building a deep reservoir computing structure. The proposed RC framework consists of three main components: memristive reservoir module, nonlinear activation function module, and hierarchical driver module. The memristive reservoir module is composed of a random array of forgetting memristors, which offers rich memristor states and memory capabilities. The nonlinear activation function module ensures that each memristive reservoir layer exhibits sufficient nonlinear behavior and avoids the issue of input saturation. The hierarchical driver module, mainly implemented using amplifiers, dynamically adjusts the output of the previous reservoir layer to drive the next layer within its optimal nonlinear operating region, thereby enhancing the nonlinearity of the entire reservoir structure. In the experimental section, three time series classification tasks are conducted: waveform classification, epilepsy detection, and arrhythmia detection. Experimental results demonstrate that DMRC achieves an NRMSE of 0.0150397 in the waveform classification task, showing its strong capability in time series classification. In the epilepsy detection task, DMRC achieves an accuracy of 95.5%, and notably, a recall rate of 99.5% for detecting status epilepticus, proving that DMRC holds excellent potential for real-world applications.

Accurate phase transition predictions of synergistic contagions using spectral approach

2025-06-23

NULL AUTHOR_ID | Physical review. E

Prediction of electrical energy consumption using principal component analysis and independent components analysis

2025-06-23

Nima Saeedi , Deniz Baharvand , Kimia Shirini +1 more | The Journal of Supercomputing

Global Mittag-Leffler Synchronization of Fractional-Order Fuzzy Inertia Neural Networks with Reaction–Diffusion Terms Under Boundary Control

2025-06-23

Lin Hu , Haijun Jiang , Cheng Hu +3 more | Fractal and Fractional

This study is devoted to solving the global Mittag-Leffler synchronization problem of fractional-order fuzzy reaction–diffusion inertial neural networks by using boundary control. Firstly, the considered network model incorporates the inertia … This study is devoted to solving the global Mittag-Leffler synchronization problem of fractional-order fuzzy reaction–diffusion inertial neural networks by using boundary control. Firstly, the considered network model incorporates the inertia term, reaction–diffusion term and fuzzy logic, thereby enhancing the existing model framework. Secondly, to prevent an increase in the number of state variables due to the reduced-order approach, a non-reduced-order method is fully utilized. Additionally, a boundary controller is designed to lower resource usage. Subsequently, under the Neumann boundary condition, the mixed boundary condition and the Robin boundary condition, three synchronization conditions are established with the help of the non-reduced-order approach and LMI technique, respectively. Lastly, two numerical examples are offered to verify the reliability of the theoretical results and the availability of the boundary controller through MATLAB simulations.

The ZNF512B-NuRD complex co-ordinates a neural-specific gene expression program

2025-06-23

Habib Francis , Mehdi Sharifi Tabar , Chirag Parsania +7 more | bioRxiv (Cold Spring Harbor Laboratory)

The nucleosome remodelling and deacetylase complex (NuRD) plays a key role in chromatin regulation and a wide range of biological processes including development, haemopoiesis, immunity and neurogenesis. Its interaction with … The nucleosome remodelling and deacetylase complex (NuRD) plays a key role in chromatin regulation and a wide range of biological processes including development, haemopoiesis, immunity and neurogenesis. Its interaction with tissue-enriched and sequence-specific transcription factors (TFs) leads to distinct functional outputs in the given tissue by targeting a specific set of genes. However, how NuRD dynamically and specifically regulates gene expression in a tissue-specific manner is poorly understood. Here, we refine an N-terminal specific NuRD-interaction motif which enables direct engagement with many transcriptional regulatory proteins. Using a series of structural modelling and biochemical techniques, we show that ZNF512B, a poorly characterised neuronal-expressed zinc finger protein, directly binds to the RBBP4 subunit of the NuRD complex. Subsequent knockdown of ZNF512B results in the downregulation of several neural-related molecular pathways suggesting that ZNF512B may play a regulatory role during neurogenesis. We also show that in NTERA-2 neural cells, the expression of ZNF512B is necessary for cell growth and survival, and is markedly enhanced during neural progenitor cell (NPC) differentiation. In summary, our data suggest that ZNF512B might regulate neural-specific transcriptional programs via engagement with the NuRD complex.

Convergence of Gradient Descent for Recurrent Neural Networks: A Nonasymptotic Analysis

2025-06-23

Semih Çaycı , Atilla Eryılmaz | SIAM Journal on Mathematics of Data Science

Fixed-time synchronization of spatiotemporal fuzzy neural network via aperiodic intermittent control with hyperbolic sine function

2025-06-22

Aminamuhan Abudireman , Abdujelil Abdurahman | Journal of Applied Mathematics and Computing

Dynamic Network Plasticity and Sample Efficiency in Neural Cultures: A Comparison with Deep Learning

2025-06-22

Moein Khajehnejad , Forough Habibollahi , Alon Loeffler +3 more | Cyborg and Bionic Systems

Measurement-induced phase transition in state estimation of chaotic systems and the directed polymer

2025-06-21

NULL AUTHOR_ID | Physical Review Research

Pattern recognition and machine learning

2025-06-20

Soumyajit Mandal | Elsevier eBooks

Back-Forth Algorithm: A New Metaheuristic with Forward and Backward Shifts as Adaptive Mechanism

2025-06-20

| International journal of intelligent engineering and systems

Explainability in AI Using Fuzzy Inference Systems For the Regression Problem

2025-06-20

| Applied Mathematics & Information Sciences

Constraining the 3HDM parameter space using active learning

2025-06-20

NULL AUTHOR_ID | Physical review. D/Physical review. D.

A brain-inspired projection contrastive learning network for instantaneous learning

2025-06-19

Yanli Yang | Engineering Applications of Artificial Intelligence

Convexity in ReLU Neural Networks: Beyond ICNNs?

2025-06-19

A. Gagneux , Mathurin Massias , Emmanuel Soubies +1 more | Journal of Mathematical Imaging and Vision

View PDF Chat

A novel cognitive diagnosis system with attention residual mechanism and broad learning

2025-06-19

Jing Wang , Jialin Miao , Junwei Duan +2 more | Applied Intelligence

Rademacher learning rates for iterated random functions

2025-06-19

Nikola Sandrić | Journal of Complexity

SANTM: A Sparse Access Neural Turing Machine with local multi-head self-attention for long-term memorization

2025-06-19

Dongjing Shan , ZHU Jing , Yamei Luo | Applied Soft Computing

Data Acquisition, Processing, Displaying, and Interpretation of MT Data

2025-06-19

O.P. Mishra , D.C. Naskar | CRC Press eBooks

IEEE Transactions on Information Theory Publication Information

2025-06-19

| IEEE Transactions on Information Theory

Learning agent-based approach to the characterization of open quantum systems

2025-06-18

NULL AUTHOR_ID | Physical Review Applied

Efficient Task Allocation in Data Center Environments Based on Information Theory

2025-06-18

David H. S. Lima , André L. L. Aquino , Marília Curado | Journal of Network and Systems Management

The information-processing perspective on representation

2025-06-18

Manolo Martínez | Philosophy and the Mind Sciences

I introduce a novel framework for theorizing about representations in cognitive science, which relies on two theses. First, representations are, primarily, signals for information transmission, not as a side effect … I introduce a novel framework for theorizing about representations in cognitive science, which relies on two theses. First, representations are, primarily, signals for information transmission, not as a side effect of other functions these signals may have, but for its own sake. Second, these signals aim at efficiently trading-off three cognitive budgets: rate (or transmission and storage costs), distortion (or faithfulness of the transmitted information), and computational complexity of coders. The way to provide empirical evidence that some entity is a representation—and hence that it is involved in information transmission for the sake of information transmission, as per the first thesis—is to show that it present adaptations for efficient information transmission—as per the second thesis. The kinds of properties that cognitive scientists routinely associate with paradigmatic instances of representations are generated by adaptations for rate-distortion-complexity efficiency.

DEQ-KAN: Deep equilibrium Kolmogorov–Arnold networks for robust classification

2025-06-18

Jaber Qezelbash-Chamak | Biomedical Signal Processing and Control

ANOVA-boosting for Random Fourier Features

2025-06-18

Daniel Potts , Laura Weidensager | Applied and Computational Harmonic Analysis

View PDF Chat

Harmonic Analysis for CNNs

2025-06-17

Mehdi Ghayoumi | Chapman and Hall/CRC eBooks

Dynamic Analysis, FPGA Implementation and Application of Memristive Hopfield Neural Network with Synapse Crosstalk

2025-06-17

Meihui Shan , Yi Yang , Qing Tang +2 more | Electronics

In a biological nervous system, neurons are connected to each other via synapses to transmit information. Synaptic crosstalk is the phenomenon of mutual interference or interaction of neighboring synapses between … In a biological nervous system, neurons are connected to each other via synapses to transmit information. Synaptic crosstalk is the phenomenon of mutual interference or interaction of neighboring synapses between neurons. This phenomenon is prevalent in biological neural networks and has an important impact on the function and information processing of the neural system. In order to simulate and study this phenomenon, this paper proposes a memristor model based on hyperbolic tangent function for simulating the activation function of neurons, and constructs a three-neuron HNN model by coupling two memristors, which brings it close to the real behavior of biological neural networks, and provides a new tool for studying complex neural dynamics. The intricate nonlinear dynamics of the MHNN are examined using techniques like Lyapunov exponent analysis and bifurcation diagrams. The viability of the MHNN is confirmed through both analog circuit simulation and FPGA implementation. Moreover, an image encryption approach based on the chaotic system and a dynamic key generation mechanism are presented, highlighting the potential of the MHNN for real-world applications. The histogram shows that the encryption algorithm is effective in destroying the features of the original image. According to the sensitivity analysis, the bit change rate of the key is close to 50% when small perturbations are applied to each of the three parameters of the system, indicating that the system is highly resistant to differential attacks. The findings indicate that the MHNN displays a wide range of dynamical behaviors and high sensitivity to initial conditions, making it well-suited for applications in neuromorphic computing and information security.

Neural Optimization Techniques for Noisy-Data Observer-Based Neuro-Adaptive Control for Strict-Feedback Control Systems: Addressing Tracking and Predefined Accuracy Constraints

2025-06-17

Abdulaziz Garba Ahmad , Taher Alzahrani | Fractal and Fractional

This research proposes a fractional-order adaptive neural control scheme using an optimized backstepping (OB) approach to address strict-feedback nonlinear systems with uncertain control directions and predefined performance requirements. The OB … This research proposes a fractional-order adaptive neural control scheme using an optimized backstepping (OB) approach to address strict-feedback nonlinear systems with uncertain control directions and predefined performance requirements. The OB framework integrates both fractional-order virtual and actual controllers to achieve global optimization, while a Nussbaum-type function is introduced to handle unknown control paths. To ensure convergence to desired accuracy within a prescribed time, a fractional-order dynamic-switching mechanism and a quartic-barrier Lyapunov function are employed. An input-to-state practically stable (ISpS) auxiliary signal is designed to mitigate unmodeled dynamics, leveraging classical lemmas adapted to fractional-order systems. The study further investigates a decentralized control scenario for large-scale stochastic nonlinear systems with uncertain dynamics, undefined control directions, and unmeasurable states. Fuzzy logic systems are employed to approximate unknown nonlinearities, while a fuzzy-phase observer is designed to estimate inaccessible states. The use of Nussbaum-type functions in decentralized architectures addresses uncertainties in control directions. A key novelty of this work lies in the combination of fractional-order adaptive control, fuzzy logic estimation, and Nussbaum-based decentralized backstepping to guarantee that all closed-loop signals remain bounded in probability. The proposed method ensures that system outputs converge to a small neighborhood around the origin, even under stochastic disturbances. The simulation results confirm the effectiveness and robustness of the proposed control strategy.

Fixed-time synchronization of time-varying delayed BAM neural networks in quaternion variable

2025-06-17

Aayushi Sharma , Sudeshna Ghosh , Javid Ali +2 more | Physica Scripta

Abstract This article explores the Fixed-time (FXT) synchronization of time-varying delayed BAM neural networks in quaternion variable. By leveraging fixed-time stability theory, the Lyapunov functional method, and specific inequality techniques, … Abstract This article explores the Fixed-time (FXT) synchronization of time-varying delayed BAM neural networks in quaternion variable. By leveraging fixed-time stability theory, the Lyapunov functional method, and specific inequality techniques, a novel synchronization criterion is proposed to ensure convergence within a predefined fixed time, independent of initial conditions. Additionally, a more precise calculation of the settling time is provided, highlighting its independence from the system’s initial states. To facilitate the synchronization process, a nonlinear delayed controller is introduced. The theoretical findings are substantiated through a numerical example, confirming the framework’s effectiveness.