The mutual information of a stochastic binary channel: validity of the replica symmetry ansatz
The mutual information of a stochastic binary channel: validity of the replica symmetry ansatz
We calculate the mutual information (MI) of a two-layered neural network with noiseless, continuous inputs and binary, stochastic outputs under several assumptions on the synaptic efficiencies. The interesting regime corresponds to the limit where the number of both input and output units is large but their ratio is kept fixed …