François Baccelli and Jae Oh Woo initiated a study on the entropy and mutual information of point processes (On the Entropy and Mutual Information of Point Processes). The main new mathematical objects are the relative entropy rate and the mutual information rate of two stationary point processes. They also derived expression of the mutual information rate in the case of a homogeneous point process and its displacement. This machinery is used to revisit the Gaussian noise channel in the Shannon-Poltyrev regime recently introduced in Capacity and error exponents of stationary point processes under random additive displacements.
The most basic capacity and error exponent questions of information theory can be expressed in terms of random geometric objects living in Euclidean spaces with dimensions tending to infinity. This approach was introduced by Venkat Anantharam and François Baccelli to evaluate random coding error exponents in channels with additive stationary and ergodic noise. More generally, the analysis of stochastic geometry in the Shannon regime leads to new high dimension stochastic geometry questions that are currently investigated. Eliza O’Reilly and and Francois Baccelli have also studied determinantal point processes in high dimensions. This work describes the strength and reach of repulsion of a typical point of certain parametric families of determinantal point process in the Shannon regime.
Recent advances in neuroscience provide theoretical neuroscientists with a vast wealth of new data and open questions related to information theory, high-dimensional geometry of representation and computation, and dynamics in the brain. The groups of Ila Fiete, Ngoc Mai Tran and Thibaud Taillefumier study these questions from analytical and numerical perspectives. Fiete and Tran have recently studied the learning capacity of neural networks (see “A binary Hopfield network with 1/\log(n) information rate and applications to grid cell decoding“, “ Robust exponential memory in Hopfield networks“, and “ Associative content-addressable networks with exponentially many robust stable states“).