Information theory

Topics

Entropy of point processes

François Baccelli and Jae Oh Woo initiated a study on the entropy and mutual information of point processes (On the Entropy and Mutual Information of Point Processes). The main new mathematical objects are the relative entropy rate and the mutual information rate of two stationary point processes. They also derived expression of the mutual information rate in the case of a homogeneous point process and its displacement. This machinery is used to revisit the Gaussian noise channel in the Shannon-Poltyrev regime recently introduced in Capacity and error exponents of stationary point processes under random additive displacements.

Information theory and high dimensional stochastic geometry

The most basic capacity and error exponent questions of information theory can be expressed in terms of random geometric objects living in Euclidean spaces with dimensions tending to infinity. This approach was introduced by Venkat Anantharam and François Baccelli to evaluate random coding error exponents in channels with additive stationary and ergodic noise. More generally, the analysis of stochastic geometry in the Shannon regime leads to new high dimension stochastic geometry questions that are currently investigated. Eliza O’Reilly and and Francois Baccelli have also studied determinantal point processes in high dimensions. This work describes the strength and reach of repulsion of a typical point of certain parametric families of determinantal point process in the Shannon regime.

Mathematical problems in neuroscience

Recent advances in neuroscience provide theoretical neuroscientists with a vast wealth of new data and open questions related to information theory, high-dimensional geometry of representation and computation, and dynamics in the brain. The groups of Ila Fiete, Ngoc Mai Tran and Thibaud Taillefumier study these questions from analytical and numerical perspectives. Fiete and Tran have recently studied the learning capacity of neural networks (see “A binary Hopfield network with 1/\log(n) information rate and applications to grid cell decoding“, “ Robust exponential memory in Hopfield networks“, and “ Associative content-addressable networks with exponentially many robust stable states“).

Members

Thibaud Taillefumier

Department of Mathematics and Department of Neuroscience
taillef@math.utexas.edu
512 475 8145
Read More »

Jae Oh Woo

Department of Mathematics and Department of Electrical and Computer Engineering, UT Austin
jaeoh.woo@utexas.edu
Read More »

Rachel Ward

Department of Mathematics, UT Austin
rward@math.utexas.edu
Read More »

Sujay Sanghavi

Department of Electrical and Computer Engineering, UT Austin
sanghavi@mail.utexas.edu
512 475 9798
Read More »

Abishek Sankararaman

Department of Electrical and Computer Engineering, UT Austin
abishek@utexas.edu
Read More »

Mayank Manjrekar

Department of Mathematics, UT Austin
mmanjrekar@math.utexas.edu
Read More »

Sriram Vishwanath

Department of Electrical and Computer Engineering, UT Austin
sriram@ece.utexas.edu
512 471 1190
Read More »

Gustavo de Veciana

Department of Electrical and Computer Engineering, UT Austin
gustavo@ece.utexas.edu
512 471 1573
Read More »

François Baccelli

Department of Mathematics and Deparment of Electrical and Computer Engineering, UT Austin
baccelli@math.utexas.edu
512 471 17 54
Read More »

Publications

2017_1000869 2016-08-11_1000660

On the Entropy and Mutual Information of Point Processes

François Baccelli, Jae Oh Woo IEEE International Symposium on Information Theory 2016/695-699
Download PDF
2015-04-21_1000233

Capacity and Error Exponents of Stationary Point Processes under Random Additive Displacements

Venkat Anantharam and François Baccelli Advances Applied Prob. Volume 47, Number 1, 2015
Download PDF
2014-11-17_1000382

Robust exponential memory in Hopfield networks

Christopher Hillar and Ngoc Mai Tran arXiV 2014
Download PDF
2014-07-22_1000381

A binary Hopfield network with 1/\log(n) information rate and applications to grid cell decoding

Ila Fiete, David J. Schwab and Ngoc M Tran 2nd Workshop on Biological Distributed Algorithms 2014
Download PDF