Information theory

Topics

Entropy of point processes

François Baccelli and Jae Oh Woo initiated a study on the entropy and mutual information of point processes (On the Entropy and Mutual Information of Point Processes). The main new mathematical objects are the relative entropy rate and the mutual information rate of two stationary point processes. They also derived expression of the mutual information rate in the case of a homogeneous point process and its displacement. This machinery is used to revisit the Gaussian noise channel in the Shannon-Poltyrev regime recently introduced in Capacity and error exponents of stationary point processes under random additive displacements.

Information theory and high dimensional stochastic geometry

The most basic capacity and error exponent questions of information theory can be expressed in terms of random geometric objects living in Euclidean spaces with dimensions tending to infinity. This approach was introduced by Venkat Anantharam and François Baccelli to evaluate random coding error exponents in channels with additive stationary and ergodic noise. More generally, the analysis of stochastic geometry in the Shannon regime leads to new high dimension stochastic geometry questions that are currently investigated. Eliza O’Reilly and and Francois Baccelli have also studied determinantal point processes in high dimensions. This work describes the strength and reach of repulsion of a typical point of certain parametric families of determinantal point process in the Shannon regime.

Mathematical problems in neuroscience

Recent advances in neuroscience provide theoretical neuroscientists with a vast wealth of new data and open questions related to information theory, high-dimensional geometry of representation and computation, and dynamics in the brain. The groups of Ila Fiete, Ngoc Mai Tran and Thibaud Taillefumier study these questions from analytical and numerical perspectives. Fiete and Tran have recently studied the learning capacity of neural networks (see “A binary Hopfield network with 1/\log(n) information rate and applications to grid cell decoding“, “ Robust exponential memory in Hopfield networks“, and “ Associative content-addressable networks with exponentially many robust stable states“).

Spatial Birth-Death Wireless Networks

In this paper, Abishek Sankararaman and François Baccelli introduce a new form of spatial dynamics motivated by ad-hoc wireless networks. They study a birth death process where particles arrive in space as a Poisson process in space and time, and depart the system on completion of file transfer. The instantaneous rate of file transfer of any link is given by the Shannon formula of treating interference as noise. As the instantaneous interference seen by a link is dependent on the configuration of links present, this dynamics is an example of one where dynamics shapes geometry and in turn the geometry shapes the dynamics. In this paper, the authors establish a sharp phase-transition for stability of such dynamics. Moreover, whenever such dynamics is stable, they prove that the steady state is a clustered point process. Through simulations, they also argue that such dynamics cannot be simplified to any form of non-spatial queuing type dynamics. Lately, this paper with Sergey Foss extended the dynamics on grids to show a similar stability phase-transition in the case of infinite network, i.e. in an infinite grid.

Members

Thibaud Taillefumier

Department of Mathematics and Department of Neuroscience
taillef@math.utexas.edu
512 475 8145
Read More »

Jae Oh Woo

Department of Mathematics and Department of Electrical and Computer Engineering, UT Austin
jaeoh.woo@utexas.edu
Read More »

Rachel Ward

Department of Mathematics, UT Austin
rward@math.utexas.edu
Read More »

Sujay Sanghavi

Department of Electrical and Computer Engineering, UT Austin
sanghavi@mail.utexas.edu
512 475 9798
Read More »

Abishek Sankararaman

Department of Electrical and Computer Engineering, UT Austin
abishek@utexas.edu
Read More »

Mayank Manjrekar

Department of Mathematics, UT Austin
mmanjrekar@math.utexas.edu
Read More »

Sriram Vishwanath

Department of Electrical and Computer Engineering, UT Austin
sriram@ece.utexas.edu
512 471 1190
Read More »

Gustavo de Veciana

Department of Electrical and Computer Engineering, UT Austin
gustavo@ece.utexas.edu
512 471 1573
Read More »

François Baccelli

Department of Mathematics and Deparment of Electrical and Computer Engineering, UT Austin
baccelli@math.utexas.edu
512 471 17 54
Read More »

Publications

2017-06-01_1000994

Spatial Birth Death Wireless Networks

Abishek Sankararaman and François Baccelli IEEE Transactions on Information Theory 63(6): 3964-3982 (2017)
Download PDF
2017_1000869 2016-08-11_1000660

On the Entropy and Mutual Information of Point Processes

François Baccelli, Jae Oh Woo IEEE International Symposium on Information Theory 2016/695-699
Download PDF
2015-04-21_1000233

Capacity and Error Exponents of Stationary Point Processes under Random Additive Displacements

Venkat Anantharam and François Baccelli Advances Applied Prob. Volume 47, Number 1, 2015
Download PDF
2014-11-17_1000382

Robust exponential memory in Hopfield networks

Christopher Hillar and Ngoc Mai Tran arXiV 2014
Download PDF
2014-07-22_1000381

A binary Hopfield network with 1/\log(n) information rate and applications to grid cell decoding

Ila Fiete, David J. Schwab and Ngoc M Tran 2nd Workshop on Biological Distributed Algorithms 2014
Download PDF
2000-01-01_1000992

Community Detection on Euclidean Random Graphs

Abishek Sankararaman and François Baccelli ArXiv 2017
Download PDF
2000-01-01_1001035

Interference Queuing Networks on Grids

Abishek Sankararaman, François Baccelli and Sergey Foss ArXiv 2017
Download PDF