Computations in Science Seminars
Mar 2023
22
Wed 12:15
Mikhail Tikhonov, Washington University
Host: Arvind Murugan ()
Organizer: Daniel Seara ()
Quantifying the Coarse-Grainability of Microbial Ecosystems

Sequencing-based technologies allow resolving the composition of microbial ecosystems to strain-level detail; however, coarser representations are often found to be more reproducible and more predictive of community-level properties. The general principles for selecting an appropriate level of description for modeling remain elusive. I will describe a framework for systematically comparing all possible coarse-grained descriptions by explicitly quantifying their prediction power and information content, allowing us to define the Pareto front of optimal descriptions for a given property of interest. Crucially, this Pareto front depends on ecological context; in particular, a high diversity of strains (while nominally more complex) may, in fact, facilitate coarse-grainability. I'll discuss an empirical example of diversity-enhanced coarse-grainability, and show how our framework nuances the notion of "emergent simplicity" in microbial ecology.

Mar 2023
29
Wed 12:15
Yuhai Tu, IBM TJ Watson Research
Host: Arvind Murugan ()
Organizer: Kabir Husain ()
Can physicists help understand Deep Learning?

Despite the great success of deep learning, it remains largely a black box. In this seminar, we will describe our recent work in understanding learning dynamics and generalization of deep neural networks based on concepts and tools from statistical physics.

SGD Learning dynamics: The main search engine in deep neural networks is the Stochastic Gradient Descent (SGD) algorithm. However, despite its effectiveness, little is known about how SGD finds ``good" solutions (low generalization error) in the high-dimensional weight space. By studying weight fluctuations in SGD, we find a robust inverse relation between the weight variance in SGD and the landscape flatness, which is the opposite to the fluctuation-dissipation(response) relation in equilibrium statistical physics. We show that the noise strength in SGD depends inversely on the landscape flatness, which explains the inverse variance-flatness relation. Our study suggests that SGD serves as an ``intelligent" annealing strategy where the effective temperature self-adjusts according to the loss landscape, which allows it to find the flat minimum regions that contain generalizable solutions. Time permit, we discuss some application of these insights for solving machine learning problems [1,2].

Geometric determinants of generalization: We first report the discovery of an exact duality relation between changes in activities in a densely connected layer of neurons and the changes in their weights connecting to the next layer. The activity-weight duality leads to an explicit expression for the generalization loss, which can be decomposed into contributions from different directions in weight space. We find that the generalization loss from each direction is the product of two geometric factors (determinants): sharpness of the loss landscape at the solution and the standard deviation of the dual weights, which scales as an activity-weighted norm of the solution. By using the generalization loss decomposition, we uncover how hyperparameters in SGD, different regularization schemes (e.g., weight decay and dropout), training data size, and labeling noise affect generalization by controlling one or both factors [3].

[1] “The inverse variance-flatness relation in Stochastic-Gradient-Descent is critical for finding flat minima”, Y. Feng and Y. Tu, PNAS, 118 (9), 2021.

[2] “Phases of learning dynamics in artificial neural networks: in the absence and presence of mislabeled data”, Y. Feng and Y. Tu, Machine Learning: Science and Technology (MLST), July 19, 2021. https://iopscience.iop.org/article/10.1088/2632-2153/abf5b9/pdf

[3] “The activity-weight duality in feed forward neural networks: The geometric determinants of generalization”, Y. Feng and Y. Tu, https://arxiv.org/abs/2203.10736

Apr 2023
5
Wed 12:15
Saverio Spagnolie, University of Wisconsin-Madison
Apr 2023
12
Wed 12:15
OPEN
Apr 2023
19
Wed 12:15
OPEN
Apr 2023
26
Wed 12:15
Michael Moshe, Hebrew University of Jerusalem
Host: Efi Efrati ()
A General Theory of Mechanical Screening and Hexatic Mechanics in Granular Matter

Holes in elastic metamaterials, defects in 2D curved crystals, localized plastic deformations in amorphous matter and T1 transitions in epithelial tissue, are typical realizations of stress-relaxation mechanisms in different solid-like structures, interpreted as mechanical screening.

While screening theories are well established in other fields of physics, e.g. electrostatics, a unifying theory of mechanical screening applicable to crystalline, amorphous, and living-cellular matter, is still lacking. In this talk I will present a general mechanical screening theory that generalizes classical theories of solids, and introduces new moduli that are missing from the classical theories. Contrary to its electrostatic analog, the screening theory in solids is richer even in the linear case, with multiple screening regimes, predicting qualitatively new mechanical responses. Specifically, we predict a regime of screening that is mechanically similar to the celebrated Hexatic phase, in disordered matter.

The theory is tested in different physical systems, among which are disordered granular solids and models of epithelial tissue. Experiments and numeric simulations in granular, glass, and tissue models uncover a mechanical response that strictly deviate from classical elasticity, and is in full agreement with the theory. Finally I will discuss the relevance of the theory to 3D granular solids and a new Hexatic-like state in three-dimensional matter.

May 2023
3
Wed 12:15
OPEN
May 2023
10
Wed 12:15
William Bialek, Princeton University
Host: Stephanie Palmer ()
May 2023
17
Wed 12:15
Itai Cohen, Cornell University
Host: Stephanie Palmer (), Sid Nagel ()
May 2023
24
Wed 12:15
Benjamin Machta, Yale University
Organizer: Daniel Seara ()
May 2023
31
Wed 12:15
Petia Vlahovska, Northwestern University