Sequencing-based technologies allow resolving the composition of microbial ecosystems to strain-level detail; however, coarser representations are often found to be more reproducible and more predictive of community-level properties. The general principles for selecting an appropriate level of description for modeling remain elusive. I will describe a framework for systematically comparing all possible coarse-grained descriptions by explicitly quantifying their prediction power and information content, allowing us to define the Pareto front of optimal descriptions for a given property of interest. Crucially, this Pareto front depends on ecological context; in particular, a high diversity of strains (while nominally more complex) may, in fact, facilitate coarse-grainability. I'll discuss an empirical example of diversity-enhanced coarse-grainability, and show how our framework nuances the notion of "emergent simplicity" in microbial ecology.

Despite the great success of deep learning, it remains largely a black box. In this seminar, we will describe our recent work in understanding learning dynamics and generalization of deep neural networks based on concepts and tools from statistical physics.

SGD Learning dynamics: The main search engine in deep neural networks is the Stochastic Gradient Descent (SGD) algorithm. However, despite its effectiveness, little is known about how SGD finds ``good" solutions (low generalization error) in the high-dimensional weight space. By studying weight fluctuations in SGD, we find a robust inverse relation between the weight variance in SGD and the landscape flatness, which is the opposite to the fluctuation-dissipation(response) relation in equilibrium statistical physics. We show that the noise strength in SGD depends inversely on the landscape flatness, which explains the inverse variance-flatness relation. Our study suggests that SGD serves as an ``intelligent" annealing strategy where the effective temperature self-adjusts according to the loss landscape, which allows it to find the flat minimum regions that contain generalizable solutions. Time permit, we discuss some application of these insights for solving machine learning problems [1,2].

Geometric determinants of generalization: We first report the discovery of an exact duality relation between changes in activities in a densely connected layer of neurons and the changes in their weights connecting to the next layer. The activity-weight duality leads to an explicit expression for the generalization loss, which can be decomposed into contributions from different directions in weight space. We find that the generalization loss from each direction is the product of two geometric factors (determinants): sharpness of the loss landscape at the solution and the standard deviation of the dual weights, which scales as an activity-weighted norm of the solution. By using the generalization loss decomposition, we uncover how hyperparameters in SGD, different regularization schemes (e.g., weight decay and dropout), training data size, and labeling noise affect generalization by controlling one or both factors [3].

[1] “The inverse variance-flatness relation in Stochastic-Gradient-Descent is critical for finding flat minima”, Y. Feng and Y. Tu, PNAS, 118 (9), 2021.

[2] “Phases of learning dynamics in artificial neural networks: in the absence and presence of mislabeled data”, Y. Feng and Y. Tu, Machine Learning: Science and Technology (MLST), July 19, 2021. https://iopscience.iop.org/article/10.1088/2632-2153/abf5b9/pdf

[3] “The activity-weight duality in feed forward neural networks: The geometric determinants of generalization”, Y. Feng and Y. Tu, https://arxiv.org/abs/2203.10736

Holes in elastic metamaterials, defects in 2D curved crystals, localized plastic deformations in amorphous matter and T1 transitions in epithelial tissue, are typical realizations of stress-relaxation mechanisms in different solid-like structures, interpreted as mechanical screening.

While screening theories are well established in other fields of physics, e.g. electrostatics, a unifying theory of mechanical screening applicable to crystalline, amorphous, and living-cellular matter, is still lacking. In this talk I will present a general mechanical screening theory that generalizes classical theories of solids, and introduces new moduli that are missing from the classical theories. Contrary to its electrostatic analog, the screening theory in solids is richer even in the linear case, with multiple screening regimes, predicting qualitatively new mechanical responses. Specifically, we predict a regime of screening that is mechanically similar to the celebrated Hexatic phase, in disordered matter.

The theory is tested in different physical systems, among which are disordered granular solids and models of epithelial tissue. Experiments and numeric simulations in granular, glass, and tissue models uncover a mechanical response that strictly deviate from classical elasticity, and is in full agreement with the theory. Finally I will discuss the relevance of the theory to 3D granular solids and a new Hexatic-like state in three-dimensional matter.