Papers by M. D. Kvalheim and E.D. Sontag

Given a "data manifold" M⊂ ℝ^n and "latent space" ℝ^ℓ, an autoencoder is a pair of continuous maps consisting of an "encoder" E: ℝ^n→ ℝ^ℓ and "decoder" D: ℝ^ℓ→ ℝ^n such that the "round trip" map D E is as close as possible to the identity map id_M on M. We present various topological limitations and capabilites inherent to the search for an autoencoder, and describe capabilities for autoencoding dynamical systems having M as an invariant manifold.

We give a proof of an extension of the Hartman-Grobman theorem to nonhyperbolic but asymptotically stable equilibria of vector fields. Moreover, the linearizing topological conjugacy is (i) defined on the entire basin of attraction if the vector field is complete, and (ii) a C^k≥ 1 diffeomorphism on the complement of the equilibrium if the vector field is C^k and the underlying space is not 5-dimensional. We also show that the C^k statement in the 5-dimensional case is equivalent to the 4-dimensional smooth Poincaré conjecture.

Deep neural network autoencoders are routinely used computationally for model reduction. They allow recognizing the intrinsic dimension of data that lie in a k-dimensional subset K of an input Euclidean space ℝ^n. The underlying idea is to obtain both an encoding layer that maps ℝ^n into ℝ^k (called the bottleneck layer or the space of latent variables) and a decoding layer that maps ℝ^k back into ℝ^n, in such a way that the input data from the set K is recovered when composing the two maps. This is achieved by adjusting parameters (weights) in the network to minimize the discrepancy between the input and the reconstructed output. Since neural networks (with continuous activation functions) compute continuous maps, the existence of a network that achieves perfect reconstruction would imply that K is homeomorphic to a k-dimensional subset of ℝ^k, so clearly there are topological obstructions to finding such a network. On the other hand, in practice the technique is found to "work" well, which leads one to ask if there is a way to explain this effectiveness. We show that, up to small errors, indeed the method is guaranteed to work. This is done by appealing to certain facts from differential geometry. A computational example is also included to illustrate the ideas.