1. F. Albertini and E.D. Sontag. Transitivity and forward accessibility of discrete-time nonlinear systems. In Analysis of controlled dynamical systems (Lyon, 1990), volume 8 of Progr. Systems Control Theory, pages 21-34. Birkhäuser Boston, Boston, MA, 1991.



2. E.D. Sontag. Capabilities and training of feedforward nets. In Neural networks (New Brunswick, NJ, 1990), pages 303-321. Academic Press, Boston, MA, 1991. [PDF] Keyword(s): machine learning, machine learning, neural networks. Abstract:

   This paper surveys recent work by the author on learning and representational capabilities of feedforward nets. The learning results show that, among two possible variants of the so-called backpropagation training method for sigmoidal nets, both of which variants are used in practice, one is a better generalization of the older perceptron training algorithm than the other. The representation results show that nets consisting of sigmoidal neurons have at least twice the representational capabilities of nets that use classical threshold neurons, at least when this increase is quantified in terms of classification power. On the other hand, threshold nets are shown to be more useful when approximating implicit functions, as illustrated with an application to a typical control problem.




3. E.D. Sontag. Input/output and state-space stability. In New trends in systems theory (Genoa, 1990), volume 7 of Progr. Systems Control Theory, pages 684-691. Birkhäuser Boston, Boston, MA, 1991. [PDF] Keyword(s): input to state stability, input to state stability. Abstract:

   This conference paper reviews various results relating state-space (Lyapunov) stabilization and exponential stabilization to several notions of input/output or bounded-input bounded-output stabilization. It also provides generalizations of some of these results to systems with saturating controls. Some of these latter results were not included in journal papers.




4. E.D. Sontag. Kalman's controllability rank condition: from linear to nonlinear. In Mathematical system theory, pages 453-462. Springer, Berlin, 1991. [PDF] Keyword(s): controllability. Abstract:

   The notion of controllability was identified by Kalman as one of the central properties determining system behavior. His simple rank condition is ubiquitous in linear systems analysis. This article presents an elementary and expository overview of the generalizations of this test to a condition for testing accessibility of discrete and continuous time nonlinear systems.




5. Y. Lin and E.D. Sontag. A universal formula for stabilization with bounded controls. Systems Control Lett., 16(6):393-397, 1991. [PDF] [doi:http://dx.doi.org/10.1016/0167-6911(91)90111-Q] Keyword(s): stabilization, nonlinear systems, saturation, bounded inputs, control-Lyapunov functions, real-analytic functions. Abstract:

   We provide a formula for a stabilizing feedback law using a bounded control, under the assumption that an appropriate control-Lyapunov function is known. Such a feedback, smooth away from the origin and continuous everywhere, is known to exist via Artstein's Theorem. As in the unbounded-control case treated in a previous note, we provide an explicit and ``universal'' formula given by an algebraic function of Lie derivatives. In particular, we extend to the bounded case the result that the feedback can be chosen analytic if the Lyapunov function and the vector fields defining the system are analytic.




6. H. T. Siegelmann and E.D. Sontag. Turing computability with neural nets. Appl. Math. Lett., 4(6):77-80, 1991. [PDF] Keyword(s): machine learning, neural networks, computational complexity, recurrent neural networks. Abstract:

   This paper shows the existence of a finite neural network, made up of sigmoidal neurons, which simulates a universal Turing machine. It is composed of less than 100,000 synchronously evolving processors, interconnected linearly. High-order connections are not required. (Note: this paper was placed here by special request. The results in this paper have been by now improved considerably: see the JCSS pape which among other aspects provides a polynomial time simulation. This paper, based on a unary encoding, results in an exponential slowdown).




7. E.D. Sontag and H.J. Sussmann. Back propagation separates where perceptrons do. Neural Networks, 4(2):243-249, 1991. [PDF] [doi:http://dx.doi.org/10.1016/0893-6080(91)90008-S] Keyword(s): machine learning, neural networks. Abstract:

   Feedforward nets with sigmoidal activation functions are often designed by minimizing a cost criterion. It has been pointed out before that this technique may be outperformed by the classical perceptron learning rule, at least on some problems. In this paper, we show that no such pathologies can arise if the error criterion is of a threshold LMS type, i.e., is zero for values ``beyond'' the desired target values. More precisely, we show that if the data are linearly separable, and one considers nets with no hidden neurons, then an error function as above cannot have any local minima that are not global. In addition, the proof gives the following stronger result, under the stated hypotheses: the continuous gradient adjustment procedure is such that from any initial weight configuration a separating set of weights is obtained in finite time. This is a precise analogue of the Perceptron Learning Theorem. The results are then compared with the more classical pattern recognition problem of threshold LMS with linear activations, where no spurious local minima exist even for nonseparable data: here it is shown that even if using the threshold criterion, such bad local minima may occur, if the data are not separable and sigmoids are used. keywords = { neural networks , feedforward neural nets },

1. F. Albertini and E.D. Sontag. Accessibility of discrete-time nonlinear systems, and some relations to chaotic dynamics. In Proc. Conf. Inform. Sci. and Systems, John Hopkins University, March 1991, pages 731-736, 1991.



2. F. Albertini and E.D. Sontag. Some connections between chaotic dynamical systems and control systems. In Proc. European Control Conf. , Vol 1, Grenoble, July 1991, pages 58-163, 1991. [PDF] Keyword(s): chaotic systems, controllability. Abstract:

   This paper shows how to extend recent results of Colonius and Kliemann, regarding connections between chaos and controllability, from continuous to discrete time. The extension is nontrivial because the results all rely on basic properties of the accessibility Lie algebra which fail to hold in discrete time. Thus, this paper first develops further results in nonlinear accessibility, and then shows how a theorem can be proved, which while analogous to the one given in the work by Colonius and Klieman, also exhibits some important differences. A counterexample is used to show that the theorem given in continuous time cannot be generalized in a straightforward manner.




3. Y. Lin and E.D. Sontag. Further universal formulas for Lyapunov approaches to nonlinear stabilization. In Proc. Conf. Inform. Sci. and Systems, John Hopkins University, March 1991, pages 541-546, 1991.



4. W. Maass, G. Schnitger, and E.D. Sontag. On the computational power of sigmoid versus Boolean threshold circuits (extended abstract). In Proceedings of the 32nd annual symposium on Foundations of computer science, Los Alamitos, CA, USA, pages 767-776, 1991. IEEE Computer Society Press. Keyword(s): machine learning, neural networks, theory of computing and complexity.



5. R. Schwarzschild and E.D. Sontag. Algebraic theory of sign-linear systems. In Proc. Amer. Automatic Control Conf., Boston, June 1991, pages 799-804, 1991.



6. R. Schwarzschild and E.D. Sontag. Quantized systems, saturated measurements, and sign-linear systems. In Proc. Conf. Inform. Sci. and Systems, John Hopkins University, March 1991, pages 134-139, 1991. Keyword(s): observability, saturation.



7. E.D. Sontag. Capabilities of four- vs three-layer nets, and control applications. In Proc. Conf. Inform. Sci. and Systems, John Hopkins University, March 1991, pages 558-563, 1991.



8. E.D. Sontag. Feedback Stabilization Using Two-Hidden-Layer Nets. In Proc. Amer. Automatic Control Conf. , Boston, June 1991, pages 815-820, 1991.



9. E.D. Sontag and Y. Wang. I/O equations for nonlinear systems and observation spaces. In Proc. IEEE Conf. Decision and Control, Brighton, UK, Dec. 1991, IEEE Publications, 1991, pages 720-725, 1991. [PDF] Keyword(s): identifiability, observability, realization theory, real-analytic functions. Abstract:

   This paper studies various types of input/output representations for nonlinear continuous time systems. The algebraic and analytic i/o equations studied in previous papers by the authors are generalized to integral and integro-differential equations, and an abstract notion is also considered. New results are given on generic observability, and these results are then applied to give conditions under which that the minimal order of an equation equals the minimal possible dimension of a realization, just as with linear systems but in contrast to the discrete time nonlinear theory.

This material is presented to ensure timely dissemination of scholarly and technical work. Copyright and all rights therein are retained by authors or by other copyright holders.

This document was translated from BibT_EX by bibtex2html