BACK TO INDEX

Publications about 'feedforward'
Articles in journal or book chapters
  1. E.D. Sontag. Examples of computation of exact moment dynamics for chemical reaction networks. In R. Tempo, S. Yurkovich, and P. Misra, editors, Emerging Applications of Control and Systems Theory, volume 473 of Lecture Notes in Control and Inform. Sci., pages 295-312. Springer-Verlag, Berlin, 2018. [PDF] Keyword(s): chemical master equations, stochastics, moments, chemical reaction networks, incoherent feedforward loop, feedforward, IFFL.
    Abstract:
    The study of stochastic biomolecular networks is a key part of systems biology, as such networks play a central role in engineered synthetic biology constructs as well as in naturally occurring cells. This expository paper reviews in a unified way a pair of recent approaches to the finite computation of statistics for chemical reaction networks.


  2. T.H. Segall-Shapiro, E. D. Sontag, and C. A. Voigt. Engineered promoters enable constant gene expression at any copy number in bacteria. Nature Biotechnology, 36:352-358, 2018. [PDF] Keyword(s): synthetic biology, systems biology, genetic circuits, gene copy number, incoherent feedforward loop, feedforward, IFFL.
    Abstract:
    This paper deals with the design of promoters that maintain constant levels of expression, whether they are carried at single copy in the genome or on high-copy plasmids. The design is based on an incoherent feedforward loop (iFFL) with a perfectly non-cooperative repression. The circuits are implemented in E. coli using Transcription Activator Like Effectors (TALEs). The resulting stabilized promoters generate near identical expression across different genome locations and plasmid backbones (pSC101, p15a, ColE1, pUC), and also provide robustness to strain mutations and growth media. Further, their strength is tunable and can be used to maintain constant ratios between proteins.


  3. J. K. Kim and E.D. Sontag. Reduction of multiscale stochastic biochemical reaction networks using exact moment derivation. PLoS Computational Biology, 13:13(6): e1005571, 2017. [PDF] Keyword(s): systems biology, biochemical networks, stochastic systems, Chemical Master Equation, chemical reaction networks, moments, molecular networks, complex-balanced networks.
    Abstract:
    Biochemical reaction networks in cells frequently consist of reactions with disparate timescales. Stochastic simulations of such multiscale BRNs are prohibitively slow due to the high computational cost incurred in the simulations of fast reactions. One way to resolve this problem is to replace fast species by their stationary conditional expectation values conditioned on slow species. While various approximations schemes for this quasi-steady state approximation have been developed, they often lead to considerable errors. This paper considers two classes of multiscale BRNs which can be reduced by through an exact QSS rather than approximations. Specifically, we assume that fast species constitute either a feedforward network or a complex balanced network. Exact reductions for various examples are derived, and the computational advantages of this approach are illustrated through simulations.


  4. S. J. Rahi, J. Larsch, K. Pecani, N. Mansouri, A. Y. Katsov, K. Tsaneva-Atanasova, E. D. Sontag, and F. R. Cross. Oscillatory stimuli differentiate adapting circuit topologies. Nature Methods, 14:1010-1016, 2017. [PDF] Keyword(s): biochemical networks, periodic behaviors, monotone systems, entrainment, oscillations, incoherent feedforward loop, feedforward, IFFL.
    Abstract:
    Elucidating the structure of biological intracellular networks from experimental data remains a major challenge. This paper studies two types of ``response signatures'' to identify specific circuit motifs, from the observed response to periodic inputs. In particular, the objective is to distinguish negative feedback loops (NFLs) from incoherent feedforward loops (IFFLs), which are two types of circuits capable of producing exact adaptation. The theory of monotone systems with inputs is used to show that ``period skipping'' (non-harmonic responses) is ruled out in IFFL's, and a notion called ``refractory period stabilization'' is also analyzed. The approach is then applied to identify a circuit dominating cell cycle timing in yeast, and to uncover a calcium-mediated NFL circuit in \emph{C.elegans} olfactory sensory neurons.


  5. E.D. Sontag. A dynamical model of immune responses to antigen presentation predicts different regions of tumor or pathogen elimination. Cell Systems, 4:231-241, 2017. [PDF] Keyword(s): scale invariance, fold change detection, T cells, incoherent feedforward loops, immunology, cancer, internal model principle, incoherent feedforward loop, feedforward, IFFL.
    Abstract:
    Since the early 1990s, many authors have independently suggested that self/nonself recognition by the immune system might be modulated by the rates of change of antigen challenges. This paper introduces an extremely simple and purely conceptual mathematical model that allows dynamic discrimination of immune challenges. The main component of the model is a motif which is ubiquitous in systems biology, the incoherent feedforward loop, which endows the system with the capability to estimate exponential growth exponents, a prediction which is consistent with experimental work showing that exponentially increasing antigen stimulation is a determinant of immune reactivity. Combined with a bistable system and a simple feedback repression mechanism, an interesting phenomenon emerges as a tumor growth rate increases: elimination, tolerance (tumor growth), again elimination, and finally a second zone of tolerance (tumor escape). This prediction from our model is analogous to the ``two-zone tumor tolerance'' phenomenon experimentally validated since the mid 1970s. Moreover, we provide a plausible biological instantiation of our circuit using combinations of regulatory and effector T cells.


  6. T. Kang, R. Moore, Y. Li, E.D. Sontag, and L. Bleris. Discriminating direct and indirect connectivities in biological networks. Proc Natl Acad Sci USA, 112:12893-12898, 2015. [PDF] Keyword(s): modular response analysis, stochastic systems, reverse engineering, gene networks, synthetic biology, feedforward.
    Abstract:
    Reverse engineering of biological pathways involves an iterative process between experiments, data processing, and theoretical analysis. In this work, we engineer synthetic circuits, subject them to perturbations, and then infer network connections using a combination of nonparametric single-cell data resampling and modular response analysis. Intriguingly, we discover that recovered weights of specific network edges undergo divergent shifts under differential perturbations, and that the particular behavior is markedly different between different topologies. Investigating topological changes under differential perturbations may address the longstanding problem of discriminating direct and indirect connectivities in biological networks.


  7. M. Skataric, E.V. Nikolaev, and E.D. Sontag. A fundamental limitation to fold-change detection by biological systems with multiple time scales. IET Systems Biology, 9:1-15, 2015. [PDF] Keyword(s): singular perturbations, scale invariance, systems biology, transient behavior, symmetries, fcd, fold-change detection, incoherent feedforward loop, feedforward, IFFL.
    Abstract:
    The phenomenon of fold-change detection, or scale invariance, is exhibited by a variety of sensory systems, in both bacterial and eukaryotic signaling pathways. It has been often remarked in the systems biology literature that certain systems whose output variables respond at a faster time scale than internal components give rise to an approximate scale-invariant behavior, allowing approximate fold-change detection in stimuli. This paper establishes a fundamental limitation of such a mechanism, showing that there is a minimal fold-change detection error that cannot be overcome, no matter how large the separation of time scales is. To illustrate this theoretically predicted limitation, we discuss two common biomolecular network motifs, an incoherent feedforward loop and a feedback system, as well as a published model of the chemotaxis signaling pathway of Dictyostelium discoideum.


  8. E.D. Sontag and A. Singh. Exact moment dynamics for feedforward nonlinear chemical reaction networks. IEEE Life Sciences Letters, 1:26-29, 2015. [PDF] Keyword(s): systems biology, biochemical networks, stochastic systems, Chemical Master Equation, chemical reaction networks.
    Abstract:
    Chemical systems are inherently stochastic, as reactions depend on random (thermal) motion. This motivates the study of stochastic models, and specifically the Chemical Master Equation (CME), a discrete-space continuous-time Markov process that describes stochastic chemical kinetics. Exact studies using the CME are difficult, and several moment closure tools related to "mass fluctuation kinetics" and "fluctuation-dissipation" formulas can be used to obtain approximations of moments. This paper, in contrast, introduces a class of nonlinear chemical reaction networks for which exact computation is possible, by means of finite-dimensional linear differential equations. This class allows second and higher order reactions, but only under special assumptions on structure and/or conservation laws.


  9. L. Bleris, Z. Xie, D. Glass, A. Adadey, E.D. Sontag, and Y. Benenson. Synthetic incoherent feed-forward circuits show adaptation to the amount of their genetic template. Molecular Systems Biology, 7:519-, 2011. [PDF] Keyword(s): adaptation, feedforward loops, systems biology, synthetic biology, incoherent feedforward loop, feedforward, IFFL.
    Abstract:
    Natural and synthetic biological networks must function reliably in the face of fluctuating stoichiometry of their molecular components. These fluctuations are caused in part by changes in relative expression efficiency and the DNA template amount of the network-coding genes. Gene product levels could potentially be decoupled from these changes via built-in adaptation mechanisms, thereby boosting network reliability. Here we show that a mechanism based on an incoherent feed-forward motif enables adaptive gene expression in mammalian cells. We modeled, synthesized, and tested transcriptional and post-transcriptional incoherent loops and found that in all cases the gene product adapts to changes in DNA template abundance. We also observed that the post-transcriptional form results in superior adaptation behavior, higher absolute expression levels, and lower intrinsic fluctuations. Our results support a previously-hypothesized endogenous role in gene dosage compensation for such motifs and suggest that their incorporation in synthetic networks will improve their robustness and reliability.


  10. O. Shoval, U. Alon, and E.D. Sontag. Symmetry invariance for adapting biological systems. SIAM Journal on Applied Dynamical Systems, 10:857-886, 2011. Note: See here for a small typo: http://www.math.rutgers.edu/(tilde)sontag/FTPDIR/shoval.alon.sontag.erratum.pdf.[PDF] Keyword(s): adaptation, feedforward loops, integral feedback, scale invariance, systems biology, transient behavior, symmetries, fcd, fold-change detection, incoherent feedforward loop, feedforward, IFFL.
    Abstract:
    Often, the ultimate goal of regulation is to maintain a narrow range of concentration levels of vital quantities (homeostasis, adaptation) while at the same time appropriately reacting to changes in the environment (signal detection or sensitivity). Much theoretical, modeling, and analysis effort has been devoted to the understanding of these questions, traditionally in the context of steady-state responses to constant or step-changing stimuli. In this paper, we present a new theorem that provides a necessary and sufficient characterization of invariance of transient responses to symmetries in inputs. A particular example of this property, scale invariance (a.k.a. "fold change detection"), appears to be exhibited by biological sensory systems ranging from bacterial chemotaxis pathways to signal transduction mechanisms in eukaryotes. The new characterization amounts to the solvability of an associated partial differential equation. It is framed in terms of a notion which considerably extends equivariant actions of compact Lie groups. For several simple system motifs that are recurrent in biology, the solvability criterion may be checked explicitly.


  11. O. Shoval, L. Goentoro, Y. Hart, A. Mayo, E.D. Sontag, and U. Alon. Fold change detection and scalar symmetry of sensory input fields. Proc Natl Acad Sci USA, 107:15995-16000, 2010. [PDF] Keyword(s): adaptation, feedforward loops, integral feedback, scale invariance, systems biology, transient behavior, symmetries, fcd, fold-change detection, incoherent feedforward loop, feedforward, IFFL.
    Abstract:
    Certain cellular sensory systems display fold-change detection (FCD): a response whose entire shape, including amplitude and duration, depends only on fold-changes in input, and not on absolute changes. Thus, a step change in input from, say, level 1 to 2, gives precisely the same dynamical output as a step from level 2 to 4, since the steps have the same fold-change. We ask what is the benefit of FCD, and show that FCD is necessary and sufficient for sensory search to be independent of multiplying the input-field by a scalar. Thus the FCD search pattern depends only on the spatial profile of the input, and not on its amplitude. Such scalar symmetry occurs in a wide range of sensory inputs, such as source strength multiplying diffusing/convecting chemical fields sensed in chemotaxis, ambient light multiplying the contrast field in vision, and protein concentrations multiplying the output in cellular signaling-systems.Furthermore, we demonstrate that FCD entails two features found across sensory systems, exact adaptation and Weber's law, but that these two features are not sufficient for FCD. Finally, we present a wide class of mechanisms that have FCD, including certain non-linear feedback and feedforward loops.. We find that bacterial chemotaxis displays feedback within the present class, and hence is expected to show FCD. This can explain experiments in which chemotaxis searches are insensitive to attractant source levels. This study thus suggests a connection between properties of biological sensory systems and scalar symmetry stemming from physical properties of their input-fields.


  12. E.D. Sontag. Remarks on Feedforward Circuits, Adaptation, and Pulse Memory. IET Systems Biology, 4:39-51, 2010. [PDF] Keyword(s): adaptation, feedforward loops, integral feedback, systems biology, transient behavior, incoherent feedforward loop, feedforward, IFFL.
    Abstract:
    This note studies feedforward circuits as models for perfect adaptation to step signals in biological systems. A global convergence theorem is proved in a general framework, which includes examples from the literature as particular cases. A notable aspect of these circuits is that they do not adapt to pulse signals, because they display a memory phenomenon. Estimates are given of the magnitude of this effect.


  13. W. Maass and E.D. Sontag. Analog neural nets with Gaussian or other common noise distributions cannot recognize arbitrary regular languages. Neural Comput., 11(3):771-782, 1999. [PDF] [doi:http://dx.doi.org/10.1162/089976699300016656] Keyword(s): neural networks.
    Abstract:
    We consider recurrent analog neural nets where the output of each gate is subject to Gaussian noise, or any other common noise distribution that is nonzero on a large set. We show that many regular languages cannot be recognized by networks of this type, and we give a precise characterization of those languages which can be recognized. This result implies severe constraints on possibilities for constructing recurrent analog neural nets that are robust against realistic types of analog noise. On the other hand we present a method for constructing feedforward analog neural nets that are robust with regard to analog noise of this type.


  14. E.D. Sontag. Feedback stabilization using two-hidden-layer nets. IEEE Trans. Neural Networks, 3:981-990, 1992. [PDF] Keyword(s): neural networks, feedback stabilization.
    Abstract:
    This paper compares the representational capabilities of one hidden layer and two hidden layer nets consisting of feedforward interconnections of linear threshold units. It is remarked that for certain problems two hidden layers are required, contrary to what might be in principle expected from the known approximation theorems. The differences are not based on numerical accuracy or number of units needed, nor on capabilities for feature extraction, but rather on a much more basic classification into "direct" and "inverse" problems. The former correspond to the approximation of continuous functions, while the latter are concerned with approximating one-sided inverses of continuous functions - and are often encountered in the context of inverse kinematics determination or in control questions. A general result is given showing that nonlinear control systems can be stabilized using two hidden layers, but not in general using just one.


  15. E.D. Sontag. Feedforward nets for interpolation and classification. J. Comput. System Sci., 45(1):20-48, 1992. [PDF] [doi:http://dx.doi.org/10.1016/0022-0000(92)90039-L] Keyword(s): neural networks, VC dimension, boolean systems.
    Abstract:
    This paper deals with single-hidden-layer feedforward nets, studying various aspects of classification power and interpolation capability. In particular, a worst-case analysis shows that direct input to output connections in threshold nets double the recognition but not the interpolation power, while using sigmoids rather than thresholds allows doubling both. For other measures of classification, including the Vapnik-Chervonenkis dimension, the effect of direct connections or sigmoidal activations is studied in the special case of two-dimensional inputs.


  16. E.D. Sontag. Capabilities and training of feedforward nets. In Neural networks (New Brunswick, NJ, 1990), pages 303-321. Academic Press, Boston, MA, 1991. [PDF] Keyword(s): neural networks, neural networks.
    Abstract:
    This paper surveys recent work by the author on learning and representational capabilities of feedforward nets. The learning results show that, among two possible variants of the so-called backpropagation training method for sigmoidal nets, both of which variants are used in practice, one is a better generalization of the older perceptron training algorithm than the other. The representation results show that nets consisting of sigmoidal neurons have at least twice the representational capabilities of nets that use classical threshold neurons, at least when this increase is quantified in terms of classification power. On the other hand, threshold nets are shown to be more useful when approximating implicit functions, as illustrated with an application to a typical control problem.


  17. E.D. Sontag and H.J. Sussmann. Back propagation separates where perceptrons do. Neural Networks, 4(2):243-249, 1991. [PDF] [doi:http://dx.doi.org/10.1016/0893-6080(91)90008-S] Keyword(s): neural networks, neural networks.
    Abstract:
    Feedforward nets with sigmoidal activation functions are often designed by minimizing a cost criterion. It has been pointed out before that this technique may be outperformed by the classical perceptron learning rule, at least on some problems. In this paper, we show that no such pathologies can arise if the error criterion is of a threshold LMS type, i.e., is zero for values ``beyond'' the desired target values. More precisely, we show that if the data are linearly separable, and one considers nets with no hidden neurons, then an error function as above cannot have any local minima that are not global. In addition, the proof gives the following stronger result, under the stated hypotheses: the continuous gradient adjustment procedure is such that from any initial weight configuration a separating set of weights is obtained in finite time. This is a precise analogue of the Perceptron Learning Theorem. The results are then compared with the more classical pattern recognition problem of threshold LMS with linear activations, where no spurious local minima exist even for nonseparable data: here it is shown that even if using the threshold criterion, such bad local minima may occur, if the data are not separable and sigmoids are used. keywords = { neural networks , feedforward neural nets },


Conference articles
  1. F. Blanchini, H. El-Samad, G. Giordano, and E. D. Sontag. Control-theoretic methods for biological networks. In Proc. 2018 IEEE Conf. Decision and Control, pages 466-483, 2018. [PDF] Keyword(s): systems biology, dynamic response phenotypes, multi-stability, oscillations, feedback, nonlinear systems, incoherent feedforward loop, feedforward, IFFL.
    Abstract:
    This is a tutorial paper on control-theoretic methods for the analysis of biological systems.


  2. J. Huang, A. Isidori, L. Marconi, M. Mischiati, E. D. Sontag, and W. M. Wonham. Internal models in control, biology and neuroscience. In Proc. 2018 IEEE Conf. Decision and Control, pages 5370-5390, 2018. [PDF] Keyword(s): feeedback, internal model principle, nonlinear systems, incoherent feedforward loop, feedforward, IFFL.
    Abstract:
    This tutorial paper deals with the Internal Model Principle (IMP) from different perspectives. The goal is to start from the principle as introduced and commonly used in the control theory and then enlarge the vision to other fields where "internal models" play a role. The biology and neuroscience fields are specifically targeted in the paper. The paper ends by presenting an "abstract" theory of IMP applicable to a large class of systems.


  3. E.D. Sontag. Some remarks on immune control of infections and tumors. In Proc. IEEE Conf. Decision and Control, Dec. 2016, pages 2476-2480, 2016. [PDF] Keyword(s): scale invariance, fold change detection, T cells, incoherent feedforward loops, immunology, cancer.
    Abstract:
    This is a conference paper related to the journal paper "A dynamical model of immune responses to antigen presentation predicts different regions of tumor or pathogen elimination". The conference paper includes several theorems for a simplified model which were not included in the journal paper.


  4. M. Skataric, E.V. Nikolaev, and E.D. Sontag. Scale-invariance in singularly perturbed systems. In Proc. IEEE Conf. Decision and Control, Los Angeles, Dec. 2014, pages 3035-3040, 2014. [PDF] Keyword(s): singular perturbations, scale invariance, systems biology, transient behavior, symmetries, fcd, fold-change detection, incoherent feedforward loop, feedforward, IFFL.
    Abstract:
    This conference paper (a) summarizes material from "A fundamental limitation to fold-change detection by biological systems with multiple time scales" (IET Systems Biology 2014) and presents additional remarks regarding (b) expansion techniques to compute FCD error and (c) stochastic adaptation and FCD


  5. O. Shoval, U. Alon, and E.D. Sontag. Input symmetry invariance, and applications to biological systems. In Proc. IEEE Conf. Decision and Control, Orlando, Dec. 2011, pages TuA02.5, 2011. Keyword(s): adaptation, feedforward loops, integral feedback, scale invariance, systems biology, transient behavior, symmetries, fcd, fold-change detection, jump Markov processes.
    Abstract:
    This paper studies invariance with respect to symmetries in sensory fields, a particular case of which, scale invariance, has recently been found in certain eukaryotic as well as bacterial cell signaling systems. We describe a necessary and sufficient characterization of symmetry invariance in terms of equivariant transformations, show how this characterization helps find all possible symmetries in standard models of biological adaptation, and discuss symmetry-invariant searches.


  6. T. Natschläger, W. Maass, E.D. Sontag, and A. Zador. Processing of time series by neural circuits with biologically realistic synaptic dynamics. In Todd K. Leen, Thomas G. Dietterich, and Volker Tresp, editors, Advances in Neural Information Processing Systems 13 (NIPS2000), pages 145-151, 2000. MIT Press, Cambridge. [PDF] Keyword(s): neural networks, Volterra series.
    Abstract:
    Experimental data show that biological synapses are dynamic, i.e., their weight changes on a short time scale by several hundred percent in dependence of the past input to the synapse. In this article we explore the consequences that this synaptic dynamics entails for the computational power of feedforward neural networks. It turns out that even with just a single hidden layer such networks can approximate a surprisingly large large class of nonlinear filters: all filters that can be characterized by Volterra series. This result is robust with regard to various changes in the model for synaptic dynamics. Furthermore we show that simple gradient descent suffices to approximate a given quadratic filter by a rather small neural system with dynamic synapses.


Internal reports
  1. E.D. Sontag. A remark on incoherent feedforward circuits as change detectors and feedback controllers. Technical report, arXiv:1602.00162, 2016. [PDF] Keyword(s): scale invariance, fold change detection, T cells, incoherent feedforward loops, immunology, incoherent feedforward loop, feedforward, IFFL.
    Abstract:
    This note analyzes incoherent feedforward loops in signal processing and control. It studies the response properties of IFFL's to exponentially growing inputs, both for a standard version of the IFFL and for a variation in which the output variable has a positive self-feedback term. It also considers a negative feedback configuration, using such a device as a controller. It uncovers a somewhat surprising phenomenon in which stabilization is only possible in disconnected regions of parameter space, as the controlled system's growth rate is varied.


  2. E.D. Sontag. Examples of computation of exact moment dynamics for chemical reaction networks. Technical report, arXiv:1612.02393, 2016. [PDF] Keyword(s): systems biology, biochemical networks, stochastic systems, Chemical Master Equation, chemical reaction networks, moments, molecular networks, complex-balanced networks.
    Abstract:
    We review in a unified way results for two types of stochastic chemical reaction systems for which moments can be effectively computed: feedforward networks and complex-balanced networks.


  3. E.D. Sontag. Two-zone tumor tolerance can arise from a simple immunological feedforward motif that estimates tumor growth rates. Technical report, bioRxiv https://doi.org/10.1101/095455, 2016. [PDF] Keyword(s): scale invariance, fold change detection, T cells, incoherent feedforward loops, immunology, cancer.
    Abstract:
    Preprint version of "A dynamical model of immune responses to antigen presentation predicts different regions of tumor or pathogen elimination", appeared in Cell Systems 2017. However, the journal version does not include Section 9 on degradation-based IFFL's from this preprint.


  4. E.D. Sontag. Incoherent feedforward motifs as immune change detectors. Technical report, bioRxiv http://dx.doi.org/10.1101/035600, December 2015. [PDF] Keyword(s): scale invariance, fcd, fold change detection, T cells, incoherent feedforward loops, immunology, incoherent feedforward loop, feedforward, IFFL.
    Abstract:
    We speculate that incoherent feedforward loops may be phenomenologically involved in self/nonself discrimination in immune-infection and immune-tumor interactions, acting as "change detectors". In turn, this may result in logarithmic sensing (Weber phenomenon) and even scale invariance (fold-change detection).



BACK TO INDEX




Disclaimer:

This material is presented to ensure timely dissemination of scholarly and technical work. Copyright and all rights therein are retained by authors or by other copyright holders.




Last modified: Thu Apr 11 22:06:20 2019
Author: sontag.


This document was translated from BibTEX by bibtex2html