IDEAS home Printed from https://ideas.repec.org/a/gam/jftint/v11y2018i1p7-d193662.html
   My bibliography  Save this article

Layer-Wise Compressive Training for Convolutional Neural Networks

Author

Listed:
  • Matteo Grimaldi

    (Department of Control and Computer Engineering, Politecnico di Torino, Turin 10129, Italy)

  • Valerio Tenace

    (Department of Control and Computer Engineering, Politecnico di Torino, Turin 10129, Italy)

  • Andrea Calimera

    (Department of Control and Computer Engineering, Politecnico di Torino, Turin 10129, Italy)

Abstract

Convolutional Neural Networks (CNNs) are brain-inspired computational models designed to recognize patterns. Recent advances demonstrate that CNNs are able to achieve, and often exceed, human capabilities in many application domains. Made of several millions of parameters, even the simplest CNN shows large model size. This characteristic is a serious concern for the deployment on resource-constrained embedded-systems, where compression stages are needed to meet the stringent hardware constraints. In this paper, we introduce a novel accuracy-driven compressive training algorithm. It consists of a two-stage flow: first, layers are sorted by means of heuristic rules according to their significance; second, a modified stochastic gradient descent optimization is applied on less significant layers such that their representation is collapsed into a constrained subspace. Experimental results demonstrate that our approach achieves remarkable compression rates with low accuracy loss (<1%).

Suggested Citation

  • Matteo Grimaldi & Valerio Tenace & Andrea Calimera, 2018. "Layer-Wise Compressive Training for Convolutional Neural Networks," Future Internet, MDPI, vol. 11(1), pages 1-15, December.
  • Handle: RePEc:gam:jftint:v:11:y:2018:i:1:p:7-:d:193662
    as

    Download full text from publisher

    File URL: https://www.mdpi.com/1999-5903/11/1/7/pdf
    Download Restriction: no

    File URL: https://www.mdpi.com/1999-5903/11/1/7/
    Download Restriction: no
    ---><---

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. FabĂ­ola Martins Campos de Oliveira & Edson Borin, 2019. "Partitioning Convolutional Neural Networks to Maximize the Inference Rate on Constrained IoT Devices," Future Internet, MDPI, vol. 11(10), pages 1-30, September.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:gam:jftint:v:11:y:2018:i:1:p:7-:d:193662. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    We have no bibliographic references for this item. You can help adding them by using this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: MDPI Indexing Manager (email available below). General contact details of provider: https://www.mdpi.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.