IDEAS home Printed from https://ideas.repec.org/a/plo/pone00/0192944.html
   My bibliography  Save this article

Optimization by Adaptive Stochastic Descent

Author

Listed:
  • Cliff C Kerr
  • Salvador Dura-Bernal
  • Tomasz G Smolinski
  • George L Chadderdon
  • David P Wilson

Abstract

When standard optimization methods fail to find a satisfactory solution for a parameter fitting problem, a tempting recourse is to adjust parameters manually. While tedious, this approach can be surprisingly powerful in terms of achieving optimal or near-optimal solutions. This paper outlines an optimization algorithm, Adaptive Stochastic Descent (ASD), that has been designed to replicate the essential aspects of manual parameter fitting in an automated way. Specifically, ASD uses simple principles to form probabilistic assumptions about (a) which parameters have the greatest effect on the objective function, and (b) optimal step sizes for each parameter. We show that for a certain class of optimization problems (namely, those with a moderate to large number of scalar parameter dimensions, especially if some dimensions are more important than others), ASD is capable of minimizing the objective function with far fewer function evaluations than classic optimization methods, such as the Nelder-Mead nonlinear simplex, Levenberg-Marquardt gradient descent, simulated annealing, and genetic algorithms. As a case study, we show that ASD outperforms standard algorithms when used to determine how resources should be allocated in order to minimize new HIV infections in Swaziland.

Suggested Citation

  • Cliff C Kerr & Salvador Dura-Bernal & Tomasz G Smolinski & George L Chadderdon & David P Wilson, 2018. "Optimization by Adaptive Stochastic Descent," PLOS ONE, Public Library of Science, vol. 13(3), pages 1-16, March.
  • Handle: RePEc:plo:pone00:0192944
    DOI: 10.1371/journal.pone.0192944
    as

    Download full text from publisher

    File URL: https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0192944
    Download Restriction: no

    File URL: https://journals.plos.org/plosone/article/file?id=10.1371/journal.pone.0192944&type=printable
    Download Restriction: no

    File URL: https://libkey.io/10.1371/journal.pone.0192944?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Andrew J Shattock & Clemens Benedikt & Aliya Bokazhanova & Predrag Đurić & Irina Petrenko & Lolita Ganina & Sherrie L Kelly & Robyn M Stuart & Cliff C Kerr & Tatiana Vinichenko & Shufang Zhang & Chris, 2017. "Kazakhstan can achieve ambitious HIV targets despite expected donor withdrawal by combining improved ART procurement mechanisms with allocative and implementation efficiencies," PLOS ONE, Public Library of Science, vol. 12(2), pages 1-15, February.
    2. Luis Rios & Nikolaos Sahinidis, 2013. "Derivative-free optimization: a review of algorithms and comparison of software implementations," Journal of Global Optimization, Springer, vol. 56(3), pages 1247-1293, July.
    3. Gary Charness & Edi Karni & Dan Levin, 2007. "Individual and group decision making under risk: An experimental study of Bayesian updating and violations of first-order stochastic dominance," Journal of Risk and Uncertainty, Springer, vol. 35(2), pages 129-148, October.
    4. Peter W. Glynn & Ward Whitt, 1992. "The Asymptotic Efficiency of Simulation Estimators," Operations Research, INFORMS, vol. 40(3), pages 505-520, June.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Jonas Bjerg Thomsen & Francesco Ferri & Jens Peter Kofoed & Kevin Black, 2018. "Cost Optimization of Mooring Solutions for Large Floating Wave Energy Converters," Energies, MDPI, vol. 11(1), pages 1-23, January.
    2. David J. Cooper & Krista Saral & Marie Claire Villeval, 2021. "Why Join a Team?," Management Science, INFORMS, vol. 67(11), pages 6980-6997, November.
    3. Faralla, Valeria & Borà, Guido & Innocenti, Alessandro & Novarese, Marco, 2020. "Promises in group decision making," Research in Economics, Elsevier, vol. 74(1), pages 1-11.
    4. W. Kip Viscusi & Scott DeAngelis, 2018. "Decision irrationalities involving deadly risks," Journal of Risk and Uncertainty, Springer, vol. 57(3), pages 225-252, December.
    5. Besedes, Tibor & Deck, Cary & Quintanar, Sarah & Sarangi, Sudipta & Shor, Mikhael, 2011. "Free-Riding and Performance in Collaborative and Non-Collaborative Groups," MPRA Paper 33948, University Library of Munich, Germany.
    6. Giesecke, K. & Schwenkler, G., 2019. "Simulated likelihood estimators for discretely observed jump–diffusions," Journal of Econometrics, Elsevier, vol. 213(2), pages 297-320.
    7. Stöckl, Thomas & Huber, Jürgen & Kirchler, Michael & Lindner, Florian, 2015. "Hot hand and gambler's fallacy in teams: Evidence from investment experiments," Journal of Economic Behavior & Organization, Elsevier, vol. 117(C), pages 327-339.
    8. Gabriela Simonet & Julie Subervie & Driss Ezzine-De-Blas & Marina Cromberg & Amy Duchelle, 2015. "Paying smallholders not to cut down the amazon forest: impact evaluation of a REDD+ pilot project," Working Papers 1514, Chaire Economie du climat.
    9. Gantner, Anita & Horn, Kristian & Kerschbamer, Rudolf, 2019. "The role of communication in fair division with subjective claims," Journal of Economic Behavior & Organization, Elsevier, vol. 167(C), pages 72-89.
    10. Somayeh Moazeni & Warren B. Powell & Boris Defourny & Belgacem Bouzaiene-Ayari, 2017. "Parallel Nonstationary Direct Policy Search for Risk-Averse Stochastic Optimization," INFORMS Journal on Computing, INFORMS, vol. 29(2), pages 332-349, May.
    11. Stephen Cheung & Stefan Palan, 2012. "Two heads are less bubbly than one: team decision-making in an experimental asset market," Experimental Economics, Springer;Economic Science Association, vol. 15(3), pages 373-397, September.
    12. Thomas Kourouxous & Thomas Bauer, 2019. "Violations of dominance in decision-making," Business Research, Springer;German Academic Association for Business Research, vol. 12(1), pages 209-239, April.
    13. Guay, François & Schwenkler, Gustavo, 2021. "Efficient estimation and filtering for multivariate jump–diffusions," Journal of Econometrics, Elsevier, vol. 223(1), pages 251-275.
    14. Jakubik, Johannes & Binding, Adrian & Feuerriegel, Stefan, 2021. "Directed particle swarm optimization with Gaussian-process-based function forecasting," European Journal of Operational Research, Elsevier, vol. 295(1), pages 157-169.
    15. Francesco Feri & Bernd Irlenbusch & Matthias Sutter, 2010. "Efficiency Gains from Team-Based Coordination—Large-Scale Experimental Evidence," American Economic Review, American Economic Association, vol. 100(4), pages 1892-1912, September.
    16. Crosetto, Paolo & Filippin, Antonio & Katuščák, Peter & Smith, John, 2020. "Central tendency bias in belief elicitation," Journal of Economic Psychology, Elsevier, vol. 78(C).
    17. Michael B. Gordy & Sandeep Juneja, 2010. "Nested Simulation in Portfolio Risk Measurement," Management Science, INFORMS, vol. 56(10), pages 1833-1848, October.
    18. Christophe Gouel & Nicolas Legrand, 2017. "Estimating the Competitive Storage Model with Trending Commodity Prices," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 32(4), pages 744-763, June.
    19. Zhao, Jake, 2020. "Accounting for the corporate cash increase," European Economic Review, Elsevier, vol. 123(C).
    20. Hannes Schwarz & Valentin Bertsch & Wolf Fichtner, 2018. "Two-stage stochastic, large-scale optimization of a decentralized energy system: a case study focusing on solar PV, heat pumps and storage in a residential quarter," OR Spectrum: Quantitative Approaches in Management, Springer;Gesellschaft für Operations Research e.V., vol. 40(1), pages 265-310, January.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:plo:pone00:0192944. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: plosone (email available below). General contact details of provider: https://journals.plos.org/plosone/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.