IDEAS home Printed from https://ideas.repec.org/a/kap/transp/v51y2024i6d10.1007_s11116-023-10423-y.html
   My bibliography  Save this article

Size matters: the use and misuse of statistical significance in discrete choice models in the transportation academic literature

Author

Listed:
  • Giancarlos Parady

    (The University of Tokyo)

  • Kay W. Axhausen

    (IVT, ETH Zürich)

Abstract

In this paper we review the academic transportation literature published between 2014 and 2018 to evaluate where the field stands regarding the use and misuse of statistical significance in empirical analysis, with a focus on discrete choice models. Our results show that 39% of studies explained model results exclusively based on the sign of the coefficient, 67% of studies did not distinguish statistical significance from economic, policy or scientific significance in their conclusions, and none of the reviewed studies considered the statistical power of the tests. Based on these results we put forth a set of recommendations aimed at shifting the focus away from statistical significance towards proper and comprehensive assessment of effect magnitudes and other policy relevant quantities.

Suggested Citation

  • Giancarlos Parady & Kay W. Axhausen, 2024. "Size matters: the use and misuse of statistical significance in discrete choice models in the transportation academic literature," Transportation, Springer, vol. 51(6), pages 2393-2425, December.
    • Handle: RePEc:kap:transp:v:51:y:2024:i:6:d:10.1007_s11116-023-10423-y
    DOI: 10.1007/s11116-023-10423-y
    as

    Download full text from publisher

    File URL: http://link.springer.com/10.1007/s11116-023-10423-y
    File Function: Abstract
    Download Restriction: Access to full text is restricted to subscribers.
    File URL: https://libkey.io/10.1007/s11116-023-10423-y?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Deirdre N. McCloskey & Stephen T. Ziliak, 1996. "The Standard Error of Regressions," Journal of Economic Literature, American Economic Association, vol. 34(1), pages 97-114, March.
    2. Hensher, David A. & Ho, Chinh Q., 2016. "Experience conditioning in commuter modal choice modelling – Does it make a difference?," Transportation Research Part E: Logistics and Transportation Review, Elsevier, vol. 95(C), pages 164-176.
    3. Ho, Chinh Q. & Hensher, David A., 2016. "A workplace choice model accounting for spatial competition and agglomeration effects," Journal of Transport Geography, Elsevier, vol. 51(C), pages 193-203.
    4. Milad Mehdizadeh & Trond Nordfjaern & AmirReza Mamdoohi, 2018. "The role of socio-economic, built environment and psychological factors in parental mode choice for their children in an Iranian setting," Transportation, Springer, vol. 45(2), pages 523-543, March.
    5. Raj Bridgelall, 2014. "Campus parking supply impacts on transportation mode choice," Transportation Planning and Technology, Taylor & Francis Journals, vol. 37(8), pages 711-737, December.
    6. Efthymiou, Dimitrios & Antoniou, Constantinos, 2017. "Understanding the effects of economic crisis on public transport users’ satisfaction and demand," Transport Policy, Elsevier, vol. 53(C), pages 89-97.
    7. Hasnine, Md Sami & Habib, Khandker Nurul, 2018. "What about the dynamics in daily travel mode choices? A dynamic discrete choice approach for tour-based mode choice modelling," Transport Policy, Elsevier, vol. 71(C), pages 70-80.
    8. Collins, Patricia A. & MacFarlane, Robert, 2018. "Evaluating the determinants of switching to public transit in an automobile-oriented mid-sized Canadian city: A longitudinal analysis," Transportation Research Part A: Policy and Practice, Elsevier, vol. 118(C), pages 682-695.
    9. Habib, Khandker M. Nurul & Sasic, Ana, 2014. "A GEV model with scale heterogeneity for investigating the role of mobility tool ownership in peak period non-work travel mode choices," Journal of choice modelling, Elsevier, vol. 10(C), pages 46-59.
    10. Rahman, Mohammad Lutfur & Baker, Douglas, 2018. "Modelling induced mode switch behaviour in Bangladesh: A multinomial logistic regression approach," Transport Policy, Elsevier, vol. 71(C), pages 81-91.
    11. Gerber, Philippe & Ma, Tai-Yu & Klein, Olivier & Schiebel, Julien & Carpentier-Postel, Samuel, 2017. "Cross-border residential mobility, quality of life and modal shift: A Luxembourg case study," Transportation Research Part A: Policy and Practice, Elsevier, vol. 104(C), pages 238-254.
    12. Esther Bekker-Grob & Bas Donkers & Marcel Jonker & Elly Stolk, 2015. "Sample Size Requirements for Discrete-Choice Experiments in Healthcare: a Practical Guide," The Patient: Patient-Centered Outcomes Research, Springer;International Academy of Health Preference Research, vol. 8(5), pages 373-384, October.
    13. Liu, Yang & Ji, Yanjie & Shi, Zhuangbin & He, Baohong & Liu, Qiyang, 2018. "Investigating the effect of the spatial relationship between home, workplace and school on parental chauffeurs’ daily travel mode choice," Transport Policy, Elsevier, vol. 69(C), pages 78-87.
    14. Allard, Ryan F. & Moura, Filipe, 2018. "Effect of transport transfer quality on intercity passenger mode choice," Transportation Research Part A: Policy and Practice, Elsevier, vol. 109(C), pages 89-107.
    15. Marcel Paulssen & Dirk Temme & Akshay Vij & Joan Walker, 2014. "Values, attitudes and travel behavior: a hierarchical latent variable mixed logit model of travel mode choice," Transportation, Springer, vol. 41(4), pages 873-888, July.
    16. Bhat, Chandra R. & Dubey, Subodh K. & Nagel, Kai, 2015. "Introducing non-normality of latent psychological constructs in choice modeling with an application to bicyclist route choice," Transportation Research Part B: Methodological, Elsevier, vol. 78(C), pages 341-363.
    17. John Rose & David Hensher, 2014. "Demand for taxi services: new elasticity evidence," Transportation, Springer, vol. 41(4), pages 717-743, July.
    18. Parady, Giancarlos & Ory, David & Walker, Joan, 2021. "The overreliance on statistical goodness-of-fit and under-reliance on model validation in discrete choice models: A review of validation practices in the transportation academic literature," Journal of choice modelling, Elsevier, vol. 38(C).
    19. Rotaris, Lucia & Danielis, Romeo, 2015. "Commuting to college: The effectiveness and social efficiency of transportation demand management policies," Transport Policy, Elsevier, vol. 44(C), pages 158-168.
    20. Di Ciommo, Floridea & Comendador, Julio & López-Lambas, María Eugenia & Cherchi, Elisabetta & Ortúzar, Juan de Dios, 2014. "Exploring the role of social capital influence variables on travel behaviour," Transportation Research Part A: Policy and Practice, Elsevier, vol. 68(C), pages 46-55.
    21. Rose, John M. & Bliemer, Michiel C.J. & Hensher, David A. & Collins, Andrew T., 2008. "Designing efficient stated choice experiments in the presence of reference alternatives," Transportation Research Part B: Methodological, Elsevier, vol. 42(4), pages 395-406, May.
    22. Hsin-Ping Hsu & Jean-Daniel Saphores, 2014. "Impacts of parental gender and attitudes on children’s school travel mode and parental chauffeuring behavior: results for California based on the 2009 National Household Travel Survey," Transportation, Springer, vol. 41(3), pages 543-565, May.
    23. Khandker Habib, 2014. "Household-level commuting mode choices, car allocation and car ownership level choices of two-worker households: the case of the city of Toronto," Transportation, Springer, vol. 41(3), pages 651-672, May.
    24. Habib, Khandker M. Nurul & Swait, Joffre & Salem, Sarah, 2014. "Using repeated cross-sectional travel surveys to enhance forecasting robustness: Accounting for changing mode preferences," Transportation Research Part A: Policy and Practice, Elsevier, vol. 67(C), pages 110-126.
    25. Kristoffersson, Ida & Daly, Andrew & Algers, Staffan, 2018. "Modelling the attraction of travel to shopping destinations in large-scale modelling," Transport Policy, Elsevier, vol. 68(C), pages 52-62.
    26. Mohammad Ali Arman & Navid Khademi & Matthieu de Lapparent, 2018. "Women’s mode and trip structure choices in daily activity-travel: a developing country perspective," Transportation Planning and Technology, Taylor & Francis Journals, vol. 41(8), pages 845-877, November.
    27. Hess, Stephane & Spitz, Greg & Bradley, Mark & Coogan, Matt, 2018. "Analysis of mode choice for intercity travel: Application of a hybrid choice model to two distinct US corridors," Transportation Research Part A: Policy and Practice, Elsevier, vol. 116(C), pages 547-567.
    28. Vij, Akshay & Gorripaty, Sreeta & Walker, Joan L., 2017. "From trend spotting to trend ’splaining: Understanding modal preference shifts in the San Francisco Bay Area," Transportation Research Part A: Policy and Practice, Elsevier, vol. 95(C), pages 238-258.
    29. Ermagun, Alireza & Samimi, Amir, 2015. "Promoting active transportation modes in school trips," Transport Policy, Elsevier, vol. 37(C), pages 203-211.
    30. Sylvia Y. He & Genevieve Giuliano, 2017. "Factors affecting children’s journeys to school: a joint escort-mode choice model," Transportation, Springer, vol. 44(1), pages 199-224, January.
    31. Khan, Mobashwir & M. Kockelman, Kara & Xiong, Xiaoxia, 2014. "Models for anticipating non-motorized travel choices, and the role of the built environment," Transport Policy, Elsevier, vol. 35(C), pages 117-126.
    32. John Rose & Michiel Bliemer, 2013. "Sample size requirements for stated choice experiments," Transportation, Springer, vol. 40(5), pages 1021-1041, September.
    33. Fernández-Antolín, Anna & Guevara, C. Angelo & de Lapparent, Matthieu & Bierlaire, Michel, 2016. "Correcting for endogeneity due to omitted attitudes: Empirical assessment of a modified MIS method using RP mode choice data," Journal of choice modelling, Elsevier, vol. 20(C), pages 1-15.
    34. Tilahun, Nebiyou & Thakuriah, Piyushimita (Vonu) & Li, Moyin & Keita, Yaye, 2016. "Transit use and the work commute: Analyzing the role of last mile issues," Journal of Transport Geography, Elsevier, vol. 54(C), pages 359-368.
    35. Andrew Clark & Darren Scott & Nikolaos Yiannakoulias, 2014. "Examining the relationship between active travel, weather, and the built environment: a multilevel approach using a GPS-enhanced dataset," Transportation, Springer, vol. 41(2), pages 325-338, March.
    36. Maria Kamargianni & Moshe Ben-Akiva & Amalia Polydoropoulou, 2014. "Incorporating social interaction into hybrid choice models," Transportation, Springer, vol. 41(6), pages 1263-1285, November.
    37. Huanmei Qin & Jianqiang Gao & Hongzhi Guan & Hongbo Chi, 2017. "Estimating heterogeneity of car travelers on mode shifting behavior based on discrete choice models," Transportation Planning and Technology, Taylor & Francis Journals, vol. 40(8), pages 914-927, November.
    38. Zaidan, Esmat & Abulibdeh, Ammar, 2018. "Modeling ground access mode choice behavior for Hamad International Airport in the 2022 FIFA World Cup city, Doha, Qatar," Journal of Air Transport Management, Elsevier, vol. 73(C), pages 32-45.
    39. Liu, Chengxi & Susilo, Yusak O. & Karlström, Anders, 2015. "The influence of weather characteristics variability on individual’s travel mode choice in different seasons and regions in Sweden," Transport Policy, Elsevier, vol. 41(C), pages 147-158.
    40. Stone, Michelle & Larsen, Kristian & Faulkner, Guy E.J. & Buliung, Ron N. & Arbour-Nicitopoulos, Kelly P. & Lay, Jennifer, 2014. "Predictors of driving among families living within 2km from school: Exploring the role of the built environment," Transport Policy, Elsevier, vol. 33(C), pages 8-16.
    41. Javier Anta & José B. Pérez-López & Ana Martínez-Pardo & Margarita Novales & Alfonso Orro, 2016. "Influence of the weather on mode choice in corridors with time-varying congestion: a mixed data study," Transportation, Springer, vol. 43(2), pages 337-355, March.
    42. Motoaki, Yutaka & Daziano, Ricardo A., 2015. "A hybrid-choice latent-class model for the analysis of the effects of weather on cycling demand," Transportation Research Part A: Policy and Practice, Elsevier, vol. 75(C), pages 217-230.
    43. Manoj, M. & Verma, Ashish, 2015. "Activity-travel behaviour of non-workers belonging to different income group households in Bangalore, India," Journal of Transport Geography, Elsevier, vol. 49(C), pages 99-109.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Gustavo García-Melero & Rubén Sainz-González & Pablo Coto-Millán & Alejandra Valencia-Vásquez, 2021. "Sustainable Mobility Policy Analysis Using Hybrid Choice Models: Is It the Right Choice?," Sustainability, MDPI, vol. 13(5), pages 1-16, March.
    2. Kim, Seheon & Rasouli, Soora, 2022. "The influence of latent lifestyle on acceptance of Mobility-as-a-Service (MaaS): A hierarchical latent variable and latent class approach," Transportation Research Part A: Policy and Practice, Elsevier, vol. 159(C), pages 304-319.
    3. Saad AlQuhtani, 2023. "Factors Affecting Active Commuting to School in Sprawled Cities: The Case of Najran City, Saudi Arabia," Sustainability, MDPI, vol. 15(12), pages 1-17, June.
    4. Vij, Akshay & Walker, Joan L., 2016. "How, when and why integrated choice and latent variable models are latently useful," Transportation Research Part B: Methodological, Elsevier, vol. 90(C), pages 192-217.
    5. Saeed Esmaeli & Kayvan Aghabayk & Nirajan Shiwakoti, 2024. "Measuring the Effect of Built Environment on Students’ School Trip Method Using Neighborhood Environment Walkability Scale," Sustainability, MDPI, vol. 16(5), pages 1-15, February.
    6. Bouscasse, H., 2018. "Integrated choice and latent variable models: A literature review on mode choice," Working Papers 2018-07, Grenoble Applied Economics Laboratory (GAEL).
    7. Kim, Sung Hoo & Mokhtarian, Patricia L., 2023. "Finite mixture (or latent class) modeling in transportation: Trends, usage, potential, and future directions," Transportation Research Part B: Methodological, Elsevier, vol. 172(C), pages 134-173.
    8. Liu, Yang & Min, Shisheng & Shi, Zhuangbin & He, Mingwei, 2024. "Exploring students' choice of active travel to school in different spatial environments: A case study in a mountain city," Journal of Transport Geography, Elsevier, vol. 115(C).
    9. Scheiner, Joachim & Huber, Oliver & Lohmüller, Stefan, 2019. "Children's independent travel to and from primary school: Evidence from a suburban town in Germany," Transportation Research Part A: Policy and Practice, Elsevier, vol. 120(C), pages 116-131.
    10. Chinedu, Obi & Sanou, Edouard & Tur-Cardona, Juan & Bartolini, Fabio & Gheysen, Godelieve & Speelman, Stijn, 2018. "Farmers’ valuation of transgenic biofortified sorghum for nutritional improvement in Burkina Faso: A latent class approach," Food Policy, Elsevier, vol. 79(C), pages 132-140.
    11. John Buckell & David A Hensher & Stephane Hess, 2021. "Kicking the habit is hard: A hybrid choice model investigation into the role of addiction in smoking behavior," Health Economics, John Wiley & Sons, Ltd., vol. 30(1), pages 3-19, January.
    12. Antonio Borriello & John M. Rose, 2021. "Global versus localised attitudinal responses in discrete choice," Transportation, Springer, vol. 48(1), pages 131-165, February.
    13. Hélène Bouscasse, 2018. "Integrated choice and latent variable models: A literature review on mode choice," Working Papers hal-01795630, HAL.
    14. Rahman, Moshiur & Yasmin, Shamsunnahar & Eluru, Naveen, 2019. "Controlling for endogeneity between bus headway and bus ridership: A case study of the Orlando region," Transport Policy, Elsevier, vol. 81(C), pages 208-219.
    15. Thorhauge, Mikkel & Swait, Joffre & Cherchi, Elisabetta, 2020. "The habit-driven life: Accounting for inertia in departure time choices for commuting trips," Transportation Research Part A: Policy and Practice, Elsevier, vol. 133(C), pages 272-289.
    16. Marley, A.A.J. & Swait, J., 2017. "Goal-based models for discrete choice analysis," Transportation Research Part B: Methodological, Elsevier, vol. 101(C), pages 72-88.
    17. Lehmann, Nico & Sloot, Daniel & Ardone, Armin & Fichtner, Wolf, 2022. "Willingness to pay for regional electricity generation – A question of green values and regional product beliefs?," Energy Economics, Elsevier, vol. 110(C).
    18. Nabipour, Mohammad & Rosenberg, Mark W. & Nasseri, Seyed Hadi, 2022. "The built environment, networks design, and safety features: An analysis of pedestrian commuting behavior in intermediate-sized cities," Transport Policy, Elsevier, vol. 129(C), pages 14-23.
    19. Zhao, Chunli & Xue, Mengtian & Hamidi, Zahra, 2024. "Potential of demand responsive transport for young people in Sweden," Transportation Research Part A: Policy and Practice, Elsevier, vol. 184(C).
    20. Piras, Francesco & Sottile, Eleonora & Tuveri, Giovanni & Meloni, Italo, 2021. "Could psychosocial variables help assess pro-cycling policies?," Transportation Research Part A: Policy and Practice, Elsevier, vol. 154(C), pages 108-128.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:kap:transp:v:51:y:2024:i:6:d:10.1007_s11116-023-10423-y. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.