IDEAS home Printed from https://ideas.repec.org/a/eee/epplan/v68y2018icp81-89.html
   My bibliography  Save this article

A counterfactual impact evaluation of a bilingual program on students’ grade point average at a spanish university

Author

Listed:
  • Arco-Tirado, J.L.
  • Fernández-Martín, F.
  • Ramos-García, A.M.
  • Littvay, L.
  • Villoria, J.
  • Naranjo, J.A.

Abstract

This observational study intends to estimate the causal effects of an English as a Medium of Instruction (EMI) program (as predictor) on students Grade Point Average (GPA) (as outcome) at a particular University in Spain by using a Counterfactual Impact Evaluation (CIE). The need to address the crucial question of causal inferences in EMI programs to produce credible evidences of successful interventions contrasts, however, with the absence of experimental or quasi-experimental research and evaluation designs in the field. CIE approach is emerging as a methodologically viable solution to bridge that gap. The program evaluated here consisted in delivering an EMI program in a Primary Education Teacher Training Degree group. After achieving balance on the observed covariates and recreating a situation that would have been expected in a randomized experiment, three matching approaches such as genetic matching, nearest neighbor matching and Coarsened Exact Matching were used to analyze observational data from a total of 1288 undergraduate students, including both treatment and control group. Results show unfavorable effects of the bilingual group treatment condition. Potential interpretations and recommendations are provided in order to strengthen future causal evidences of bilingual education programs’ effectiveness in Higher Education.

Suggested Citation

  • Arco-Tirado, J.L. & Fernández-Martín, F. & Ramos-García, A.M. & Littvay, L. & Villoria, J. & Naranjo, J.A., 2018. "A counterfactual impact evaluation of a bilingual program on students’ grade point average at a spanish university," Evaluation and Program Planning, Elsevier, vol. 68(C), pages 81-89.
    • Handle: RePEc:eee:epplan:v:68:y:2018:i:c:p:81-89
    DOI: 10.1016/j.evalprogplan.2018.02.013
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S014971891730335X
    Download Restriction: Full text for ScienceDirect subscribers only
    File URL: https://libkey.io/10.1016/j.evalprogplan.2018.02.013?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. repec:idb:brikps:58518 is not listed on IDEAS
    2. Brindusa Anghel & Antonio Cabrales & Jesus M. Carro, 2016. "Evaluating A Bilingual Education Program In Spain: The Impact Beyond Foreign Language Learning," Economic Inquiry, Western Economic Association International, vol. 54(2), pages 1202-1223, April.
    3. Marco Caliendo & Sabine Kopeinig, 2008. "Some Practical Guidance For The Implementation Of Propensity Score Matching," Journal of Economic Surveys, Wiley Blackwell, vol. 22(1), pages 31-72, February.
    4. Vincent Tinto, 1997. "Classrooms as Communities," The Journal of Higher Education, Taylor & Francis Journals, vol. 68(6), pages 599-623, November.
    5. Ho, Daniel E. & Imai, Kosuke & King, Gary & Stuart, Elizabeth A., 2007. "Matching as Nonparametric Preprocessing for Reducing Model Dependence in Parametric Causal Inference," Political Analysis, Cambridge University Press, vol. 15(3), pages 199-236, July.
    6. David E. Card & Pablo Ibarraran & Juan Miguel Villa, 2011. "Building in an Evaluation Component for Active Labor Market Programs: A Practitioner's Guide," SPD Working Papers 1101, Inter-American Development Bank, Office of Strategic Planning and Development Effectiveness (SPD).
    7. Iacus, Stefano M. & King, Gary & Porro, Giuseppe, 2012. "Causal Inference without Balance Checking: Coarsened Exact Matching," Political Analysis, Cambridge University Press, vol. 20(1), pages 1-24, January.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Teng Lin & Jun Lei, 2021. "English-Medium Instruction and Content Learning in Higher Education: Effects of Medium of Instruction, English Proficiency, and Academic Ability," SAGE Open, , vol. 11(4), pages 21582440211, November.
    2. Lucia Švábová & Katarína Kramárová & Marek Ďurica, 2021. "Evaluation of the Effects of the Graduate Practice in Slovakia: Comparison of Results of Counterfactual Methods," Central European Business Review, Prague University of Economics and Business, vol. 2021(4), pages 1-31.
    3. Pelucha, Martin & Kveton, Viktor & Potluka, Oto, 2019. "Using mixed method approach in measuring effects of training in firms: Case study of the European Social Fund support," Evaluation and Program Planning, Elsevier, vol. 73(C), pages 146-155.
    4. Lucia Svabova & Eva Nahalkova Tesarova & Marek Durica & Lenka Strakova, 2021. "Evaluation of the impacts of the COVID-19 pandemic on the development of the unemployment rate in Slovakia: counterfactual before-after comparison," Equilibrium. Quarterly Journal of Economics and Economic Policy, Institute of Economic Research, vol. 16(2), pages 261-284, June.
    5. del Campo, Cristina & Urquía-Grande, Elena & Pascual-Ezama, David, 2023. "Internationalizing the business school: A comparative analysis of English-medium and Spanish-medium instruction impact on student performance," Evaluation and Program Planning, Elsevier, vol. 98(C).

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Gerhard Krug, 2017. "Augmenting propensity score equations to avoid misspecification bias – Evidence from a Monte Carlo simulation [Erweiterung der Propensity Score Gleichung zur Vermeidung von Fehlspezifikationen? Ein," AStA Wirtschafts- und Sozialstatistisches Archiv, Springer;Deutsche Statistische Gesellschaft - German Statistical Society, vol. 11(3), pages 205-231, December.
    2. Gary King & Christopher Lucas & Richard A. Nielsen, 2017. "The Balance‐Sample Size Frontier in Matching Methods for Causal Inference," American Journal of Political Science, John Wiley & Sons, vol. 61(2), pages 473-489, April.
    3. Jones, A.M.; & Pastore, C.; & Rice, N.;, 2018. "Tracking pupils into adulthood: selective schools and long-term well-being in the 1958 British cohort," Health, Econometrics and Data Group (HEDG) Working Papers 18/32, HEDG, c/o Department of Economics, University of York.
    4. Adeola Oyenubi & Martin Wittenberg, 2021. "Does the choice of balance-measure matter under genetic matching?," Empirical Economics, Springer, vol. 61(1), pages 489-502, July.
    5. Dudel Christian & Garbuszus Jan Marvin & Ott Notburga & Werding Martin, 2017. "Matching as Non-Parametric Preprocessing for the Estimation of Equivalence Scales," Journal of Economics and Statistics (Jahrbuecher fuer Nationaloekonomie und Statistik), De Gruyter, vol. 237(2), pages 115-141, April.
    6. Jian Jiu Chen & Sai Yin Ho & Wing Man Au & Man Ping Wang & Tai Hing Lam, 2015. "Family Smoking, Exposure to Secondhand Smoke at Home and Family Unhappiness in Children," IJERPH, MDPI, vol. 12(11), pages 1-14, November.
    7. Kube, Roland & von Graevenitz, Kathrine & Löschel, Andreas & Massier, Philipp, 2019. "Do voluntary environmental programs reduce emissions? EMAS in the German manufacturing sector," Energy Economics, Elsevier, vol. 84(S1).
    8. K. Poehlmann & R. Helm & O. Mauroner & J. Auburger, 2021. "Corporate spin-offs’ success factors: management lessons from a comparative empirical analysis with research-based spin-offs," Review of Managerial Science, Springer, vol. 15(6), pages 1767-1796, August.
    9. Bensch, Gunther & Peters, Jörg & Sievert, Maximiliane, 2012. "Fear of the Dark? – How Access to Electric Lighting Affects Security Attitudes and Nighttime Activities in Rural Senegal," Ruhr Economic Papers 369, RWI - Leibniz-Institut für Wirtschaftsforschung, Ruhr-University Bochum, TU Dortmund University, University of Duisburg-Essen.
    10. Robert L. Obenchain & S. Stanley Young, 2017. "Local Control Strategy: Simple Analyses of Air Pollution Data Can Reveal Heterogeneity in Longevity Outcomes," Risk Analysis, John Wiley & Sons, vol. 37(9), pages 1742-1753, September.
    11. Augusto Cerqua & Guido Pellegrini, 2013. "Beyond the SUTVA: how industrial policy evaluations change when we allow for interaction among firms," ERSA conference papers ersa13p340, European Regional Science Association.
    12. Petrov, Ivan & Ryan, Lisa, 2021. "The landlord-tenant problem and energy efficiency in the residential rental market," Energy Policy, Elsevier, vol. 157(C).
    13. Macchioni Giaquinto, Annarita & Jones, Andrew M. & Rice, Nigel & Zantomio, Francesca, 2021. "Labour supply and informal care responses to health shocks within couples: evidence from the UKHLS," GLO Discussion Paper Series 806, Global Labor Organization (GLO).
    14. Sung‐Hee Jeon, 2017. "The Long‐Term Effects of Cancer on Employment and Earnings," Health Economics, John Wiley & Sons, Ltd., vol. 26(5), pages 671-684, May.
    15. Weber, Olaf & Ahmad, Adnan, 2014. "Empowerment Through Microfinance: The Relation Between Loan Cycle and Level of Empowerment," World Development, Elsevier, vol. 62(C), pages 75-87.
    16. Jeffrey Smith & Arthur Sweetman, 2016. "Viewpoint: Estimating the causal effects of policies and programs," Canadian Journal of Economics, Canadian Economics Association, vol. 49(3), pages 871-905, August.
    17. Robert J. R. Elliott & Liza Jabbour & Liyun Zhang, 2016. "Firm productivity and importing: Evidence from Chinese manufacturing firms," Canadian Journal of Economics/Revue canadienne d'économique, John Wiley & Sons, vol. 49(3), pages 1086-1124, August.
    18. Simone Bertoli & Francesca Marchetta, 2014. "Migration, Remittances and Poverty in Ecuador," Journal of Development Studies, Taylor & Francis Journals, vol. 50(8), pages 1067-1089, August.
    19. repec:zbw:rwirep:0369 is not listed on IDEAS
    20. Veronika Hedija, 2015. "The Effect of Female Managers on Gender Wage Differences," Prague Economic Papers, Prague University of Economics and Business, vol. 2015(1), pages 38-59.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:epplan:v:68:y:2018:i:c:p:81-89. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/locate/evalprogplan .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.