IDEAS home Printed from https://ideas.repec.org/a/ibn/eltjnl/v13y2020i9p94.html
   My bibliography  Save this article

A Rasch-Based Validation of ELT Certificate-LORT

Author

Listed:
  • Xin Qu

Abstract

The present study was executed with the purpose of validating ELT Certificate Lesson Observation and Report Task (ELTC-LORT), which was developed by China Language Assessment to certify China’s EFL teachers by performance-based testing. The ELT Certificate has high-stakes considering its impacts on candidates’ recruitment, ELT in China and quality of education, so it is crucially important for its validation so as to guarantee fairness and justice. The validity of task construct and rating rubric went through a process suited for many-facet Rasch measurement supplemented with qualitative interviews. Participants (N = 40) were provided with a video excerpt from a real EFL lesson, and required to deliver a report on the teacher’s performance. Two raters graded the records of the candidates’ reports using rating scales developed to measure EFL teacher candidates’ oral English proficiency and ability to analyze and evaluate teaching. Many-facet Rasch analysis demonstrated a successful estimation, with a noticeable spread among the participants and their traits, proving the task functioned well in measuring candidates’ performance and reflecting the difference of their ability. The raters were found to have good internal self-consistency, but not the same leniency. The rating scales worked well, with the average measures advancing largely in line with Rasch expectations. Semi-structured interviews as well as focus group interviews were executed to provide knowledge regarding the raters’ performance levels and the functionalities of the rating scale items. The findings provide implications for further research and practice of the Certificate.

Suggested Citation

  • Xin Qu, 2020. "A Rasch-Based Validation of ELT Certificate-LORT," English Language Teaching, Canadian Center of Science and Education, vol. 13(9), pages 1-94, September.
    • Handle: RePEc:ibn:eltjnl:v:13:y:2020:i:9:p:94
    as

    Download full text from publisher

    File URL: https://ccsenet.org/journal/index.php/elt/article/download/0/0/43560/45746
    Download Restriction: no
    File URL: https://ccsenet.org/journal/index.php/elt/article/view/0/43560
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Joshua D. Angrist & Jonathan Guryan, 2004. "Teacher Testing, Teacher Education, and Teacher Characteristics," American Economic Review, American Economic Association, vol. 94(2), pages 241-246, May.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Bourdon, Jean & Frölich, Markus & Michaelowa, Katharina, 2007. "Teacher Shortages, Teacher Contracts and their Impact on Education in Africa," IZA Discussion Papers 2844, Institute of Labor Economics (IZA).
    2. Andersson, Christian & Johansson, Per & Waldenström, Nina, 2011. "Do you want your child to have a certified teacher?," Economics of Education Review, Elsevier, vol. 30(1), pages 65-78, February.
    3. Angrist, Joshua D. & Guryan, Jonathan, 2008. "Does teacher testing raise teacher quality? Evidence from state certification requirements," Economics of Education Review, Elsevier, vol. 27(5), pages 483-503, October.
    4. Bobby Chung & Jian Zou, 2021. "Teacher Licensing, Teacher Supply, and Student Achievement: Nationwide Implementation of edTPA," Working Papers 2021-039, Human Capital and Economic Opportunity Working Group.
    5. Stuart S. Yeh, 2010. "The Cost-Effectiveness of NBPTS Teacher Certification," Evaluation Review, , vol. 34(3), pages 220-241, June.
    6. Kraft, Matthew A. & Brunner, Eric J. & Dougherty, Shaun M. & Schwegman, David J., 2020. "Teacher accountability reforms and the supply and quality of new teachers," Journal of Public Economics, Elsevier, vol. 188(C).
    7. Bruhn, Jesse & Imberman, Scott & Winters, Marcus, 2022. "Regulatory arbitrage in teacher hiring and retention: Evidence from Massachusetts Charter Schools," Journal of Public Economics, Elsevier, vol. 215(C).
    8. Mario Pagliero, 2019. "Occupational Licensing in the EU: Protecting Consumers or Limiting Competition?," Review of Industrial Organization, Springer;The Industrial Organization Society, vol. 55(1), pages 137-153, August.
    9. Jonathan Guryan, 2009. "The Race between Education and Technology: A Review Article," Journal of Human Capital, University of Chicago Press, vol. 3(2), pages 177-196.
    10. Larsen, S. Eric, 2010. "Teacher MA attainment rates, 1970-2000," Economics of Education Review, Elsevier, vol. 29(5), pages 772-782, October.
    11. Hendrik Jürges & Kerstin Schneider, 2008. "Fair ranking of teachers," Studies in Empirical Economics, in: Christian Dustmann & Bernd Fitzenberger & Stephen Machin (ed.), The Economics of Education and Training, pages 157-177, Springer.
    12. Andersson, Christian & Waldenström, Nina, 2007. "Teacher certification and student achievement in Swedish compulsory schools," Working Paper Series 2007:6, IFAU - Institute for Evaluation of Labour Market and Education Policy.
    13. Mary D. Bruce & Natalia Ermasova & Linda Mattox, 2019. "The Fiscal Disparity and Achievement Gap between Extremely Wealthy and Poor School Districts in Illinois," Public Organization Review, Springer, vol. 19(4), pages 541-565, December.
    14. Pagliero, Mario, 2018. "Occupational Licensing, Labor Mobility, and the Unfairness of Entry Standards," CEPR Discussion Papers 13076, C.E.P.R. Discussion Papers.
    15. Cao, Yueming & Wang, Dan & Bai, Yu & Ru, Tong, 2024. "Effect of ethnic stereotype threat on academic performance: Experimental evidence from rural China," China Economic Review, Elsevier, vol. 86(C).

    More about this item

    JEL classification:

    • R00 - Urban, Rural, Regional, Real Estate, and Transportation Economics - - General - - - General
    • Z0 - Other Special Topics - - General

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:ibn:eltjnl:v:13:y:2020:i:9:p:94. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Canadian Center of Science and Education (email available below). General contact details of provider: https://edirc.repec.org/data/cepflch.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.