IDEAS home Printed from https://ideas.repec.org/a/spr/comgts/v9y2012i3p401-415.html
   My bibliography  Save this article

Simple and efficient classification scheme based on specific vocabulary

Author

Listed:
  • Jacques Savoy
  • Olena Zubaryeva

Abstract

Assuming a binomial distribution for word occurrence, we propose computing a standardized Z score to define the specific vocabulary of a subset compared to that of the entire corpus. This approach is applied to weight terms (character n-gram, word, stem, lemma or sequence of them) which characterize a document. We then show how these Z score values can be used to derive a simple and efficient categorization scheme. To evaluate this proposition and demonstrate its effectiveness, we develop two experiments. First, the system must categorize speeches given by B. Obama as being either electoral or presidential speech. In a second experiment, sentences are extracted from these speeches and then categorized under the headings electoral or presidential. Based on these evaluations, the proposed classification scheme tends to perform better than a support vector machine model for both experiments, on the one hand, and on the other, shows a better performance level than a Naïve Bayes classifier on the first test and a slightly lower performance on the second (10-fold cross validation). Copyright Springer-Verlag 2012

Suggested Citation

  • Jacques Savoy & Olena Zubaryeva, 2012. "Simple and efficient classification scheme based on specific vocabulary," Computational Management Science, Springer, vol. 9(3), pages 401-415, August.
    • Handle: RePEc:spr:comgts:v:9:y:2012:i:3:p:401-415
    DOI: 10.1007/s10287-012-0149-z
    as

    Download full text from publisher

    File URL: http://hdl.handle.net/10.1007/s10287-012-0149-z
    Download Restriction: Access to full text is restricted to subscribers.
    File URL: https://libkey.io/10.1007/s10287-012-0149-z?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Rong Zheng & Jiexun Li & Hsinchun Chen & Zan Huang, 2006. "A framework for authorship identification of online messages: Writing‐style features and classification techniques," Journal of the American Society for Information Science and Technology, Association for Information Science & Technology, vol. 57(3), pages 378-393, February.
    2. Claire Fautsch & Jacques Savoy, 2009. "Algorithmic stemmers or morphological analysis? An evaluation," Journal of the American Society for Information Science and Technology, Association for Information Science & Technology, vol. 60(8), pages 1616-1624, August.
    3. Aidan Finn & Nicholas Kushmerick, 2006. "Learning to classify documents according to genre," Journal of the American Society for Information Science and Technology, Association for Information Science & Technology, vol. 57(11), pages 1506-1518, September.
    4. Efstathios Stamatatos, 2009. "A survey of modern authorship attribution methods," Journal of the American Society for Information Science and Technology, Association for Information Science & Technology, vol. 60(3), pages 538-556, March.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Silvia Corbara & Alejandro Moreo & Fabrizio Sebastiani, 2023. "Syllabic quantity patterns as rhythmic features for Latin authorship attribution," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 74(1), pages 128-141, January.
    2. Nils-Axel M?rner, 2018. "Evaluation of the Performance and Efficiency of the Automated Linguistic Features for Author Identification in Short Text Messages Using Different Variable Selection Techniques," Studies in Media and Communication, Redfame publishing, vol. 6(2), pages 83-102, December.
    3. Maryam Ebrahimpour & Tālis J Putniņš & Matthew J Berryman & Andrew Allison & Brian W-H Ng & Derek Abbott, 2013. "Automated Authorship Attribution Using Advanced Signal Classification Techniques," PLOS ONE, Public Library of Science, vol. 8(2), pages 1-12, February.
    4. Diego R Amancio, 2015. "Probing the Topological Properties of Complex Networks Modeling Short Written Texts," PLOS ONE, Public Library of Science, vol. 10(2), pages 1-17, February.
    5. Sanda-Maria Avram & Mihai Oltean, 2022. "A Comparison of Several AI Techniques for Authorship Attribution on Romanian Texts," Mathematics, MDPI, vol. 10(23), pages 1-35, December.
    6. Ballandonne, Matthieu & Cersosimo, Igor, 2022. "Towards a “Text as Data” Approach in the History of Economics: An Application to Adam Smith’s Classics," OSF Preprints mg3zb, Center for Open Science.
    7. Malik Muhammad Saad Missen & Sajeeha Qureshi & Nadeem Salamat & Nadeem Akhtar & Hina Asmat & Mickaël Coustaty & V. B. Surya Prasath, 2020. "Scientometric analysis of social science and science disciplines in a developing nation: a case study of Pakistan in the last decade," Scientometrics, Springer;Akadémiai Kiadó, vol. 123(1), pages 113-142, April.
    8. Teso, E. & Olmedilla, M. & Martínez-Torres, M.R. & Toral, S.L., 2018. "Application of text mining techniques to the analysis of discourse in eWOM communications from a gender perspective," Technological Forecasting and Social Change, Elsevier, vol. 129(C), pages 131-142.
    9. Jinghui (Jove) Hou & Xiao Ma, 2022. "Space Norms for Constructing Quality Reviews on Online Consumer Review Sites," Information Systems Research, INFORMS, vol. 33(3), pages 1093-1112, September.
    10. Matthew J. Schneider & Shawn Mankad, 2021. "A Two-Stage Authorship Attribution Method Using Text and Structured Data for De-Anonymizing User-Generated Content," Customer Needs and Solutions, Springer;Institute for Sustainable Innovation and Growth (iSIG), vol. 8(3), pages 66-83, September.
    11. Kargin, Vladislav, 2016. "On variation of word frequencies in Russian literary texts," Physica A: Statistical Mechanics and its Applications, Elsevier, vol. 445(C), pages 328-334.
    12. Andi Rexha & Mark Kröll & Hermann Ziak & Roman Kern, 2018. "Authorship identification of documents with high content similarity," Scientometrics, Springer;Akadémiai Kiadó, vol. 115(1), pages 223-237, April.
    13. Ruchika Malhotra & Anjali Sharma, 2017. "Quantitative evaluation of web metrics for automatic genre classification of web pages," International Journal of System Assurance Engineering and Management, Springer;The Society for Reliability, Engineering Quality and Operations Management (SREQOM),India, and Division of Operation and Maintenance, Lulea University of Technology, Sweden, vol. 8(2), pages 1567-1579, November.
    14. Rutherford, Brian A., 2013. "A genre-theoretic approach to financial reporting research," The British Accounting Review, Elsevier, vol. 45(4), pages 297-310.
    15. Michael Scholz & Markus Franz & Oliver Hinz, 2016. "The Ambiguous Identifier Clustering Technique," Electronic Markets, Springer;IIM University of St. Gallen, vol. 26(2), pages 143-156, May.
    16. Mingfang Wu & David Hawking & Andrew Turpin & Falk Scholer, 2012. "Using anchor text for homepage and topic distillation search tasks," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 63(6), pages 1235-1255, June.
    17. Haoran Zhu & Lei Lei, 2022. "The Research Trends of Text Classification Studies (2000–2020): A Bibliometric Analysis," SAGE Open, , vol. 12(2), pages 21582440221, April.
    18. Chunneng Huang & Tianjun Fu & Hsinchun Chen, 2010. "Text‐based video content classification for online video‐sharing sites," Journal of the American Society for Information Science and Technology, Association for Information Science & Technology, vol. 61(5), pages 891-906, May.
    19. Oleg Sobchuk & Artjoms Šeļa, 2024. "Computational thematics: comparing algorithms for clustering the genres of literary fiction," Palgrave Communications, Palgrave Macmillan, vol. 11(1), pages 1-12, December.
    20. Jennifer A. Byrne & Cyril Labbé, 2017. "Striking similarities between publications from China describing single gene knockdown experiments in human cancer cell lines," Scientometrics, Springer;Akadémiai Kiadó, vol. 110(3), pages 1471-1493, March.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:spr:comgts:v:9:y:2012:i:3:p:401-415. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.