IDEAS home Printed from https://ideas.repec.org/p/arx/papers/2110.00952.html
   My bibliography  Save this paper

Information Elicitation Meets Clustering

Author

Listed:
  • Yuqing Kong

Abstract

In the setting where we want to aggregate people's subjective evaluations, plurality vote may be meaningless when a large amount of low-effort people always report "good" regardless of the true quality. "Surprisingly popular" method, picking the most surprising answer compared to the prior, handle this issue to some extent. However, it is still not fully robust to people's strategies. Here in the setting where a large number of people are asked to answer a small number of multi-choice questions (multi-task, large group), we propose an information aggregation method that is robust to people's strategies. Interestingly, this method can be seen as a rotated "surprisingly popular". It is based on a new clustering method, Determinant MaxImization (DMI)-clustering, and a key conceptual idea that information elicitation without ground-truth can be seen as a clustering problem. Of independent interest, DMI-clustering is a general clustering method that aims to maximize the volume of the simplex consisting of each cluster's mean multiplying the product of the cluster sizes. We show that DMI-clustering is invariant to any non-degenerate affine transformation for all data points. When the data point's dimension is a constant, DMI-clustering can be solved in polynomial time. In general, we present a simple heuristic for DMI-clustering which is very similar to Lloyd's algorithm for k-means. Additionally, we also apply the clustering idea in the single-task setting and use the spectral method to propose a new aggregation method that utilizes the second-moment information elicited from the crowds.

Suggested Citation

  • Yuqing Kong, 2021. "Information Elicitation Meets Clustering," Papers 2110.00952, arXiv.org.
  • Handle: RePEc:arx:papers:2110.00952
    as

    Download full text from publisher

    File URL: http://arxiv.org/pdf/2110.00952
    File Function: Latest version
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. A. P. Dawid & A. M. Skene, 1979. "Maximum Likelihood Estimation of Observer Error‐Rates Using the EM Algorithm," Journal of the Royal Statistical Society Series C, Royal Statistical Society, vol. 28(1), pages 20-28, March.
    2. Dražen Prelec & H. Sebastian Seung & John McCoy, 2017. "A solution to the single-question crowd wisdom problem," Nature, Nature, vol. 541(7638), pages 532-535, January.
    3. David V. Budescu & Eva Chen, 2015. "Identifying Expertise to Extract the Wisdom of Crowds," Management Science, INFORMS, vol. 61(2), pages 267-280, February.
    4. Hsin-Hsiung Huang & Jie Yang, 2020. "Affine-transformation invariant clustering models," Journal of Statistical Distributions and Applications, Springer, vol. 7(1), pages 1-24, December.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Jaspersen, Johannes G., 2022. "Convex combinations in judgment aggregation," European Journal of Operational Research, Elsevier, vol. 299(2), pages 780-794.
    2. Shinitzky, Hilla & Shemesh, Yhonatan & Leiser, David & Gilead, Michael, 2024. "Improving geopolitical forecasts with 100 brains and one computer," International Journal of Forecasting, Elsevier, vol. 40(3), pages 958-970.
    3. Tang, Ming & Liao, Huchang, 2024. "Group efficiency and individual fairness tradeoff in making wise decisions," Omega, Elsevier, vol. 124(C).
    4. Jon Atwell & Marlon Twyman II, 2023. "Metawisdom of the Crowd: How Choice Within Aided Decision Making Can Make Crowd Wisdom Robust," Papers 2308.15451, arXiv.org.
    5. Cem Peker, 2023. "Extracting the collective wisdom in probabilistic judgments," Theory and Decision, Springer, vol. 94(3), pages 467-501, April.
    6. Tom Wilkening & Marcellin Martinie & Piers D. L. Howe, 2022. "Hidden Experts in the Crowd: Using Meta-Predictions to Leverage Expertise in Single-Question Prediction Problems," Management Science, INFORMS, vol. 68(1), pages 487-508, January.
    7. Nir Billfeld & Moshe Kim, 2019. "Semiparametric correction for endogenous truncation bias with Vox Populi based participation decision," Papers 1902.06286, arXiv.org.
    8. Asa B. Palley & Jack B. Soll, 2019. "Extracting the Wisdom of Crowds When Information Is Shared," Management Science, INFORMS, vol. 67(5), pages 2291-2309, May.
    9. Liu, Fang & Chen, Ya-Ru & Zhou, Da-Hai, 2023. "A two-dimensional approach to flexibility degree of XOR numbers with application to group decision making," Mathematics and Computers in Simulation (MATCOM), Elsevier, vol. 207(C), pages 267-287.
    10. Vitalii Antoshchuk & Volodymyr Filippov & Varvara Kuvaieva, 2021. "Development of methodological support for improving the quality of expert assessment of business processes," Technology audit and production reserves, Socionet;Technology audit and production reserves, vol. 1(4(57)), pages 22-27.
    11. Dan Zhu & Qingwei Wang & John Goddard, 2022. "A new hedging hypothesis regarding prediction interval formation in stock price forecasting," Journal of Forecasting, John Wiley & Sons, Ltd., vol. 41(4), pages 697-717, July.
    12. Huang, He & Chen, Yahong & Ma, Yefeng, 2021. "Modeling the competitive diffusions of rumor and knowledge and the impacts on epidemic spreading," Applied Mathematics and Computation, Elsevier, vol. 388(C).
    13. Bernd Frick & Franziska Prockl, 2018. "Information Precision In Online Communities: Player Valuations On Www.Transfermarkt.De," Working Papers Dissertations 37, Paderborn University, Faculty of Business Administration and Economics.
    14. repec:cup:judgdm:v:14:y:2019:i:4:p:395-411 is not listed on IDEAS
    15. Yuqing Kong, 2019. "Dominantly Truthful Multi-task Peer Prediction with a Constant Number of Tasks," Papers 1911.00272, arXiv.org.
    16. Dai, Min & Jia, Yanwei & Kou, Steven, 2021. "The wisdom of the crowd and prediction markets," Journal of Econometrics, Elsevier, vol. 222(1), pages 561-578.
    17. Benchimol, Jonathan & El-Shagi, Makram & Saadon, Yossi, 2022. "Do expert experience and characteristics affect inflation forecasts?," Journal of Economic Behavior & Organization, Elsevier, vol. 201(C), pages 205-226.
    18. Funk, Patrick & Davis, Alex & Vaishnav, Parth & Dewitt, Barry & Fuchs, Erica, 2020. "Individual inconsistency and aggregate rationality: Overcoming inconsistencies in expert judgment at the technical frontier," Technological Forecasting and Social Change, Elsevier, vol. 155(C).
    19. Brown, Alasdair & Reade, J. James, 2019. "The wisdom of amateur crowds: Evidence from an online community of sports tipsters," European Journal of Operational Research, Elsevier, vol. 272(3), pages 1073-1081.
    20. Bergemann, Dirk & Ottaviani, Marco, 2021. "Information Markets and Nonmarkets," CEPR Discussion Papers 16459, C.E.P.R. Discussion Papers.
    21. Patrick Afflerbach & Christopher Dun & Henner Gimpel & Dominik Parak & Johannes Seyfried, 2021. "A Simulation-Based Approach to Understanding the Wisdom of Crowds Phenomenon in Aggregating Expert Judgment," Business & Information Systems Engineering: The International Journal of WIRTSCHAFTSINFORMATIK, Springer;Gesellschaft für Informatik e.V. (GI), vol. 63(4), pages 329-348, August.

    More about this item

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:arx:papers:2110.00952. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: arXiv administrators (email available below). General contact details of provider: http://arxiv.org/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.