IDEAS home Printed from https://ideas.repec.org/p/arx/papers/1911.00272.html
   My bibliography  Save this paper

Dominantly Truthful Multi-task Peer Prediction with a Constant Number of Tasks

Author

Listed:
  • Yuqing Kong

Abstract

In the setting where participants are asked multiple similar possibly subjective multi-choice questions (e.g. Do you like Panda Express? Y/N; do you like Chick-fil-A? Y/N), a series of peer prediction mechanisms are designed to incentivize honest reports and some of them achieve dominantly truthfulness: truth-telling is a dominant strategy and strictly dominate other "non-permutation strategy" with some mild conditions. However, a major issue hinders the practical usage of those mechanisms: they require the participants to perform an infinite number of tasks. When the participants perform a finite number of tasks, these mechanisms only achieve approximated dominant truthfulness. The existence of a dominantly truthful multi-task peer prediction mechanism that only requires a finite number of tasks remains to be an open question that may have a negative result, even with full prior knowledge. This paper answers this open question by proposing a new mechanism, Determinant based Mutual Information Mechanism (DMI-Mechanism), that is dominantly truthful when the number of tasks is at least 2C and the number of participants is at least 2. C is the number of choices for each question (C=2 for binary-choice questions). In addition to incentivizing honest reports, DMI-Mechanism can also be transferred into an information evaluation rule that identifies high-quality information without verification when there are at least 3 participants. To the best of our knowledge, DMI-Mechanism is the first dominantly truthful mechanism that works for a finite number of tasks, not to say a small constant number of tasks.

Suggested Citation

  • Yuqing Kong, 2019. "Dominantly Truthful Multi-task Peer Prediction with a Constant Number of Tasks," Papers 1911.00272, arXiv.org.
  • Handle: RePEc:arx:papers:1911.00272
    as

    Download full text from publisher

    File URL: http://arxiv.org/pdf/1911.00272
    File Function: Latest version
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Nolan Miller & Paul Resnick & Richard Zeckhauser, 2005. "Eliciting Informative Feedback: The Peer-Prediction Method," Management Science, INFORMS, vol. 51(9), pages 1359-1373, September.
    2. Dražen Prelec & H. Sebastian Seung & John McCoy, 2017. "A solution to the single-question crowd wisdom problem," Nature, Nature, vol. 541(7638), pages 532-535, January.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Bergemann, Dirk & Ottaviani, Marco, 2021. "Information Markets and Nonmarkets," CEPR Discussion Papers 16459, C.E.P.R. Discussion Papers.
    2. Zhou, Fan & Page, Lionel & Perrons, Robert K. & Zheng, Zuduo & Washington, Simon, 2019. "Long-term forecasts for energy commodities price: What the experts think," Energy Economics, Elsevier, vol. 84(C).
    3. Asa B. Palley & Jack B. Soll, 2019. "Extracting the Wisdom of Crowds When Information Is Shared," Management Science, INFORMS, vol. 67(5), pages 2291-2309, May.
    4. Steffen Borgwardt & Rafael M. Frongillo, 2019. "Power Diagram Detection with Applications to Information Elicitation," Journal of Optimization Theory and Applications, Springer, vol. 181(1), pages 184-196, April.
    5. Huang, He & Chen, Yahong & Ma, Yefeng, 2021. "Modeling the competitive diffusions of rumor and knowledge and the impacts on epidemic spreading," Applied Mathematics and Computation, Elsevier, vol. 388(C).
    6. Peysakhovich, Alexander & Plagborg-Møller, Mikkel, 2012. "A note on proper scoring rules and risk aversion," Economics Letters, Elsevier, vol. 117(1), pages 357-361.
    7. Dai, Min & Jia, Yanwei & Kou, Steven, 2021. "The wisdom of the crowd and prediction markets," Journal of Econometrics, Elsevier, vol. 222(1), pages 561-578.
    8. Funk, Patrick & Davis, Alex & Vaishnav, Parth & Dewitt, Barry & Fuchs, Erica, 2020. "Individual inconsistency and aggregate rationality: Overcoming inconsistencies in expert judgment at the technical frontier," Technological Forecasting and Social Change, Elsevier, vol. 155(C).
    9. Jaspersen, Johannes G., 2022. "Convex combinations in judgment aggregation," European Journal of Operational Research, Elsevier, vol. 299(2), pages 780-794.
    10. Aperjis, Christina & Zeckhauser, Richard J. & Miao, Yali, 2014. "Variable temptations and black mark reputations," Games and Economic Behavior, Elsevier, vol. 87(C), pages 70-90.
    11. Cristiano Codagnone & Federico Biagi & Fabienne Abadie, 2016. "The Passions and the Interests: Unpacking the ‘Sharing Economy’," JRC Research Reports JRC101279, Joint Research Centre.
    12. Chrysanthos Dellarocas, 2006. "Strategic Manipulation of Internet Opinion Forums: Implications for Consumers and Firms," Management Science, INFORMS, vol. 52(10), pages 1577-1593, October.
    13. Charness, Gary & Gneezy, Uri & Rasocha, Vlastimil, 2021. "Experimental methods: Eliciting beliefs," Journal of Economic Behavior & Organization, Elsevier, vol. 189(C), pages 234-256.
    14. Siddarth Srinivasan & Jamie Morgenstern, 2021. "Auctions and Peer Prediction for Academic Peer Review," Papers 2109.00923, arXiv.org, revised May 2023.
    15. Weijia (Daisy) Dai & Ginger Jin & Jungmin Lee & Michael Luca, 2018. "Aggregation of consumer ratings: an application to Yelp.com," Quantitative Marketing and Economics (QME), Springer, vol. 16(3), pages 289-339, September.
    16. Shinitzky, Hilla & Shemesh, Yhonatan & Leiser, David & Gilead, Michael, 2024. "Improving geopolitical forecasts with 100 brains and one computer," International Journal of Forecasting, Elsevier, vol. 40(3), pages 958-970.
    17. António Osório, 2017. "Judgement and ranking: living with hidden bias," Annals of Operations Research, Springer, vol. 253(1), pages 501-518, June.
    18. Hitoshi Matsushima & Shunya Noda, 2020. "Unique Information Elicitation," CARF F-Series CARF-F-496, Center for Advanced Research in Finance, Faculty of Economics, The University of Tokyo.
    19. Matsushima, Hitoshi, 2022. "Epistemological implementation of social choice functions," Games and Economic Behavior, Elsevier, vol. 136(C), pages 389-402.
    20. Benjamin Van Roy & Xiang Yan, 2010. "Manipulation Robustness of Collaborative Filtering," Management Science, INFORMS, vol. 56(11), pages 1911-1929, November.

    More about this item

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:arx:papers:1911.00272. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: arXiv administrators (email available below). General contact details of provider: http://arxiv.org/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.