IDEAS home Printed from https://ideas.repec.org/p/arx/papers/2206.02566.html
   My bibliography  Save this paper

Towards Group Learning: Distributed Weighting of Experts

Author

Listed:
  • Ben Abramowitz
  • Nicholas Mattei

Abstract

Aggregating signals from a collection of noisy sources is a fundamental problem in many domains including crowd-sourcing, multi-agent planning, sensor networks, signal processing, voting, ensemble learning, and federated learning. The core question is how to aggregate signals from multiple sources (e.g. experts) in order to reveal an underlying ground truth. While a full answer depends on the type of signal, correlation of signals, and desired output, a problem common to all of these applications is that of differentiating sources based on their quality and weighting them accordingly. It is often assumed that this differentiation and aggregation is done by a single, accurate central mechanism or agent (e.g. judge). We complicate this model in two ways. First, we investigate the setting with both a single judge, and one with multiple judges. Second, given this multi-agent interaction of judges, we investigate various constraints on the judges' reporting space. We build on known results for the optimal weighting of experts and prove that an ensemble of sub-optimal mechanisms can perform optimally under certain conditions. We then show empirically that the ensemble approximates the performance of the optimal mechanism under a broader range of conditions.

Suggested Citation

  • Ben Abramowitz & Nicholas Mattei, 2022. "Towards Group Learning: Distributed Weighting of Experts," Papers 2206.02566, arXiv.org.
    • Handle: RePEc:arx:papers:2206.02566
    as

    Download full text from publisher

    File URL: http://arxiv.org/pdf/2206.02566
    File Function: Latest version
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Eyal Baharad & Jacob Goldberger & Moshe Koppel & Shmuel Nitzan, 2012. "Beyond Condorcet: optimal aggregation rules using voting records," Theory and Decision, Springer, vol. 72(1), pages 113-130, January.
    2. Malik Magdon-Ismail & Lirong Xia, 2018. "A Mathematical Model for Optimal Decisions in a Representative Democracy," Papers 1807.06157, arXiv.org.
    3. Scott Feld & Bernard Grofman, 1984. "The accuracy of group majority decisions in groups with added members," Public Choice, Springer, vol. 42(3), pages 273-285, January.
    4. Lloyd Shapley & Bernard Grofman, 1984. "Optimizing group judgmental accuracy in the presence of interdependencies," Public Choice, Springer, vol. 43(3), pages 329-343, January.
    5. Arnold Cédrick SOH VOUTSA, 2020. "Approval Voting & Majority Judgment in Weighted Representative Democracy," THEMA Working Papers 2020-15, THEMA (THéorie Economique, Modélisation et Applications), Université de Cergy-Pontoise.
    6. Daniel Berend & Jacob Paroush, 1998. "When is Condorcet's Jury Theorem valid?," Social Choice and Welfare, Springer;The Society for Social Choice and Welfare, vol. 15(4), pages 481-488.
    7. Nitzan, Shmuel & Paroush, Jacob, 1982. "Optimal Decision Rules in Uncertain Dichotomous Choice Situations," International Economic Review, Department of Economics, University of Pennsylvania and Osaka University Institute of Social and Economic Research Association, vol. 23(2), pages 289-297, June.
    8. Pivato, Marcus & Soh, Arnold, 2020. "Weighted representative democracy," Journal of Mathematical Economics, Elsevier, vol. 88(C), pages 52-63.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Ben Abramowitz & Omer Lev & Nicholas Mattei, 2022. "Who Reviews The Reviewers? A Multi-Level Jury Problem," Papers 2211.08494, arXiv.org, revised Dec 2023.
    2. Ruth Ben-Yashar & Shmuel Nitzan, 2014. "On the significance of the prior of a correct decision in committees," Theory and Decision, Springer, vol. 76(3), pages 317-327, March.
    3. Ruth Ben-Yashar & Shmuel Nitzan, 2017. "Are two better than one? A note," Public Choice, Springer, vol. 171(3), pages 323-329, June.
    4. Ruth Ben-Yashar & Shmuel Nitzan, 2017. "Is diversity in capabilities desirable when adding decision makers?," Theory and Decision, Springer, vol. 82(3), pages 395-402, March.
    5. Eyal Baharad & Jacob Goldberger & Moshe Koppel & Shmuel Nitzan, 2012. "Beyond Condorcet: optimal aggregation rules using voting records," Theory and Decision, Springer, vol. 72(1), pages 113-130, January.
    6. Baharad, Eyal & Ben-Yashar, Ruth & Patal, Tal, 2020. "On the merit of non-specialization in the context of majority voting," Journal of Mathematical Economics, Elsevier, vol. 87(C), pages 128-133.
    7. Eyal Baharad & Ruth Ben-Yashar & Shmuel Nitzan, 2020. "Variable Competence and Collective Performance: Unanimity Versus Simple Majority Rule," Group Decision and Negotiation, Springer, vol. 29(1), pages 157-167, February.
    8. Shmuel Nitzan & Tomoya Tajika, 2022. "Inequality of decision-makers’ power and marginal contribution," Theory and Decision, Springer, vol. 92(2), pages 275-292, March.
    9. Ruth Ben-Yashar & Miriam Krausz & Shmuel Nitzan, 2018. "The effect of democratic decision-making on investment in reputation," Public Choice, Springer, vol. 177(1), pages 155-164, October.
    10. Sapir, Luba, 2005. "Generalized means of jurors' competencies and marginal changes of jury's size," Mathematical Social Sciences, Elsevier, vol. 50(1), pages 83-101, July.
    11. Ruth Ben-Yashar & Shmuel Nitzan, 2017. "Is diversity in capabilities desirable when adding decision makers?," Theory and Decision, Springer, vol. 82(3), pages 395-402, March.
    12. Ruth Ben-Yashar, 2014. "The generalized homogeneity assumption and the Condorcet jury theorem," Theory and Decision, Springer, vol. 77(2), pages 237-241, August.
    13. Romaniega Sancho, Álvaro, 2022. "On the probability of the Condorcet Jury Theorem or the Miracle of Aggregation," Mathematical Social Sciences, Elsevier, vol. 119(C), pages 41-55.
    14. Ruth Ben-Yashar & Miriam Krausz & Shmuel Nitzan, 2018. "The effect of democratic decision-making on investment in reputation," Public Choice, Springer, vol. 177(1), pages 155-164, October.
    15. Ruth Ben-Yashar, 2023. "An application of simple majority rule to a group with an even number of voters," Theory and Decision, Springer, vol. 94(1), pages 83-95, January.
    16. Takuya Sekiguchi, 2023. "Voting Records as Assessors of Premises Behind Collective Decisions," Group Decision and Negotiation, Springer, vol. 32(2), pages 257-275, April.
    17. Dhillon, Amrita & Kotsialou, Grammateia & Xefteris, Dimitris, 2021. "Information Aggregation with Delegation of Votes," SocArXiv ubk7p, Center for Open Science.
    18. Xiaohong Yu & Zhaoyang Sun, 2022. "The company they keep: When and why Chinese judges engage in collegiality," Journal of Empirical Legal Studies, John Wiley & Sons, vol. 19(4), pages 936-1002, December.
    19. Karotkin, D. & Nitzan, S., 1993. "Some Peculiarities of Group Decision Making in Teams," Discussion Paper 1993-3, Tilburg University, Center for Economic Research.
    20. Ben-Yashar, Ruth & Nitzan, Shmuel, 2019. "Skill, value and remuneration in committees," Economics Letters, Elsevier, vol. 174(C), pages 93-95.

    More about this item

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:arx:papers:2206.02566. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: arXiv administrators (email available below). General contact details of provider: http://arxiv.org/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.