Cost-Effective Quality Assurance in Crowd Labeling
Author
Abstract
Suggested Citation
DOI: 10.1287/isre.2016.0661
Download full text from publisher
References listed on IDEAS
- Maytal Saar-Tsechansky & Prem Melville & Foster Provost, 2009. "Active Feature-Value Acquisition," Management Science, INFORMS, vol. 55(4), pages 664-684, April.
- Nikolay Archak & Anindya Ghose & Panagiotis G. Ipeirotis, 2011. "Deriving the Pricing Power of Product Features by Mining Consumer Reviews," Management Science, INFORMS, vol. 57(8), pages 1485-1509, August.
- A. P. Dawid & A. M. Skene, 1979. "Maximum Likelihood Estimation of Observer Error‐Rates Using the EM Algorithm," Journal of the Royal Statistical Society Series C, Royal Statistical Society, vol. 28(1), pages 20-28, March.
- Wolfgang Ketter & John Collins & Maria Gini & Alok Gupta & Paul Schrater, 2012. "Real-Time Tactical and Strategic Sales Management for Intelligent Agents Guided by Economic Regimes," Information Systems Research, INFORMS, vol. 23(4), pages 1263-1283, December.
- I. Robert Chiang & Vijay S. Mookerjee, 2004. "A Fault Threshold Policy to Manage Software Development Projects," Information Systems Research, INFORMS, vol. 15(1), pages 3-21, March.
- Gediminas Adomavicius & Alok Gupta & Dmitry Zhdanov, 2009. "Designing Intelligent Software Agents for Auctions with Limited Information Feedback," Information Systems Research, INFORMS, vol. 20(4), pages 507-526, December.
- Antonio Moreno & Christian Terwiesch, 2014. "Doing Business with Strangers: Reputation in Online Service Marketplaces," Information Systems Research, INFORMS, vol. 25(4), pages 865-886, December.
- Maytal Saar-Tsechansky & Foster Provost, 2007. "Decision-Centric Active Learning of Binary-Outcome Models," Information Systems Research, INFORMS, vol. 18(1), pages 4-22, March.
- Christina Aperjis & Ramesh Johari, 2010. "Optimal Windows for Aggregating Ratings in Electronic Marketplaces," Management Science, INFORMS, vol. 56(5), pages 864-880, May.
- Zhiqiang Zheng & Balaji Padmanabhan, 2006. "Selectively Acquiring Customer Information: A New Data Acquisition Problem and an Active Learning-Based Solution," Management Science, INFORMS, vol. 52(5), pages 697-712, May.
Citations
Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
Cited by:
- Fügener, A. & Grahl, J. & Gupta, A. & Ketter, W., 2019. "Cognitive challenges in human-AI collaboration: Investigating the path towards productive delegation," ERIM Report Series Research in Management ERS-2019-003-LIS, Erasmus Research Institute of Management (ERIM), ERIM is the joint research institute of the Rotterdam School of Management, Erasmus University and the Erasmus School of Economics (ESE) at Erasmus University Rotterdam.
- Ruyi Ge & Zhiqiang (Eric) Zheng & Xuan Tian & Li Liao, 2021. "Human–Robot Interaction: When Investors Adjust the Usage of Robo-Advisors in Peer-to-Peer Lending," Information Systems Research, INFORMS, vol. 32(3), pages 774-785, September.
- Xuan Bi & Mochen Yang & Gediminas Adomavicius, 2024. "Consumer Acquisition for Recommender Systems: A Theoretical Framework and Empirical Evaluations," Information Systems Research, INFORMS, vol. 35(1), pages 339-362, March.
- Tomer Geva & Maytal Saar‐Tsechansky, 2021. "Who Is a Better Decision Maker? Data‐Driven Expert Ranking Under Unobserved Quality," Production and Operations Management, Production and Operations Management Society, vol. 30(1), pages 127-144, January.
Most related items
These are the items that most often cite the same works as this one and are cited by the same works as this one.- Dominik Gutt & Jürgen Neumann & Steffen Zimmermann & Dennis Kundisch & Jianqing Chen, 2018. "Design of Review Systems - A Strategic Instrument to shape Online Review Behavior and Economic Outcomes," Working Papers Dissertations 42, Paderborn University, Faculty of Business Administration and Economics.
- Micha Kahlen & Karsten Schroer & Wolfgang Ketter & Alok Gupta, 2024. "Smart Markets for Real-Time Allocation of Multiproduct Resources: The Case of Shared Electric Vehicles," Information Systems Research, INFORMS, vol. 35(2), pages 871-889, June.
- Borchert, Philipp & Coussement, Kristof & De Weerdt, Jochen & De Caigny, Arno, 2024. "Industry-sensitive language modeling for business," European Journal of Operational Research, Elsevier, vol. 315(2), pages 691-702.
- Lingfang (Ivy) Li & Erte Xiao, 2014. "Money Talks: Rebate Mechanisms in Reputation System Design," Management Science, INFORMS, vol. 60(8), pages 2054-2072, August.
- Yixin Lu & Alok Gupta & Wolfgang Ketter & Eric van Heck, 2019. "Dynamic Decision Making in Sequential Business-to-Business Auctions: A Structural Econometric Approach," Management Science, INFORMS, vol. 65(8), pages 3853-3876, August.
- Meghana Deodhar & Joydeep Ghosh & Maytal Saar-Tsechansky & Vineet Keshari, 2017. "Active Learning with Multiple Localized Regression Models," INFORMS Journal on Computing, INFORMS, vol. 29(3), pages 503-522, August.
- Apostolos Filippas & John Horton & Joseph M. Golden, 2017. "Reputation in the Long-Run," CESifo Working Paper Series 6750, CESifo.
- Yingfei Wang & Inbal Yahav & Balaji Padmanabhan, 2024. "Smart Testing with Vaccination: A Bandit Algorithm for Active Sampling for Managing COVID-19," Information Systems Research, INFORMS, vol. 35(1), pages 120-144, March.
- Kaiquan Xu & Stephen Shaoyi Liao & Raymond Y. K. Lau & J. Leon Zhao, 2014. "Effective Active Learning Strategies for the Use of Large-Margin Classifiers in Semantic Annotation: An Optimal Parameter Discovery Perspective," INFORMS Journal on Computing, INFORMS, vol. 26(3), pages 461-483, August.
- Wolfgang Ketter & Karsten Schroer & Konstantina Valogianni, 2023. "Information Systems Research for Smart Sustainable Mobility: A Framework and Call for Action," Information Systems Research, INFORMS, vol. 34(3), pages 1045-1065, September.
- Ransome Epie Bawack & Samuel Fosso Wamba & Kevin Daniel André Carillo & Shahriar Akter, 2022. "Artificial intelligence in E-Commerce: a bibliometric study and literature review," Electronic Markets, Springer;IIM University of St. Gallen, vol. 32(1), pages 297-338, March.
- Alain Bensoussan & Radha Mookerjee & Vijay Mookerjee & Wei T. Yue, 2009. "Maintaining Diagnostic Knowledge-Based Systems: A Control-Theoretic Approach," Management Science, INFORMS, vol. 55(2), pages 294-310, February.
- Apostolos Filippas & John J. Horton & Joseph M. Golden, 2019. "Reputation Inflation," NBER Working Papers 25857, National Bureau of Economic Research, Inc.
- Foster, Joshua, 2022. "How rating mechanisms shape user search, quality inference and engagement in online platforms: Experimental evidence," Journal of Business Research, Elsevier, vol. 142(C), pages 791-807.
- Mochen Yang & Gediminas Adomavicius & Gordon Burtch & Yuqing Rena, 2018. "Mind the Gap: Accounting for Measurement Error and Misclassification in Variables Generated via Data Mining," Information Systems Research, INFORMS, vol. 29(1), pages 4-24, March.
- Jorge Mejia & Shawn Mankad & Anandasivam Gopal, 2019. "A for Effort? Using the Crowd to Identify Moral Hazard in New York City Restaurant Hygiene Inspections," Information Systems Research, INFORMS, vol. 30(4), pages 1363-1386, December.
- Xuan Bi & Mochen Yang & Gediminas Adomavicius, 2024. "Consumer Acquisition for Recommender Systems: A Theoretical Framework and Empirical Evaluations," Information Systems Research, INFORMS, vol. 35(1), pages 339-362, March.
- Jinsoo Park & Hamirahanim Abdul Rahman & Jihae Suh & Hazami Hussin, 2019. "A Study of Integrative Bargaining Model with Argumentation-Based Negotiation," Sustainability, MDPI, vol. 11(23), pages 1-21, December.
- Joe Cox & Daniel Kaimann, 2013. "The Signaling Effect of Critics - Evidence from a Market for Experience Goods," Working Papers CIE 68, Paderborn University, CIE Center for International Economics.
- Baxendale, Shane & Macdonald, Emma K. & Wilson, Hugh N., 2015. "The Impact of Different Touchpoints on Brand Consideration," Journal of Retailing, Elsevier, vol. 91(2), pages 235-253.
More about this item
Keywords
crowd labeling; quality assurance; dynamic label allocation; worker performance metrics;All these keywords.
Statistics
Access and download statisticsCorrections
All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:inm:orisre:v:28:y:2017:i:1:p:137-158. See general information about how to correct material in RePEc.
If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.
If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .
If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.
For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Chris Asher (email available below). General contact details of provider: https://edirc.repec.org/data/inforea.html .
Please note that corrections may take a couple of weeks to filter through the various RePEc services.