IDEAS home Printed from https://ideas.repec.org/a/eee/csdana/v179y2023ics0167947322002365.html
   My bibliography  Save this article

Extending the Fellegi-Sunter record linkage model for mixed-type data with application to the French national health data system

Author

Listed:
  • Vo, Thanh Huan
  • Chauvet, Guillaume
  • Happe, André
  • Oger, Emmanuel
  • Paquelet, Stéphane
  • Garès, Valérie

Abstract

Probabilistic record linkage is a process of combining data from different sources, when such data refer to common entities and identifying information is not available. A probabilistic record linkage framework that takes into account multiple non-identifying information that this is limited to simple binary comparison between matching variables has been previously proposed. An extension of this method is proposed for mixed-type comparison vectors. A mixture model for handling comparison values of low prevalence categorical matching variables, and a mixture of hurdle gamma distribution for handling comparison values of continuous matching variables have been developed. The parameters are estimated by means of the Expectation Conditional Maximization (ECM) algorithm. Through a Monte Carlo simulation study, both the posterior probability estimation for a record pair to be a match and the prediction of matched record pairs are evaluated. The simulation results indicate that the proposed methods outperform existing ones in most considered cases. The proposed methods are applied on a real dataset, to perform linkage between a registry of patients suffering from venous thromboembolism in the Brest district area (GETBO) and the French national health information system (SNDS).

Suggested Citation

  • Vo, Thanh Huan & Chauvet, Guillaume & Happe, André & Oger, Emmanuel & Paquelet, Stéphane & Garès, Valérie, 2023. "Extending the Fellegi-Sunter record linkage model for mixed-type data with application to the French national health data system," Computational Statistics & Data Analysis, Elsevier, vol. 179(C).
    • Handle: RePEc:eee:csdana:v:179:y:2023:i:c:s0167947322002365
    DOI: 10.1016/j.csda.2022.107656
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0167947322002365
    Download Restriction: Full text for ScienceDirect subscribers only.
    File URL: https://libkey.io/10.1016/j.csda.2022.107656?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Sayers, Adrian & Ben-Shlomo, Yoav & Blom, Ashley W. & Steele, Fiona, 2015. "Probabilistic record linkage," LSE Research Online Documents on Economics 64894, London School of Economics and Political Science, LSE Library.
    2. P. Lahiri & Michael D. Larsen, 2005. "Regression Analysis With Linked Data," Journal of the American Statistical Association, American Statistical Association, vol. 100, pages 222-230, March.
    3. Rebecca C. Steorts & Rob Hall & Stephen E. Fienberg, 2016. "A Bayesian Approach to Graphical Record Linkage and Deduplication," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 111(516), pages 1660-1672, October.
    4. Hofert, Marius & Mächler, Martin, 2016. "Parallel and Other Simulations in R Made Easy: An End-to-End Study," Journal of Statistical Software, Foundation for Open Access Statistics, vol. 69(i04).
    5. Enamorado, Ted & Fifield, Benjamin & Imai, Kosuke, 2019. "Using a Probabilistic Model to Assist Merging of Large-Scale Administrative Records," American Political Science Review, Cambridge University Press, vol. 113(2), pages 353-371, May.
    6. J. B. Copas & F. J. Hilton, 1990. "Record Linkage: Statistical Models for Matching Computer Records," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 153(3), pages 287-312, May.
    7. Kim, Gunky & Chambers, Raymond, 2012. "Regression analysis under incomplete linkage," Computational Statistics & Data Analysis, Elsevier, vol. 56(9), pages 2756-2770.
    8. Mauricio Sadinle, 2017. "Bayesian Estimation of Bipartite Matchings for Record Linkage," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 112(518), pages 600-612, April.
    9. Abdullah-Al Mamun & Robert Aseltine & Sanguthevar Rajasekaran, 2016. "Efficient Record Linkage Algorithms Using Complete Linkage Clustering," PLOS ONE, Public Library of Science, vol. 11(4), pages 1-21, April.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Li‐Chun Zhang & Tiziana Tuoto, 2021. "Linkage‐data linear regression," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 184(2), pages 522-547, April.
    2. Bera Sabyasachi & Chatterjee Snigdhansu, 2020. "High dimensional, robust, unsupervised record linkage," Statistics in Transition New Series, Statistics Poland, vol. 21(4), pages 123-143, August.
    3. Sabyasachi Bera & Snigdhansu Chatterjee, 2020. "High dimensional, robust, unsupervised record linkage," Statistics in Transition New Series, Polish Statistical Association, vol. 21(4), pages 123-143, August.
    4. Ben Powell & Paul A. Smith, 2020. "Computing expectations and marginal likelihoods for permutations," Computational Statistics, Springer, vol. 35(2), pages 871-891, June.
    5. Han Ying, 2020. "Discussion of “Small area estimation: its evolution in five decades”, by Malay Ghosh," Statistics in Transition New Series, Statistics Poland, vol. 21(4), pages 30-34, August.
    6. Ying Han, 2020. "Discussion of "Small area estimation: its evolution in five decades", by Malay Ghosh," Statistics in Transition New Series, Polish Statistical Association, vol. 21(4), pages 30-34, August.
    7. Daniel H. Weinberg & John M. Abowd & Robert F. Belli & Noel Cressie & David C. Folch & Scott H. Holan & Margaret C. Levenstein & Kristen M. Olson & Jerome P. Reiter & Matthew D. Shapiro & Jolene Smyth, 2017. "Effects of a Government-Academic Partnership: Has the NSF-Census Bureau Research Network Helped Improve the U.S. Statistical System?," Working Papers 17-59r, Center for Economic Studies, U.S. Census Bureau.
    8. John M. Abowd & Joelle Abramowitz & Margaret C. Levenstein & Kristin McCue & Dhiren Patki & Trivellore Raghunathan & Ann M. Rodgers & Matthew D. Shapiro & Nada Wasi & Dawn Zinsser, 2021. "Finding Needles in Haystacks: Multiple-Imputation Record Linkage Using Machine Learning," Working Papers 21-35, Center for Economic Studies, U.S. Census Bureau.
    9. Duncan Smith, 2020. "Re‐identification in the Absence of Common Variables for Matching," International Statistical Review, International Statistical Institute, vol. 88(2), pages 354-379, August.
    10. Tatiana Komarova & Denis Nekipelov & Evgeny Yakovlev, 2018. "Identification, data combination, and the risk of disclosure," Quantitative Economics, Econometric Society, vol. 9(1), pages 395-440, March.
    11. Angelo Moretti & Natalie Shlomo, 2023. "Improving Probabilistic Record Linkage Using Statistical Prediction Models," International Statistical Review, International Statistical Institute, vol. 91(3), pages 368-394, December.
    12. Betancourt, Brenda & Sosa, Juan & Rodríguez, Abel, 2022. "A prior for record linkage based on allelic partitions," Computational Statistics & Data Analysis, Elsevier, vol. 172(C).
    13. N. Salvati & E. Fabrizi & M. G. Ranalli & R. L. Chambers, 2021. "Small area estimation with linked data," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 83(1), pages 78-107, February.
    14. Ray Chambers & Andrea Diniz da Silva, 2020. "Improved secondary analysis of linked data: a framework and an illustration," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 183(1), pages 37-59, January.
    15. Al-Kandari Noriah M. & Lahiri Partha, 2016. "Prediction of a Function of Misclassified Binary Data," Statistics in Transition New Series, Statistics Poland, vol. 17(3), pages 429-447, September.
    16. Lee, Gyumin & Lee, Sungjun & Lee, Changyong, 2023. "Inventor–licensee matchmaking for university technology licensing: A fastText approach," Technovation, Elsevier, vol. 125(C).
    17. Durrant, Gabriele B. & D'Arrigo, Julia & Steele, Fiona, 2011. "Using field process data to predict best times of contact conditioning on household and interviewer influences," LSE Research Online Documents on Economics 52201, London School of Economics and Political Science, LSE Library.
    18. Kim, Gunky & Chambers, Raymond, 2012. "Regression analysis under incomplete linkage," Computational Statistics & Data Analysis, Elsevier, vol. 56(9), pages 2756-2770.
    19. Eric Chyn & Kareem Haggag, 2023. "Moved to Vote: The Long-Run Effects of Neighborhoods on Political Participation," The Review of Economics and Statistics, MIT Press, vol. 105(6), pages 1596-1605, November.
    20. Debabrata Dey, 2003. "Record Matching in Data Warehouses: A Decision Model for Data Consolidation," Operations Research, INFORMS, vol. 51(2), pages 240-254, April.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:csdana:v:179:y:2023:i:c:s0167947322002365. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/locate/csda .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.