A Systematic Assessment of MHC Class II Peptide Binding Predictions and Evaluation of a Consensus Approach

The identification of MHC class II restricted peptide epitopes is an important goal in immunological research. A number of computational tools have been developed for this purpose, but there is a lack of large-scale systematic evaluation of their performance. Herein, we used a comprehensive dataset consisting of more than 10,000 previously unpublished MHC-peptide binding affinities, 29 peptide/MHC crystal structures, and 664 peptides experimentally tested for CD4+ T cell responses to systematically evaluate the performances of publicly available MHC class II binding prediction tools. While in selected instances the best tools were associated with AUC values up to 0.86, in general, class II predictions did not perform as well as historically noted for class I predictions. It appears that the ability of MHC class II molecules to bind variable length peptides, which requires the correct assignment of peptide binding cores, is a critical factor limiting the performance of existing prediction tools. To improve performance, we implemented a consensus prediction approach that combines methods with top performances. We show that this consensus approach achieved best overall performance. Finally, we make the large datasets used publicly available as a benchmark to facilitate further development of MHC class II binding peptide prediction methods.Author Summary: A critical step in developing immune response against pathogens is the recognition of antigenic peptides presented by MHC class II molecules. Since experiments for MHC class II binding peptide identification are expensive and time consuming, computational tools have been developed as fast alternatives but with inferior performance. Here, we carried out a large-scale systematic evaluation of existing prediction tools with the aim of establishing a benchmark for performance comparison and to identify directions that can further improve prediction performance. We provide an unbiased ranking of the performance of publicly available MHC class II prediction tools and demonstrate that the MHC class II prediction tools did not perform as well as the MHC class I tools. In addition, we show that the size of training data and the correct identification of the binding core are the two factors limiting the performance of existing tools. Finally, we make available to the immunology community a large dataset to facilitate the evaluation and development of MHC class II binding prediction tools.