Diffusion enables integration of heterogeneous data and user-driven learning in a desktop knowledge-base

Integrating reference datasets (e.g. from high-throughput experiments) with unstructured and manually-assembled information (e.g. notes or comments from individual researchers) has the potential to tailor bioinformatic analyses to specific needs and to lead to new insights. However, developing bespoke analysis pipelines from scratch is time-consuming, and general tools for exploring such heterogeneous data are not available. We argue that by treating all data as text, a knowledge-base can accommodate a range of bioinformatic data types and applications. We show that a database coupled to nearest-neighbor algorithms can address common tasks such as gene-set analysis as well as specific tasks such as ontology translation. We further show that a mathematical transformation motivated by diffusion can be effective for exploration across heterogeneous datasets. Diffusion enables the knowledge-base to begin with a sparse query, impute more features, and find matches that would otherwise remain hidden. This can be used, for example, to map multi-modal queries consisting of gene symbols and phenotypes to descriptions of diseases. Diffusion also enables user-driven learning: when the knowledge-base cannot provide satisfactory search results in the first instance, users can improve the results in real-time by adding domain-specific knowledge. User-driven learning has implications for data management, integration, and curation.Author summary: Biological datasets can be too large to unravel without computational tools and, at the same time, too small to benefit from machine-learning approaches that require large data volumes to train. Analyses of niche datasets are also hindered by the difficulty of incorporating knowledge from domain experts into practical algorithms. In this work, we argue that such challenges may be addressed by data integration platforms that can learn in real-time from users. We support the argument with an implementation of a practical, general-purpose, tool. We show that its search capabilities can solve common bioinformatic tasks such as gene-set analysis and matching genes and phenotypes to diseases, which are usually tackled using statistical methods and bespoke algorithms. Importantly, our tool leaves domain experts full control to modulate search results in transparent, biologically meaningful ways. It also allows users to improve search outputs if the default results are incomplete. These features extend the capabilities of existing knowledge-bases and empower domain experts to tailor data exploration to their needs.