STaRK-Prime
STaRK-Prime targets complex biomedical inquiries related to diseases, drugs, genes/proteins, etc. It features multi-hop queries that incorporate diverse biomedical properties and ensure each query reflects a real-world scenario. Furthermore, STaRK-Prime enhances query diversity by adopting three perspectives — medical scientist, doctor, and patient — during query generation to comprehensively evaluate retrieval systems.
Semi-structured Knowledge Base
The Prime SKB contains ten entity types (such as disease, gene/protein, molecular function, pathway), and eighteen relation types (such as associated_with, indication/contraindication, parent-child, side_effect). Compared to the Amazon and MAG SKBs, Prime SKB has a denser structure with a broader variety of relations. While textual information is provided for disease and drug entities, we enhance the dataset with integrated data for gene/protein and pathway entities from various databases, covering genomic positions, gene activity summaries, and pathway events.
Data Statistics
| Num of entity types | 10 |
|---|---|
| Num of relation types | 18 |
| Num of avg. degree | 125.2 |
| Num of entities | 129,375 |
| Num of relations | 8,100,498 |
| Num of tokens | 31,844,769 |
STaRK QA Dataset
Data Statistics
| Synthesized | Human-generated | |
|---|---|---|
| Num of queries | 11,204 | 109 |
| Num of queries with multiple answers | 4,188 | 43 |
| Average num of answers | 2.56 | 2.61 |
License: CC-BY-4.0
Query Dataset Examples
Synthesized
Q: I am looking for a gene or protein that plays a role in ribosomal operations and has an interaction with the protein RPL23. It should be linked to a disorder common with RPL23, and its absence should be a known cause of Diamond-Blackfan anemia. Which gene or protein fits these criteria?
A: 2496, 2977, 4933, 59, 717, 1518, 6384, 4789, 2137, 827, 1948, 349, 5054, 7423
Q: Can you identify any medications that, when used in combination with Bimatoprost, enhance its effectiveness in lowering intraocular pressure for patients diagnosed with open-angle glaucoma or ocular hypertension, and that are also approved for treating these conditions as Bimatoprost is?
A: 15105, 15779, 15079, 15156, 14677, 15095, 17434, 17435, 17436
Q: What is the name of the inflammatory disease that primarily targets the small intestine and is linked to Crohn's ileitis and jejunitis?
A: 94833, 95279, 3581
Human-generated
Q: I think I have the flu but it's not flu season. Also I'm really tired and my temperature is over 100. I was recently in France and drank a lot of tap water there. Any idea why I might be sick?
A: 33473
Q: My mom got into ICU unfortunately and had ventilator on. The doctor said she got COPD and pneumonia, which drug should the doctor give my mom?
A: 21446, 15254, 14873, 15229, 15614
Q: Which gene in chromosme 11 and is iinvolved in heart left ventricle, liver and is a pseudogene?
A: 78514
Reference
[1] Payal Chandak, Kexin Huang, and Marinka Zitnik. 2023. Building a knowledge graph to enable precision medicine. Scientific Data (2023).