Biomedical Knowledge Graph Benchmark

100 queries. 4 knowledge graphs. 74 million nodes. 1 billion edges. One query language.

Samyama’s biomedical benchmark demonstrates real-world cross-knowledge-graph queries across the largest open biomedical dataset we know of — unified in a single queryable graph on commodity hardware.

The Dataset

Results Summary

100 queries executed. 96 returned data. 3 returned empty (specific data not in snapshot). 1 timed out.

Infrastructure

• Instance: r6a.8xlarge (32 vCPU, 256 GB RAM, AMD EPYC)
• Cost: ~$2.50 (AWS spot, ap-south-1)
• Import time: 31 minutes from v2 snapshots
• NCT bridge: 1,018,483 REFERENCED_IN edges created in 109 seconds
• Index creation: 10 indexes in ~320 seconds

PubMed Queries (35)

Point Lookups

1-Hop Traversals

Multi-Hop Analytics

Aggregations

Clinical Trials Queries (20)

Sample Queries

Aggregations

Complex Queries

Pathways Queries (15)

Drug Interactions Queries (15)

Cross-Knowledge-Graph Queries (15)

These traverse REFERENCED_IN edges connecting 747,505 PubMed articles to clinical trials via PMID↔NCT ID mapping from AACT study_references.

The Headline Results

-- XK02: What drugs are tested in cancer research trials?
-- Spans: MeSH → Article → ClinicalTrial → Intervention (3 KGs)
MATCH (m:MeSHTerm)<-[:ANNOTATED_WITH]-(a:Article)
      -[:REFERENCED_IN]->(t:ClinicalTrial)-[:TESTS]->(i:Intervention)
WHERE m.name = 'Neoplasms'
RETURN i.name, count(DISTINCT t) AS trials ORDER BY trials DESC LIMIT 10

Time: 5.2s — Pembrolizumab (Keytruda) is the most-tested non-placebo cancer drug.

-- XK03: What drugs are tested in diabetes research trials?
MATCH (m:MeSHTerm)<-[:ANNOTATED_WITH]-(a:Article)
      -[:REFERENCED_IN]->(t:ClinicalTrial)-[:TESTS]->(i:Intervention)
WHERE m.name = 'Diabetes Mellitus'
RETURN i.name, count(DISTINCT t) AS trials ORDER BY trials DESC LIMIT 10

Time: 2.4s

-- XK04: What adverse events appear in heart disease trials?
MATCH (m:MeSHTerm)<-[:ANNOTATED_WITH]-(a:Article)
      -[:REFERENCED_IN]->(t:ClinicalTrial)-[:REPORTED]->(ae:AdverseEvent)
WHERE m.name = 'Heart Diseases'
RETURN ae.term, count(DISTINCT t) AS trials ORDER BY trials DESC LIMIT 10

Time: 2.0s

-- XK06: What adverse events appear in Metformin-linked trials?
-- Spans: Chemical → Article → ClinicalTrial → AdverseEvent (4 entities)
MATCH (c:Chemical)<-[:MENTIONS_CHEMICAL]-(a:Article)
      -[:REFERENCED_IN]->(t:ClinicalTrial)-[:REPORTED]->(ae:AdverseEvent)
WHERE c.name = 'Metformin'
RETURN ae.term, count(DISTINCT t) AS trials ORDER BY trials DESC LIMIT 10

Time: 2.1s — Diarrhoea is a known Metformin side effect, confirmed across PubMed + ClinicalTrials.gov.

All Cross-KG Results

What These Results Mean

1. Pembrolizumab dominates cancer trials — The immunotherapy revolution is visible in the data. Across all PubMed articles annotated with “Neoplasms” that link to clinical trials, Keytruda appears in 137 trials, more than any classic chemotherapy agent.

2. Metformin’s GI side effects confirmed cross-database — Diarrhoea ranks 4th in Metformin-linked trial adverse events (185 trials), consistent with clinical knowledge. This was found by traversing Chemical → Article → Trial → AdverseEvent — four entities across two databases.

3. The US conducts 45% of global clinical trials — 190,879 of ~420K total trials. China is second at 1,170 cancer-specific trials.

4. Nature has 140,152 articles in PubMed — making it the most-indexed journal. DNA is its most-mentioned chemical (3,726 articles).

5. TP53 is the most connected protein — 571 interaction partners in Reactome, confirming its role as the “guardian of the genome.”

6. NCI NIH funds 46,137 cancer research papers — and those papers link to 933 placebo-controlled trials.

Query Files

The full query catalog and results are available as CSV for automated benchmarking:

• pubmed-queries.csv — 35 PubMed queries
• clinical-trials-queries.csv — 20 Clinical Trials queries
• pathways-queries.csv — 15 Pathways queries
• drug-interactions-queries.csv — 15 Drug Interactions queries
• cross-kg-queries.csv — 15 Cross-KG queries
• verified-results.csv — All 100 results with timings

Reproducing

All knowledge graph snapshots and the benchmark runner are included in Samyama Graph Enterprise Edition. Contact us to access the pre-built snapshots and benchmark tooling.