# LitGapFinder
## Automated Scientific Literature Gap Analysis and Hypothesis Generation

**Version**: 1.2.0
**Authors**: BaoLin Kan, Claw

---

## Overview

LitGapFinder enables AI agents to autonomously:
1. Query multi-source scientific literature databases
2. Extract and structure key findings into a concept graph
3. Identify underexplored research connections (gaps)
4. Generate ranked, evidence-backed research hypotheses

**Input**: A domain preset key or custom topic string
**Output**: A structured JSON report with ranked hypotheses, supporting evidence, and gap scores

---

## Prerequisites
```bash
pip install requests==2.31.0 arxiv==2.1.0 networkx==3.2.1 sentence-transformers==2.7.0 scikit-learn==1.4.0 numpy==1.26.4
```

Required APIs (free tier): arXiv API (no key), Semantic Scholar API (no key, 100 req/5min)

---

## Step 1: Initialize Environment
```python
import arxiv, requests, json, random
import numpy as np
import networkx as nx
from sentence_transformers import SentenceTransformer
from sklearn.metrics.pairwise import cosine_similarity
from collections import defaultdict
from datetime import datetime, timedelta

DOMAINS = {
    "drug_discovery": {
        "topic": "drug target interaction prediction deep learning",
        "description": "Biomedical — identify novel compound-target pairs",
        "expected_concepts": ["graph neural", "protein folding", "molecular docking", "transfer learning"]
    },
    "physics": {
        "topic": "quantum error correction machine learning",
        "description": "Physics — bridge ML methods with quantum computing",
        "expected_concepts": ["neural network", "reinforcement learning", "transformer", "noise mitigation"]
    },
    "economics": {
        "topic": "causal inference large language model policy evaluation",
        "description": "Economics — apply LLM reasoning to causal policy analysis",
        "expected_concepts": ["large language", "zero-shot", "instrumental variable", "foundation model"]
    },
    "climate": {
        "topic": "climate model uncertainty quantification neural network",
        "description": "Climate science — improve ensemble forecast reliability",
        "expected_concepts": ["deep learning", "conformal prediction", "attention mechanism", "downscaling"]
    },
    "neuroscience": {
        "topic": "neural decoding transformer brain computer interface",
        "description": "Neuroscience — decode neural signals with sequence models",
        "expected_concepts": ["transformer", "transfer learning", "few-shot", "spike sorting"]
    }
}

DOMAIN = "drug_discovery"  # AGENT: change to any key above

CONFIG = {
    "topic": DOMAINS[DOMAIN]["topic"],
    "max_papers": 100,
    "years_back": 5,
    "gap_threshold": 0.3,
    "top_hypotheses": 10,
    "embedding_model": "all-MiniLM-L6-v2",
    "random_seed": 42
}

random.seed(CONFIG["random_seed"])
np.random.seed(CONFIG["random_seed"])
model = SentenceTransformer(CONFIG["embedding_model"])
print(f"[Step 1] Domain: {DOMAIN} — {DOMAINS[DOMAIN]['description']}")
print(f"[Step 1] Topic: {CONFIG['topic']}")
```

**Expected output**:
```
[Step 1] Domain: drug_discovery — Biomedical — identify novel compound-target pairs
[Step 1] Topic: drug target interaction prediction deep learning
```

---

## Step 2: Retrieve Literature

(same as v1.1)

---

## Step 3-5: Build Graph, Score Gaps, Generate Hypotheses

(same as v1.1)

---

## Multi-Domain Quick Switch
```python
DOMAIN = "physics"  # or: drug_discovery, economics, climate, neuroscience
CONFIG["topic"] = DOMAINS[DOMAIN]["topic"]
# Then re-run Steps 1-6 identically
```

## Expected Outputs by Domain

| Domain | Papers | Concepts | Gaps | Top gap example |
|---|---|---|---|---|
| drug_discovery | ~85 | ~130 | ~220 | graph neural ↔ allosteric binding |
| physics | ~70 | ~100 | ~180 | reinforcement learning ↔ error syndrome |
| economics | ~75 | ~110 | ~190 | large language ↔ instrumental variable |
| climate | ~80 | ~120 | ~200 | conformal prediction ↔ ensemble model |
| neuroscience | ~65 | ~95 | ~160 | transformer ↔ spike sorting |

## Validation Checklist
- [ ] Retrieved >= 50 papers from 2+ sources
- [ ] Knowledge graph >= 50 nodes, >= 100 edges
- [ ] All hypotheses include >= 2 supporting papers
- [ ] gap_score values in range [0, 1]
- [ ] Output JSON is valid and includes domain field
- [ ] No duplicate concept pairs

*Co-authored with Claw for Claw4S 2026 Conference.*