chore: track claude skills, tools, templates, reference code and research-wiki
- Add all claude skills (brainstorming, commit, debugging, TDD, etc.) - Add claude hooks (pre-commit-guard, post-edit-quality) - Add research templates (experiment plan, research brief, etc.) - Add claude tools (arxiv/semantic_scholar/openalex fetch, wiki, exa) - Add TRM4 reference implementation as algorithm fidelity baseline - Add research-wiki content (plans, index, graph, query_pack) - Update .gitignore to exclude .graphify_version runtime state
This commit is contained in:
@@ -0,0 +1,141 @@
|
||||
---
|
||||
name: novelty-check
|
||||
description: "Verify the novelty of research ideas. GPT cross-validation. Trigger phrases: novelty check, has anyone done this, check novelty."
|
||||
argument-hint: [method-or-idea-description]
|
||||
---
|
||||
|
||||
# Novelty Verification
|
||||
Verify novelty of: $ARGUMENTS
|
||||
|
||||
## Goal
|
||||
Perform a strict check on whether a method, idea, or experimental setting is actually new. The default stance is skepticism, not help-seeking for supporting evidence.
|
||||
|
||||
## Working Principles
|
||||
- Brutally honest: do not relax the standard just to make something look new.
|
||||
- `Applying X to Y` is not novel by default unless the application produces an unexpected mechanism, theoretical explanation, or clearly different experimental phenomenon.
|
||||
- Check the novelty of both the `METHOD` and the `EXPERIMENTAL SETTING`.
|
||||
- If the method itself is not new, but the findings, conclusions, experimental setup, or failure analysis are new, state that distinction explicitly.
|
||||
- Always search the last 6 months of arXiv.
|
||||
- Do not rely on titles alone; read the abstract and, when necessary, the key parts of related work, intro, method, and appendix.
|
||||
|
||||
## Workflow
|
||||
|
||||
### Phase A: Extract Core Claims
|
||||
First break the user's method description into 3-5 core technical claims. Each one should be as specific as possible.
|
||||
|
||||
For each claim, answer:
|
||||
- What is the method?
|
||||
- What problem does it solve?
|
||||
- What is the key mechanism?
|
||||
- What is the essential difference from an obvious baseline?
|
||||
|
||||
Rewrite the story-like description into searchable technical propositions and avoid vague phrasing.
|
||||
|
||||
### Phase B: Multi-source Literature Search
|
||||
Run multi-source retrieval for each claim, prioritizing recent work and similar settings.
|
||||
|
||||
For each claim, try at least 3 search-query sets, and make them complementary:
|
||||
- Direct technical terms
|
||||
- Synonyms / abbreviations / related task names
|
||||
- "Problem + mechanism" combinations
|
||||
- "Method + dataset / setting" combinations
|
||||
|
||||
#### Required Search Channels
|
||||
1. WebSearch: arXiv / Google Scholar / Semantic Scholar / conference homepages
|
||||
2. `python3 .claude/tools/arxiv_fetch.py search "QUERY" --max 10`
|
||||
3. `python3 .claude/tools/semantic_scholar_fetch.py search "QUERY" --max 10`
|
||||
4. `python3 .claude/tools/exa_search.py search "QUERY" --max 10` (if available)
|
||||
5. `python3 .claude/tools/openalex_fetch.py search "QUERY" --max 10` (if available)
|
||||
|
||||
#### Search Priorities
|
||||
- ICLR / NeurIPS / ICML 2025-2026
|
||||
- arXiv preprints from the last 6 months
|
||||
- Papers close to the method mechanism, not only papers on the same task
|
||||
- Papers close to the experimental setting, not only papers using the same method
|
||||
|
||||
#### Decision Strategy
|
||||
- Record potentially overlapping papers first; do not exclude them too early
|
||||
- Prefer reading the abstract, intro, related work, and method sections
|
||||
- If overlap looks suspicious, also read the experimental setup and appendix
|
||||
|
||||
#### Recording Requirements
|
||||
For each candidate paper, record:
|
||||
- Title
|
||||
- Year
|
||||
- Venue / status
|
||||
- Relevant point
|
||||
- The specific reason it may overlap
|
||||
- Why it might still be a different work
|
||||
|
||||
If a data source is unavailable, explicitly record the fallback reason and continue with the others; do not stop the task.
|
||||
|
||||
### Phase C: GPT Cross-Validation
|
||||
Send the method description from Phase A and all candidate papers found in Phase B to `/codex:rescue --fresh --wait` for a second review.
|
||||
|
||||
The cross-validation prompt must include:
|
||||
- proposed method description
|
||||
- the full candidate paper list
|
||||
- ask:
|
||||
- `Is this method novel?`
|
||||
- `What is the closest prior work?`
|
||||
- `What is the delta?`
|
||||
|
||||
Use high reasoning effort.
|
||||
|
||||
The goal of cross-validation is not to find even more papers. It is to force out the closest prior art, the smallest difference, and the risk of pseudo-novelty.
|
||||
|
||||
### Phase D: Output Report + Wiki Integration
|
||||
The output must be in English and follow a fixed structure.
|
||||
|
||||
#### Report Format
|
||||
```markdown
|
||||
## Novelty Check Report
|
||||
### Method Under Review
|
||||
### Core Claims
|
||||
- Claim 1: ... (novelty: high / medium / low; closest paper: ...)
|
||||
- Claim 2: ... (novelty: high / medium / low; closest paper: ...)
|
||||
|
||||
### Recent Prior Work
|
||||
| Paper | Year | Venue / Status | Overlap Point | Key Difference |
|
||||
|---|---:|---|---|---|
|
||||
|
||||
### Overall Assessment
|
||||
- score X/10
|
||||
- recommendation: continue / continue cautiously / abandon
|
||||
- key differentiator: ...
|
||||
- positioning advice: ...
|
||||
```
|
||||
|
||||
#### Evaluation Scale
|
||||
- `high`: current search shows no close prior art, and the difference is concrete and technical
|
||||
- `medium`: there is related prior work, but there is still a clear and defensible technical delta
|
||||
- `low`: mostly a reorganization of known methods, task switching, dataset switching, hyperparameter changes, or standard engineering changes
|
||||
|
||||
#### Wiki Integration
|
||||
If the project has `research-wiki/`, also ingest the knowledge there:
|
||||
- Create a `claim` entity for each core claim
|
||||
- Create a `paper` entity for each newly found paper
|
||||
- Add claim-paper / paper-paper relation edges
|
||||
- Rebuild `query_pack`
|
||||
|
||||
Prefer existing tools such as `.claude/tools/research_wiki.py`; if the wiki does not exist, skip silently and do not error.
|
||||
|
||||
Ingestion rules:
|
||||
- Only write high-confidence information
|
||||
- Claim names should be short, stable, and reusable
|
||||
- Edges must include evidence; do not create empty links
|
||||
|
||||
## Completion Criteria
|
||||
Only finish when all of the following are complete:
|
||||
1. 3-5 core claims have been extracted
|
||||
2. Multi-source search has been completed, including the last 6 months of arXiv
|
||||
3. Candidate paper abstracts have been read, and related work / method sections were read when necessary
|
||||
4. GPT cross-validation has been completed
|
||||
5. A structured English report has been produced
|
||||
6. If `research-wiki/` exists, the corresponding writes and `query_pack` rebuild have been completed
|
||||
|
||||
## Failure Handling
|
||||
- Missing tools: record the missing item and degrade gracefully
|
||||
- Too few search results: expand synonyms, abbreviations, higher-level terms, and experimental settings
|
||||
- Too many search results: prefer the most recent, most similar, and most likely overlapping work
|
||||
- Conflicting evidence: read the original abstract and method sections first; do not rely on intuition
|
||||
Reference in New Issue
Block a user