--- name: novelty-check description: "Verify the novelty of research ideas. GPT cross-validation. Trigger phrases: novelty check, has anyone done this, check novelty." argument-hint: [method-or-idea-description] --- # Novelty Verification Verify novelty of: $ARGUMENTS ## Goal Perform a strict check on whether a method, idea, or experimental setting is actually new. The default stance is skepticism, not help-seeking for supporting evidence. ## Working Principles - Brutally honest: do not relax the standard just to make something look new. - `Applying X to Y` is not novel by default unless the application produces an unexpected mechanism, theoretical explanation, or clearly different experimental phenomenon. - Check the novelty of both the `METHOD` and the `EXPERIMENTAL SETTING`. - If the method itself is not new, but the findings, conclusions, experimental setup, or failure analysis are new, state that distinction explicitly. - Always search the last 6 months of arXiv. - Do not rely on titles alone; read the abstract and, when necessary, the key parts of related work, intro, method, and appendix. ## Workflow ### Phase A: Extract Core Claims First break the user's method description into 3-5 core technical claims. Each one should be as specific as possible. For each claim, answer: - What is the method? - What problem does it solve? - What is the key mechanism? - What is the essential difference from an obvious baseline? Rewrite the story-like description into searchable technical propositions and avoid vague phrasing. ### Phase B: Multi-source Literature Search Run multi-source retrieval for each claim, prioritizing recent work and similar settings. For each claim, try at least 3 search-query sets, and make them complementary: - Direct technical terms - Synonyms / abbreviations / related task names - "Problem + mechanism" combinations - "Method + dataset / setting" combinations #### Required Search Channels 1. WebSearch: arXiv / Google Scholar / Semantic Scholar / conference homepages 2. `python3 .claude/tools/arxiv_fetch.py search "QUERY" --max 10` 3. `python3 .claude/tools/semantic_scholar_fetch.py search "QUERY" --max 10` 4. `python3 .claude/tools/exa_search.py search "QUERY" --max 10` (if available) 5. `python3 .claude/tools/openalex_fetch.py search "QUERY" --max 10` (if available) #### Search Priorities - ICLR / NeurIPS / ICML 2025-2026 - arXiv preprints from the last 6 months - Papers close to the method mechanism, not only papers on the same task - Papers close to the experimental setting, not only papers using the same method #### Decision Strategy - Record potentially overlapping papers first; do not exclude them too early - Prefer reading the abstract, intro, related work, and method sections - If overlap looks suspicious, also read the experimental setup and appendix #### Recording Requirements For each candidate paper, record: - Title - Year - Venue / status - Relevant point - The specific reason it may overlap - Why it might still be a different work If a data source is unavailable, explicitly record the fallback reason and continue with the others; do not stop the task. ### Phase C: GPT Cross-Validation Send the method description from Phase A and all candidate papers found in Phase B to `/codex:rescue --fresh --wait` for a second review. The cross-validation prompt must include: - proposed method description - the full candidate paper list - ask: - `Is this method novel?` - `What is the closest prior work?` - `What is the delta?` Use high reasoning effort. The goal of cross-validation is not to find even more papers. It is to force out the closest prior art, the smallest difference, and the risk of pseudo-novelty. ### Phase D: Output Report + Wiki Integration The output must be in English and follow a fixed structure. #### Report Format ```markdown ## Novelty Check Report ### Method Under Review ### Core Claims - Claim 1: ... (novelty: high / medium / low; closest paper: ...) - Claim 2: ... (novelty: high / medium / low; closest paper: ...) ### Recent Prior Work | Paper | Year | Venue / Status | Overlap Point | Key Difference | |---|---:|---|---|---| ### Overall Assessment - score X/10 - recommendation: continue / continue cautiously / abandon - key differentiator: ... - positioning advice: ... ``` #### Evaluation Scale - `high`: current search shows no close prior art, and the difference is concrete and technical - `medium`: there is related prior work, but there is still a clear and defensible technical delta - `low`: mostly a reorganization of known methods, task switching, dataset switching, hyperparameter changes, or standard engineering changes #### Wiki Integration If the project has `research-wiki/`, also ingest the knowledge there: - Create a `claim` entity for each core claim - Create a `paper` entity for each newly found paper - Add claim-paper / paper-paper relation edges - Rebuild `query_pack` Prefer existing tools such as `.claude/tools/research_wiki.py`; if the wiki does not exist, skip silently and do not error. Ingestion rules: - Only write high-confidence information - Claim names should be short, stable, and reusable - Edges must include evidence; do not create empty links ## Completion Criteria Only finish when all of the following are complete: 1. 3-5 core claims have been extracted 2. Multi-source search has been completed, including the last 6 months of arXiv 3. Candidate paper abstracts have been read, and related work / method sections were read when necessary 4. GPT cross-validation has been completed 5. A structured English report has been produced 6. If `research-wiki/` exists, the corresponding writes and `query_pack` rebuild have been completed ## Failure Handling - Missing tools: record the missing item and degrade gracefully - Too few search results: expand synonyms, abbreviations, higher-level terms, and experimental settings - Too many search results: prefer the most recent, most similar, and most likely overlapping work - Conflicting evidence: read the original abstract and method sections first; do not rely on intuition