This project introduces Reinforcement Learning from Community Feedback (RLCF), a novel training paradigm designed to imbue AI with "scientific taste"—the ability to discern and propose high-impact research ideas. Targeting AI researchers and developers aiming to enhance scientific discovery capabilities in AI, RLCF offers a method to align AI's output with community-valued scientific judgment, moving beyond mere executive capability.

How It Works

RLCF operates in three stages, leveraging large-scale community signals as supervision. First, it constructs community preference data by converting paper citations into pairwise preference signals, matching papers within similar fields and publication periods. Second, a "Scientific Judge" model, a generative reward model trained with GRPO on this preference data, learns to predict which paper in a pair is likely to have higher impact. Third, a "Scientific Thinker" policy model is optimized using comparison-based GRPO, employing the Scientific Judge as its reward model to generate novel research ideas with higher potential impact. The core components are the Scientific Judge (a reward model) and the Scientific Thinker (an ideation policy), both trained on the SciJudgeBench dataset, comprising 696,758 preference pairs derived from 2.1 million arXiv papers.

Quick Start & Requirements

• Models are available on Hugging Face [ Collection ].
• An online demo is accessible [ Demo ].
• Specific installation and dependency details are not explicitly provided in the README, but training and inference likely require significant computational resources, including GPUs, given the LLM-based approach.

Highlighted Details

• The Scientific Judge model demonstrates strong performance, with Scientific Judge-Qwen3-30B surpassing GPT-5.2, GLM-5, and Gemini 3 Pro on in-domain evaluation (80.6 vs. 75.7).
• Learned scientific judgment exhibits significant generalization capabilities, extending to future-year papers (+55.1 points over base), ICLR peer-review preferences, and cross-field evaluations (bioRxiv).
• The Scientific Thinker policy achieves a 54.2% average win-rate against leading LLMs, significantly outperforming its base policy (30.3%) in generating impactful research ideas.
• The SciJudgeBench dataset is substantial, containing 696,758 preference pairs across Computer Science, Mathematics, Physics, and other scientific fields.

Maintenance & Community

The project has recently released news in March 2026, including an online demo and paper availability on arXiv. Models are published on Hugging Face. The paper lists 19 authors, indicating a research-driven development effort. No specific community channels like Discord or Slack are mentioned.

Licensing & Compatibility

This project is licensed under the Apache License 2.0. This permissive license generally allows for commercial use and integration into closed-source projects without significant restrictions.

Limitations & Caveats

The README notes that the currently released training and test sets are updated versions, and results reported in the paper were based on earlier data; updated results are forthcoming. Additionally, larger models utilizing newer architectures are still under development. The project's focus is specifically on scientific taste learning for research papers, not general AI capabilities.