Contributing

Curos publishes its evaluation tooling and much of its research openly. Anyone can contribute, from fixing a small bug in an eval harness to proposing a new eval. This page explains the projects, how to get involved, and what the process looks like.

What is open

• Evaluation tooling, including the harnesses behind OversightQA and Curos-Helpfulness.
• Eval datasets that are safe to publish, with personal or sensitive content removed.
• Documentation and reproducibility notebooks for published research.

Model weights themselves are not open. Curos keeps its frontier work internal; the open projects are the evaluation and research layer around it.

Ways to contribute

• Report issues. Try the eval harnesses and file detailed bug reports with reproduction steps.
• Fix bugs. Small, well-scoped fixes are the fastest way to start.
• Propose new evals. If you see a gap in how models are tested, write a proposal describing what you would measure and why.
• Improve datasets. Suggest additions or corrections to published eval datasets.
• Write documentation. The eval harnesses need clear guides for new contributors.

Getting started

• Read the contributing guide in the evaluation repository before opening an issue or pull request. It covers coding style, testing expectations, and the review process.
• Start with issues labeled “good first issue.” They are scoped so you can complete them without deep knowledge of the codebase.
• Join the contributor channel in the Community forum. Maintainers answer questions there and tag new work as it appears.
• Follow the Contributor Covenant code of conduct. All maintainers act as enforcers for reported violations.

Review process

All contributions are reviewed by maintainers, and by the Safety & Evaluations Committee when they touch evaluation methodology:

• Pull requests need tests and a short description of what changed and why.
• Proposals for new evals go through the committee and must document how the eval would be scored and how it avoids teaching to the test.
• Dataset contributions must include provenance and a check that no sensitive personal data is included.
• Contributions are acknowledged publicly in the release notes. There is no financial compensation, and no contributor owns the work; everything is published under the project’s license.

What happens to good ideas

Promising proposals, even without a complete implementation, are tracked and discussed. The committee reviews them on a quarterly cycle, and accepted evals are integrated into the release gates for future models. If your proposal is accepted, you are listed as a contributor in the eval report.

All documentation →