Project ProtoStar: make AI research reusable before chasing a higher score

October 29, 2026. Project ProtoStar is a new effort to make publicly funded research easier for people and AI systems to reuse. Its first practical test is a climate challenge. For teams considering participation, the most valuable deliverable may be a reproducible experiment that another researcher can actually run.

The launch announcement brings together universities, Hugging Face, the Astera Institute and the Open Research Community Accelerator. It focuses on publishing scientific datasets, models and code in usable formats, then testing reuse through open challenges. Carnegie Mellon University hosts the first challenge, scheduled for November 2–13, 2026.

What the first challenge asks

CMU’s Climate Risk Intelligence challenge page currently describes forecasts for 210 cities using historical observations through 2018. It separates a live score for 2019–2021 from a final score for 2022–2025, and lists historical-average and linear-trend baselines. At the time of this check, the leaderboard was marked as forthcoming.

Check the current specification before building: the launch blog discusses predictions through 2026, while the challenge interface says 2019–2025. These pages do not yet agree on the final forecast horizon. This article reports the difference rather than resolving it by assumption; confirm the operative requirements with the organizers before submitting.

The challenge page asks for forecasts, method documentation, code, a policy idea and feedback about obstacles to reuse. It also allows a declared negative result: a documented investigation of why reuse or reproduction failed. See the current challenge instructions for submission and judging details.

Make the data handoff your first experiment

Here is a proposed preparation workflow, not a claim that we have run the challenge. Start by asking a colleague to load a small sample using only your README. Watch where they need information that exists only in your head. Every unexplained column, unit or filtering decision is a possible source of disagreement later.

1. Record the exact inputs. Save the repository revision, filenames and retrieval date. If the dataset changes, your results should still point to the inputs you used.
2. Describe the meaning of a row. Specify whether it represents a station-day, a city-year or another unit. Explain how missing observations and station changes are handled.
3. Run the simple comparison first. Implement the stated baseline before adding a complex model. Match the prediction target, dates and units so the comparison answers one question.
4. Keep a decision log. Record which years you inspected while choosing parameters. Keep the final evaluation period out of that selection process.
5. Package the failure evidence. If a file cannot be decoded or a calculation cannot be reconstructed, preserve the smallest example that demonstrates the problem.

A fictional example illustrates the stakes. Suppose one file reports a temperature in degrees Fahrenheit and another expects Celsius. Both columns may be named “temperature,” and both may contain plausible numbers. Joining them successfully is not evidence that the analysis is correct. A tiny example with the expected conversion can reveal a mistake before a larger forecasting run makes it expensive to diagnose.

Turn “the dataset is confusing” into useful feedback

A strong reuse report gives the maintainer enough information to reproduce the obstacle. Replace “the dates are wrong” with a specific observation: which file and revision you used, which two fields disagree, what interpretation you tried and how the ambiguity changes your result.

For your own project, adapt this original reporting brief:

Resource and revision: [identifier]. Intended use: [one sentence]. Reproduction steps: [minimal sequence]. Expected result: [observable outcome]. Actual result: [error or mismatch]. Evidence: [small example]. Proposed repair: [documentation, metadata or code change]. Remaining uncertainty: [what needs the maintainer’s answer].

That record can be valuable even when an ambitious AI approach does not improve the baseline. It identifies a fix the next team can use. Avoid turning a failed local setup into a sweeping claim that the entire dataset is unusable; the evidence should support the scope of the conclusion.

A forecast score is one part of a research contribution

Our analysis is that ProtoStar’s reuse focus creates a useful second question alongside predictive accuracy: can another team inspect, rerun and extend the work? A small improvement with clear provenance may be easier to build on than a larger unexplained gain. Keep the policy proposal similarly grounded: distinguish what the experiment measured from the additional evidence a city would need before acting.

For related context, our guide to interpreting AI climate simulations explains why model results need careful boundaries. ProtoStar has just been announced; it has not yet demonstrated that participating AI systems outperform the challenge’s baselines.

Illustrative photograph: storm clouds by Steve Gribble / Unsplash, under the Unsplash License. This is photography, not a challenge forecast or model output.

Sources

• Project ProtoStar launch announcement — October 29, 2026; initiative, partners and planned challenges.
• CMU Climate Risk Intelligence challenge — current task, evaluation and submission instructions, checked October 29, 2026. Forecast years differ from the launch blog as noted above.