SkillsBench · SquadOS
Method and limitations
Collected results with per-round scores and published criteria. Each task has its own ranking; different samples and environments remain identified.
Sample and ranking
Each row represents a model, execution tool and reasoning configuration. Ranking requires every unique, valid round specified by the manifest, within the same task and campaign. Valid zero scores are included. Invalid, missing and unstarted runs are separated. This subset is not the full SkillsBench benchmark.
Scores and inconclusive evidence
The score is the mean percentage of passed criteria across valid rounds. Delivery, OCR, research and audio correspondence reviews use preserved evidence only. Inconclusive criteria produce a range from the verified minimum to the possible maximum; ordering by the minimum does not resolve overlapping ranges. Original scores and binary rewards are preserved in the details. These ranges are not 95% confidence intervals.
Cost, time and cache
Estimated API cost uses public rates frozen during collection, including cache when verified. It is not necessarily the amount paid for a subscription or the Studio route charge. The table shows mean cost, median agent time and total tokens across valid rounds. Incomplete costs have no comparable value; known failure subtotals are not treated as totals. HTTP 200, timeouts or absent records do not establish zero usage.
Environments and comparison limits
Collection spanned environment revisions and repairs, identified per run. Having 3/3 rounds does not mean every run used the same build. The one-run-per-task history is exploratory; the pilot remains archived. Campaigns are not pooled and low scores are not replaced by new attempts. Few rounds and specific tasks do not establish general superiority of a model or tool.
Source and updates
This publication is a frozen snapshot of collected results, not a live test. The public JSON includes configurations, scores, reasons, revisions and available result hashes. Internal logs and session identifiers are not published. Diagnostic tests after repairs are excluded from historical cohorts. New data requires a new export and publication.