optimization on tomrochette.com

EvoSkill

tom@tomrochette.com (Tom Rochette) — Tue, 13 Oct 2026 00:00:00 +0000

EvoSkill is Sentient Labs’ Apache-2.0 skill-discovery toolkit: it runs your coding agent over a benchmark, hands the failed trajectories to a proposer, and materializes the proposed fixes as standard SKILL.md skill folders, keeping only the candidates that improve a held-out validation score.

It is the third optimizer-family column in this category, and its difference from SkillOpt and GEPA is the unit of evolution: where they train or rewrite one text parameter, EvoSkill evolves the whole agent configuration (prompt plus skill set) and can discover skills nobody wrote.

What it is
#

A Python 3.12+ CLI (installed by script or pip install -e .) that wraps Claude Code, Codex, OpenCode, OpenHands, Goose, or Harbor as the executor, with any model provider behind it (Anthropic, OpenAI, OpenRouter, Fireworks, and more). The loop runs three roles: an Executor attempts task batches with the current configuration, a Proposer reads the failed trajectories and proposes one targeted skill, and a Skill Builder writes it as a markdown folder under .claude/skills/. Selection is a Pareto frontier of the top configurations, each pinned as a git branch, so the evolutionary history is a git log you can check out, and a candidate survives only when it improves the held-out score. Apache-2.0, from Sentient Labs (the sentient-agi org), built on the March 2026 paper (arXiv 2603.02766).

Status
#

Quiet since late August: 1,250 stars, 132 forks, and 21 open issues as of 2026-10-13, created 2026-03-04, pushed 2026-08-24, with the release train stalled at v1.3.0 (2026-06-16, the Fireworks provider addition). The paper and the March 5 blog post are the whole public record, and my searches found no Hacker News story and no independent coverage of the project, so the footprint is the repository itself. The paper’s headline numbers are the authors’ own runs: OfficeQA 60.6% to 67.9% exact match with Claude Code (Opus 4.5) on a 10% training subset, SealQA 26.6% to 38.7%, and a 5.3-point zero-shot transfer from SealQA skills to BrowseComp.

Strengths
#

Failure-driven search: proposals are grounded in recorded execution trajectories, so each candidate skill answers a demonstrated capability gap rather than a random mutation.
The held-out validation gate is the same discipline that makes SkillOpt trustworthy, and the git-branch frontier makes every accepted and rejected candidate inspectable.
The output is ordinary SKILL.md folders, so induced skills install anywhere the Agent Skills standard reaches.
Cross-model and cross-agent by construction, and the transfer result (SealQA skills improving BrowseComp without modification) is the interesting evidence that skill-level optimization produces general capabilities.

Cautions
#

The project has been quiet for roughly seven weeks (last push 2026-08-24) with one release since June, so treat it as a research artifact that may be ending rather than a maintained tool.
Zero community footprint: no Hacker News thread, no independent review, and no third-party benchmark runs found as of 2026-10-13.
You need a benchmark to run it at all; the README lists benchmark-free evolution as an open direction, so teams without scored tasks get nothing from the loop.
The README credits the GEPA lineage through the org’s own gepa-plus repository rather than the canonical gepa-ai/gepa, so the ancestry is partly self-referential.
All accuracy gains are self-reported on two QA benchmarks, and the coding-agent framing of the README’s title runs ahead of what the paper measures.

Pricing
#

Free and open source under Apache-2.0. There is no hosted tier; evolution runs consume your own model API budget across many scored rollouts, which is the whole cost.

Compared to
#

SkillOpt: trains one reusable skill document against scored tasks with a strict edit budget; EvoSkill evolves the whole configuration and discovers new skills, with more freedom and less control.
GEPA: the reflective optimizer over any text parameter; EvoSkill extends that feedback-driven idea to joint prompt-plus-skill mutation and adds frontier-of-configurations selection.
Agent Skills open standard: the format EvoSkill’s output lands in; the toolkit induces folders, the standard carries them.

Bottom line
#

Recommended for teams that already hold a scored task set worth mining and want to see what skills a machine would write from their failures. Not for production skill maintenance (the loop is offline over a fixed dataset and the project is quiet), and not as evidence that auto-generated skills beat hand-written ones, which its benchmarks do not test.

Changes
#

2026-10-13 - Created from the skills entrant resolution (the mapped awesome-list scan), with the zero-footprint search and the stalled release train recorded as the critical angles.

References
#

https://github.com/sentient-agi/EvoSkill - repository, Apache-2.0, 1,250 stars, 132 forks, 21 open issues, pushed 2026-08-24, install paths and supported agents (GitHub API and README, fetched 2026-10-13)
https://arxiv.org/abs/2603.02766 - the paper: the three-role loop, the Pareto frontier, and the OfficeQA, SealQA, and BrowseComp numbers (fetched 2026-10-13)
https://www.sentient.xyz/blog/evoskill-automated-skill-induction-from-agent-failures - the Sentient Labs post: the five-step loop, the git-branch frontier, and the Claude Code Opus 4.5 claim (March 5, 2026, fetched 2026-10-13)
https://api.github.com/repos/sentient-agi/EvoSkill/releases/latest - v1.3.0 published 2026-06-16, the newest release as of 2026-10-13 (fetched 2026-10-13)
https://hn.algolia.com/api/v1/search?query=EvoSkill&tags=story - the zero-hit search behind the missing-footprint statement (fetched 2026-10-13)

Kapso

tom@tomrochette.com (Tom Rochette) — Sun, 11 Oct 2026 00:00:00 +0000

Kapso is Leeroo’s MIT-licensed framework of long-running agents that run optimization campaigns over AI and data systems, turning a stated objective and an evaluator into a tree of coding-agent experiments whose lessons are distilled into a knowledge hub for the next campaign.

Kapso is this category’s first member whose unit of work is the campaign rather than the paper or the answer: the loop is judged by evaluator outcomes at every step, its memory is evidence-priced (a lesson stays trusted only as long as it keeps holding up), and its headline results sit on leaderboards, two of which the project runs itself.

What it is
#

A Python package (pip install leeroo-kapso, Python 3.10+, created 2025-12-07) by Leeroo, a YC X25 company. It runs its inference through coding-agent CLIs with no direct-API fallback: Claude Code for ideation and implementation, Codex for research and judging, with an optional OpenAI key only for embeddings. Four documented pillars: .evolve() runs tree-searched experiments toward the objective, .learn() and .learn_knowledge() mine finished campaigns and outside repositories into knowledge cards, .research() does web research for ideas and references, and .deploy() ships the result to local, Docker, or Modal infrastructure. The technical report (arXiv 2601.21526, January 2026, by Alireza Nadafian, Alireza Mohammadshahi, and Majid Yazdani) describes a git-native experimentation engine where every attempt is a branch, a knowledge system over repositories and papers, and a cognitive memory layer distilling lessons from run logs, diffs, and evaluator feedback. Leeroopedia, the project’s hosted ML knowledge wiki, connects as an MCP server during campaigns.

Status
#

Active and accelerating: 123 stars, 15 forks, created 2025-12-07, last pushed 2026-10-12, with six releases on October 11 alone taking the train from v0.6.3 to v0.7.5 (the October 12 pushes are Leeroopedia documentation commits), as of 2026-10-13. The v0.7.0 release added cost-aware candidate selection, an estimator prices each finalized candidate before it runs, refusals and measured costs are recorded as campaign events, and the campaign stops when the next estimate would exhaust its budget, plus a vertex_claude_code agent that drives Claude Code on Vertex AI open-weight models; the checkpoint schema moved to version 3, so campaigns started before 0.7.0 cannot resume.

The claimed results are concrete: #1 open-source on MLE-Bench, top ranking on ALE-Bench (1909 Elo against ALE Agent’s 1879), RelBench wins over KumoRFM-v2 on Stanford’s official leaderboard, and the IOAI 2026 AI Model Track, where the project reports a 536.07 total above all 471 human contestants and a Grand Master Trophy among the 14 founding AI participants, with IOAI’s own account of the inaugural track published September 29, 2026. The community footprint is nearly absent: a Hacker News search for the project returns zero stories as of 2026-10-11, the 123 stars are small for the claims, and the PyPI package named kapso is an unrelated WhatsApp tool that shadows this one in installs, a collision the README itself warns about.

Strengths
#

The judge is declared with the objective: every campaign carries its own evaluator, so progress is measured rather than self-declared, the Karpathy keep-or-revert property generalized to arbitrary ML and data objectives.
The learning layer is the differentiator: lessons carry the evidence that earned them and expire when they stop holding, and outside repositories and papers fold into the same knowledge graph.
Third-party anchoring where it exists: the RelBench results sit on Stanford’s official leaderboard, and IOAI’s organizers published their own review of the track the IOAI result comes from.
It ships as a library rather than a demo: a PyPI package, documented pillars, and agent skill files for Claude Code, Codex, and OpenCode.

Cautions
#

The benchmark headline claims are self-run: the MLE-Bench and ALE-Bench numbers live in the project’s own benchmark directories, and Stanford’s RelBench leaderboard page states plainly that it holds self-reported results submitted in good faith, pointing to RelArena for controlled evaluation.
121 stars, zero Hacker News stories, and no independent replication surfaced in this run’s searches, a thin footprint for results this strong.
The loop rides on paid coding-agent subscriptions (Claude Code and Codex logins are required before anything works), and Leeroopedia adds a hosted-service account and prepaid credits to the dependency list.
The name collision is genuine adoption friction: the unrelated kapso WhatsApp package on PyPI shadows this one, and every Hacker News hit for the name is the other company.
The IOAI totals come from the company’s own benchmark writeup, with IOAI’s account of the track but no independent audit of the scores.

Pricing
#

Free and open source under MIT; there is no paid tier, so pricing does not apply. Costs are the coding-agent subscriptions the CLIs require (Claude Code, Codex), your compute, and an optional Leeroopedia account whose new accounts come with $20 of credit.

Compared to
#

Karpathy Autoresearch: the keep-or-revert ancestor with one fixed metric; Kapso generalizes the measured-judge idea to campaign objectives and adds memory across campaigns.
ARIS: the markdown-skill ML research loop with cross-model review; Kapso is a Python framework with tree search and a knowledge graph instead of skills files.
DeepAnalyze: the trained 8B model carrying the data-science loop in its weights; Kapso orchestrates general coding agents and learns between runs.

Bottom line
#

Recommended for ML and data teams that want benchmark-scored optimization campaigns with a persistent, evidence-priced memory, and for researchers studying how loops learn across runs. Not for anyone needing independently audited headline numbers or a large user community, both absent as of 2026-10-12.

Changes
#

2026-10-11 - Created.
2026-10-11 - Added the star history chart.
2026-10-12 - Recorded the October 11 release burst (v0.7.0 through v0.7.5): cost-aware candidate selection with budget caps and measured-cost ranking, the vertex_claude_code agent on Vertex AI open-weight models, and the schema-3 checkpoint break.
2026-10-13 - Harmonized the star count (the status line and the footprint line disagreed, 122 versus 121) to 123 as of 2026-10-13, and identified the October 12 pushes as Leeroopedia documentation commits.

References
#

https://github.com/Leeroo-AI/kapso - repository, MIT license, benchmark claims, and release train (as of 2026-10-11)
https://raw.githubusercontent.com/Leeroo-AI/kapso/main/README.md - the four pillars, CLI requirements, PyPI collision warning, and the IOAI and RelBench claims (fetched 200, 2026-10-11)
https://arxiv.org/abs/2601.21526 - the technical report: git-native experimentation engine, knowledge system, and cognitive memory layer (fetched 200, 2026-10-11)
https://pypi.org/pypi/leeroo-kapso/json - package version 0.7.5, MIT license, and Python requirement (fetched 200, 2026-10-12)
https://ioai-official.org/what-happens-when-autonomous-ai-takes-on-the-same-tasks-as-the-worlds-top-young-ai-talents/ - IOAI’s own account of the inaugural AI Model Track, September 29, 2026 (fetched 200, 2026-10-11)
https://star-project.stanford.edu/relbench/leaderboard/ - the official RelBench leaderboard and its self-reported-results disclaimer (fetched 200, 2026-10-11)
https://raw.githubusercontent.com/Leeroo-AI/kapso/main/benchmarks/ioai2026/README.md - the per-task IOAI numbers behind the 536.07 total and Grand Master Trophy (fetched 200, 2026-10-11)
https://hn.algolia.com/api/v1/search?query=leeroo-kapso - the zero-story Hacker News footprint (fetched 200, 2026-10-11)

optimization on tomrochette.com

EvoSkill

What it is
#

Status
#

Strengths
#

Cautions
#

Pricing
#

Compared to
#

Bottom line
#

Changes
#

See also
#

References
#

Kapso

What it is
#

Status
#

Strengths
#

Cautions
#

Pricing
#

Compared to
#

Bottom line
#

Changes
#

See also
#

References
#

optimization on tomrochette.com

EvoSkill

What it is #

Status #

Strengths #

Cautions #

Pricing #

Compared to #

Bottom line #

Changes #

See also #

References #

Kapso

What it is #

Status #

Strengths #

Cautions #

Pricing #

Compared to #

Bottom line #

Changes #

See also #

References #

What it is
#

Status
#

Strengths
#

Cautions
#

Pricing
#

Compared to
#

Bottom line
#

Changes
#

See also
#

References
#

What it is
#

Status
#

Strengths
#

Cautions
#

Pricing
#

Compared to
#

Bottom line
#

Changes
#

See also
#

References
#