We scored Microsoft's MCP course against a quality rubric

LearnOS has a command that scores a course out of 100. It exists because we import a lot of courses — from markdown, directory trees, URL crawls, YouTube transcripts — and we needed to know which ones were worth reading before anyone had read them. No engagement data, no ratings, no completion rates. Just the text.

So when we ported Microsoft's two open-source curricula, the obvious thing was to point the rubric at them.

The scores

MCP for Beginners          86.0 / 100  [A]
  67 lessons · 14 modules · 70,673 words · ~322 min

AI Agents for Beginners    90.8 / 100  [A]
  20 lessons · 7 modules · 33,718 words · ~153 min

Both are good, and that is worth saying plainly: these are carefully made courses. Every lesson in both carries source frontmatter. Every lesson has a Resources section. Neither has a single lesson with zero citations, which is rarer than it sounds — it is the first thing that collapses when a course is generated rather than written.

The scores are also not the interesting part. Where they lost points is.

Where the points went

The rubric has six categories. Both courses maxed out two of them — citations and retrieval readiness — and both lost most of their points in the same two places.

Practice. MCP for Beginners has a practice or exercises section in 21 of 67 lessons. AI Agents for Beginners has one in 1 of 20. This is the single largest deduction in both courses, and it is the most defensible one: these are reference curricula built around code samples in a repo, not around graded exercises. The rubric is measuring something the authors did not set out to build.

Module balance. MCP for Beginners spreads 67 lessons across 14 modules with a standard deviation of 6.0 against a mean of 4.8 — a coefficient of variation of 1.25. In plain terms: some modules have two lessons and one has more than twenty. That is what happens when a repo grows by accretion, and it is real. The syllabus is genuinely lumpy to read. AI Agents for Beginners, which is younger and smaller, sits at a cv of 0.35.

The sharpest difference between the two is in code:

MCP for Beginners: 613 code blocks, 8.67 per thousand words, 68 of them runnable. AI Agents for Beginners: 81 code blocks, 2.40 per thousand words, none runnable.

MCP for Beginners scores lower overall but is far denser and more executable. AI Agents for Beginners is tighter prose — average sentence length 17.3 words against 29.7 — and scores better on writing quality as a result.

That inversion is the most useful thing the rubric produced. A single composite number said the smaller course was better. The categories said the two courses are good at different things, and which one you want depends on whether you are reading to understand or reading to build.

What a score like this is actually for

The honest limitation: this rubric measures properties of text, not learning. It cannot tell you whether a course explains things well, whether its examples land, or whether anyone finishes it. A course could score 95 and teach nothing. Every category is a proxy, and the composite is a weighted sum of proxies.

What it is good for is triage. When you import fifty courses, you need to know which three to read first and which one has no citations anywhere in it. The rubric answers that in seconds, from the markdown alone, before a single person has opened the course. That is a narrow job, and it does it.

It is also a check on ourselves. We build tooling for generating courses with agents. A generated course that scores well on citations and badly on practice is telling you exactly what the generator is bad at — and in our experience, practice and module balance are precisely where agent-written courses fall down first.

Read them yourself

Both courses are on LearnOS in full, free, MIT-licensed, with Microsoft's repository as the canonical upstream. You can read them in the browser, or pull them into Claude Code, Cursor, or Codex over MCP and ask questions against the text with citations back to the source section.

The scoring command ships with the CLI:

learnos course quality "MCP for Beginners" --verbose