Who it is for

Move documentation into Markdown without cleanup

Migrating documentation usually means converting the HTML and then spending longer removing what the converter kept. The removal is what Clean Clipper was built around, and the part that was measured: zero leftover HTML tags across 512 pages.

The cleanup after the conversion

The conversion itself takes a second. What takes the week is everything that comes with it – the left nav repeated at the top of every file, the version picker, the “on this page” rail, the feedback widget, the breadcrumb and eight columns of footer. Multiply by two hundred pages and a migration becomes a find-and-replace project with a different set of selectors for every source site.

Then the parts you needed most arrive damaged. Code fences lose their language, so nothing is highlighted and a reader cannot tell a shell command from a JSON body. Tables with a list in a cell fall apart. Admonitions become orphaned paragraphs with nothing to mark them as warnings. Fixing that is not search-and-replace work; it is reading every page again.

The part nobody plans for is the audit afterwards. Six weeks in, somebody asks whether a paragraph in the new site was in the old one or was written during the migration, and if the converted files never carried the address they came from, that question has no answer short of finding the original page – which on a docs site that has since been switched off means finding nothing. A source line in every imported file costs one line and settles it, and a date line settles which version was imported.

What maps cleanly

• Headings, lists, tables, code fences and footnotes all map to standard Markdown.
• Code fences keep their language tag: 73 of 156 kept on the technical corpus, against 20 and 0 for the engines compared.
• Zero leftover HTML tags across 512 measured pages.
• Navigation sidebars, version pickers and “on this page” rails are cut, not converted.
• Admonitions and callouts become blockquotes, because Markdown has no standard syntax for them.
• A filename template and a subfolder keep an imported set organised as it grows.
• Every file carries the address it came from, so an imported paragraph can still be traced to its source page during review.
• Footnotes are collected before the sanitiser strips the element ids they depend on – an ordering detail that decides whether reference links survive at all.

## Workflow

Django can create migrations for you. Make changes to your models, then run:

```bash
python manage.py makemigrations
```

> **Note:** migrations are files on disk. Commit them with your code.

| Command         | What it does                              |
| --------------- | ----------------------------------------- |
| `makemigrations`| Writes new migrations from model changes  |
| `migrate`       | Applies migrations to the database        |

Setting it up for a migration

Two settings, one folder and one working rule about commits. The commit rule is the part that makes the review possible later.

1. Create an import/ directory in the docs repository, separate from where the finished pages will live. Raw conversions and edited pages should never be in the same folder while a migration is running.
2. Open Options from the extension icon, point the destination at import/, and set what the icon click does to “save to folder”. Thirty pages is thirty keystrokes; it should not also be thirty windows.
3. Set the filename template to {domain}-{title}. An import that draws from two or three source sites collides on titles constantly, and the domain is what keeps “Overview” from becoming overview-3.
4. Turn on title, source and extraction in the frontmatter. source is what answers the audit question six weeks later; extraction tells you which pages came from structured data rather than from what rendered.
5. Leave images as links rather than skipping them. You are not going to keep those URLs, but the link is the inventory – it records that the page had a diagram, which is what you need when you plan the asset work.
6. Before clipping a page with tabs or accordions, open them. The extension converts what the browser has rendered, and a tab whose content is inserted on click is not in the DOM until it has been clicked.
7. Commit the raw import as one commit, then restructure in the commits after it. That way every later diff shows your editorial changes rather than a mixture of your changes and the conversion’s.

Settings for an import

These are chosen for a set of files that will be reviewed by a person, page by page, and then edited – not for a folder that gets read once and forgotten.

## Installing

> **Warning:** upgrading across two major versions at once is not supported.

npm

```bash
npm install example-cli --save-dev
```

pnpm

```bash
pnpm add -D example-cli
```

Both tabs were rendered in the DOM, so both came across, one after the other.
A tab that renders only when clicked would not have.

Three migrations

Thirty pages of vendor documentation

A partner’s API reference has to live in your docs. You clip thirty pages into import/, one keystroke each. The code fences arrive tagged: on the technical corpus the language tag survived in 73 of 156 fences against 20 and 0 for the engines compared, and the left navigation, version picker and feedback widget arrive nowhere at all.

What you then edit is prose and structure. What you do not edit is the two hundred lines of sidebar per page that a straight HTML conversion would have deposited at the top of every file, which is the actual reason migrations take weeks.

A guide built out of admonitions

The source guide leans heavily on warning, note and tip callouts. Markdown has no standard syntax for any of them, so all three become blockquotes – the content survives, the distinction between the three types does not.

That is worth knowing before you start rather than after, because re-tagging by type is a manual pass and its size depends on the source. Reading the raw import for blockquotes is how you scope that pass, and it is one grep rather than a page-by-page reading.

Salvage before a site is switched off

A product is being retired and its documentation goes offline at the end of the month. There is no crawler here, so this is a page at a time – but each page arrives as a file with its original address in the header, which is what makes the salvage worth anything afterwards.

Images are the part to plan separately. They stay links to a site that is about to stop existing, so the pages that matter need their diagrams saved by hand before the deadline; the link inventory in the imported files is the checklist for that.

Against the other ways to convert

A migration usually ends up using two of these together. The honest comparison is about where the manual work lands, not whether there is any.

What needs fixing after an import

All my callouts look the same now

Warnings, notes and tips all become blockquotes, because Markdown has no standard admonition syntax to map them onto. The text is intact and the emphasis marker at the start of the line usually survives, so a warning that began with a bold “Warning:” still says so. Re-tagging by type is a pass you plan for; the size of that pass is one grep for blockquotes away.

A page came in with no heading at the top

Empty headings are dropped, and on many documentation sites the visible page title is not a heading at all but a navigation element outside the article body. The title still reaches the file through the title frontmatter field, which is taken from the page’s own metadata. Promoting it to an H1 is a mechanical step you can script across the import.

The image links still point at the old site

They will. The extension downloads no binary assets, so every image is a link to where it was, and an imported set is not self-contained. Treat the links as an inventory rather than as a result – they tell you which pages have assets and how many, which is exactly the list the asset migration needs.

A page recorded jsonld-articlebody and reads differently from the site

That page shipped its article text in structured data and never finished rendering it, so the body came from the structured data instead. The two are not always the same – a site that updates its rendered page more often than its structured data will leave an older version in that block. Those pages are the ones to read against the original before you commit them.

What it is not

It is not a migration tool: no crawler, no link rewriting, no redirect map, no asset download. Images stay links to the original site, so an imported set is not self-contained. Anything Markdown cannot express (tabbed code blocks, includes, admonition types, custom components) is flattened to the nearest plain equivalent, so the content survives and the styling does not. And it converts what the page has rendered, so text inside an unopened accordion is not in the DOM and is not clipped.

Questions