Legal knowledge-base migration

The problem

A legal knowledge base lived inside a helpdesk product the company was retiring. Each article carried PDF attachments, images embedded in its body, links to other articles, a place in a topic tree, an author-and-dates record, and the groups allowed to read it. All of it had to arrive in SharePoint intact. The alternative was weeks of copying and pasting, with the permissions wrong somewhere in the middle.

What I did

I designed and built a thirteen-stage Python pipeline against the Microsoft Graph and SharePoint APIs. It splits at the network boundary: the first stages pull everything into a local archive, and the later stages push from that archive into SharePoint. The extraction can be re-run without touching the target and the import without touching the source. Every stage is separately idempotent, and every bulk run appends to a per-record log, so what happened to one article on one run is a search, not an investigation.

All 1,029 articles were imported in one bulk run, with 82 embedded images rehosted and relinked, cross-links rewritten and 901 permission scopes applied. It is in production, with a scoped re-import pending after the department's own review. Since then it has grown a Power Platform layer: flows and Power Apps that let the legal team create a new section, page, library and permissions without a developer.

Each page needed an author and last-updated block. Stored inside the page's text, it survived the import and then vanished, because SharePoint sanitises that content on every save in the editor. I moved the data out of the page body into library columns that a built-in web part displays. Editing a page can no longer touch the metadata, because the metadata is not in the page.

A decision and its cost

The pipeline signs in as a person, not as an application. A human has to start each run, so it cannot be scheduled unattended. In exchange there is no secret to store or rotate, the pipeline can do exactly what its operator could do by hand, and every action is attributable. For a migration of sensitive legal content that is the right side of the trade.

Order mattered for security as well as correctness. Per-item permissions must be applied before the site-level grant; the other way round, the whole knowledge base would have been readable by the entire company for the length of the run. That constraint cannot be inferred from the code, so it is written in bold at the top of the runbook.

What I would do differently

I introduced one bug the department caught: the topic index was derived from the articles, which silently dropped 96 topics that held no article but had been created on purpose. The fix reads the full taxonomy instead. The lesson I kept is that the expensive bugs here were all platform behaviour, not logic, which is why I verified results through the API.