Extracting an edX Course From a mongodump Archive (Without a Full Restore)

The server was gone; a 49 GB gzipped dump was the only surviving copy of the course. Here is the exact procedure that got everything out — outline, HTML, and every uploaded file.

edXMigration Published 2026-08-29 · by the Sternfast engineering team

An edX course lives in three Mongo collections — modulestore.active_versions, modulestore.structures, and modulestore.definitions — plus uploaded files in contentstore GridFS (fs.files and fs.chunks). Restore just those with mongorestore --nsInclude, walk the published structure through fields.children, resolve each block's definition, then reassemble GridFS chunks sorted by n. No running edX platform is required.

The Open edX server no longer exists. What's left is a 49 GB gzipped mongodump --archive --gzip of the whole Mongo instance — and somewhere inside it, the only surviving copy of a course that needs to be rebuilt on Moodle™.

If you got here by googling "extract edx course from mongodump archive" or "edx contentstore gridfs export", you've probably already discovered that almost nothing is written about this. The standard advice is to restore the full dump into a matching Tutor deployment and use the platform's own export tools. That means provisioning a matching MongoDB version, enough disk for the whole instance, a working Open edX install, and a lot of hours — to recover one course.

You don't need any of it.

An edX course is three modulestore collections and a GridFS bucket. You can pull all of it out of the archive with mongorestore, a plain local Mongo, and a few dozen lines of pymongo. We did exactly this recently — a course headed from a dead Open edX (Tutor) instance to Moodle — and this post is the full procedure, including the part where our first inventory of the course was wrong.

The mechanism

Where does edX actually store the course?

Modern Open edX uses the split modulestore, which spreads one course across three MongoDB collections:

modulestore.active_versions — maps org/course/run to the ObjectId of the current published structure (and the draft one, which you can usually ignore).
modulestore.structures — the block tree. Every chapter, sequential, vertical, and leaf block, each with a block_type, a fields.display_name, a fields.children list, and a definition reference.
modulestore.definitions — the per-block content: the actual HTML markup, video ids, PDF urls.

Uploaded files — the PDFs, spreadsheets, images that instructors dragged into Studio — are not in the modulestore. They live in the contentstore, which is MongoDB GridFS: fs.files holds one metadata document per file, fs.chunks holds the binary content in ordered pieces, and each file is keyed by an asset locator that embeds the org, course, and run.

One course, four places: three modulestore collections for structure and content, GridFS for every uploaded file.

Everything else in that 49 GB — sessions, forum posts, the other courses — is dead weight for this job. Which is the whole point of the next step.

Step 1

How do I restore only what I need from the archive?

A mongodump --archive is one sequential stream. There's no index, no table of contents, no seeking to a collection. That has one unavoidable consequence and one very useful one: mongorestore must read and decompress every byte of the archive no matter what you ask for — but with --nsInclude it only writes the collections that match.

mongorestore --archive=dump.gz --gzip \
  --nsInclude="*modulestore*" \
  --nsInclude="*fs.files*" --nsInclude="*fs.chunks*"

Run this against a plain local MongoDB — nothing edX-related installed. The modulestore collections are a sliver of the archive; the GridFS collections are most of it, but you need them for the files anyway.

--nsInclude filters the writes, not the read. You pay for the full pass through the archive either way — so pass every pattern you need in one run.

Learn from our sequencing mistake. We ran the modulestore-only restore first, explored the structure, and only then went back for GridFS — which meant reading all 49 GB through gunzip a second time. The read cost is fixed per pass. Include every pattern the first time.

Step 2

How do I turn structures and definitions into a course outline?

Start from active_versions, take the published structure id, load that structure, and walk the tree from its root. On a Tutor install the database is named openedx. Note that the dot in modulestore.active_versions is part of the collection name — use bracket access in pymongo so there's no ambiguity.

from pymongo import MongoClient

db = MongoClient()["openedx"]   # Tutor's default database name

av = db["modulestore.active_versions"].find_one(
    {"org": "YourOrg", "course": "YourCourse", "run": "2024_T1"})
struct = db["modulestore.structures"].find_one(
    {"_id": av["versions"]["published-branch"]})

blocks = {(b["block_type"], b["block_id"]): b for b in struct["blocks"]}

def walk(block_key, depth=0):
    b = blocks[tuple(block_key)]
    name = b["fields"].get("display_name", "")
    print("  " * depth + b["block_type"] + "  " + name)
    for child in b["fields"].get("children", []):
        walk(child, depth + 1)

walk(struct["root"])

That prints the entire course skeleton — chapters, sequentials, verticals, leaf blocks — in order, with display names. It's the outline you'll rebuild from, and it's also your master inventory (more on why that matters below).

For the actual content, resolve each leaf block's definition:

defn = db["modulestore.definitions"].find_one({"_id": b["definition"]})
html = defn["fields"].get("data", "")   # an html block's full markup

Now the part that connects structure to files. A "Resources" tab in edX is just an HTML block whose links point at /static/<file>. Those filenames are the asset keys into GridFS. Grep the definition HTML for /static/ and collect every filename you find — that list is exactly the set of binaries the course depends on.

One warning from the trenches: the display name of a link and its href often differ. Trust the href for the asset key, always.

Step 3

How do I get the PDFs and spreadsheets out of GridFS?

GridFS sounds exotic. It's two ordinary collections: fs.files has one document per file (filename, length, content type, upload date), and fs.chunks has the binary content split into ordered pieces, each with a files_id pointing back at its file and an n giving its position.

Before touching anything, build the index — a restored dump won't necessarily have it, and without it every per-file chunk lookup is a full scan of your biggest collection:

db["fs.chunks"].create_index([("files_id", 1), ("n", 1)])

Then reassembly is: find the chunks, sort by n, concatenate. That's the entire trick.

for f in db["fs.files"].find():
    chunks = db["fs.chunks"].find({"files_id": f["_id"]}).sort("n", 1)
    data = b"".join(c["data"] for c in chunks)
    with open(outdir + "/" + f["filename"], "wb") as out:
        out.write(data)

(pymongo's gridfs module does the same thing once the index exists — the manual loop just makes the mechanism visible, and gives you a place to filter and verify.)

In a multi-course instance, don't extract everything: the asset locator that keys each file embeds the org, course, and run, so filter fs.files down to your course before looping. Or work from the other direction and fetch only the filenames your /static/ scan collected in step 2.

GridFS is metadata in fs.files plus ordered binary pieces in fs.chunks. Sort, join, verify.

Verification

How do I know the extracted files are intact?

Two cheap checks catch nearly every reassembly mistake:

assert len(data) == f["length"]        # byte-exact vs fs.files metadata
assert data[:5] == b"%PDF-"            # pdf magic bytes
assert data[:2] == b"PK"               # xlsx/docx (zip container)

The length comparison against fs.files.length proves you got every chunk and got them in order — a missing or misordered chunk can't survive it. The magic bytes prove the first chunk really is the start of the file format the filename claims.

Then cross-check in both directions: every /static/ reference from the structure walk should have a matching extracted file, and every extracted file should appear somewhere in a definition. An orphan on either side means you missed a block or pulled another course's asset.

Finally — open a sample by hand. Byte-perfect and actually-opens are the same thing in our experience, but it takes ten seconds to be sure.

Hard-won

What burned our time

The interaction tables lie about what's in the course. Our first inventory came from the SQL/student-interaction side — counting the block types the platform had recorded activity against. It said the course had 2 PDFs. The modulestore structure walk then turned up a Resources tab full of Excel financial models that no interaction table had any reason to mention — learners download those files, they don't "interact" with them, so they leave almost no footprint. If we'd rebuilt from the interaction inventory, the most valuable content in the course would have been silently dropped. Always inventory from modulestore.structures, never from usage data.

Smaller ones, in the order they cost us minutes:

Link text vs href. Display names drift out of sync with the files behind them as instructors re-upload. The href under /static/ is the truth; the visible label is decoration.
The dot is part of the name. modulestore.active_versions is one collection, not a modulestore namespace — reach for db["modulestore.active_versions"] and matching --nsInclude wildcards accordingly.
Draft vs published. active_versions carries both branches. For a rebuild you almost always want published-branch; the draft structure can reference half-finished blocks that were never live.
Set expectations on the read. The sequential stream means even the "selective" restore is a full multi-hour pass over 49 GB. Nothing is wrong. Let it run.

If the extraction is the easy half

Everything above gets the course out of the dump — that part is genuinely doable in an afternoon once you know where the pieces live, which is why we wrote it down. The longer half is usually what comes next: turning that outline, HTML, and pile of verified binaries into a working course on the destination LMS, with nothing dropped and someone accountable for checking. That's the job our LMS migration service exists for — edX to Moodle included, dead-server recoveries included. If you're staring at an archive like this one, send us a note with the org/course/run and roughly what state the dump is in, and we'll tell you honestly whether you need us or just this post. We reply within one business day.

Quick answers

Questions people ask about this

Can I extract an edX course from a mongodump without installing Open edX?

Yes. All you need is a plain MongoDB instance and mongorestore. The course structure lives in three modulestore collections and the uploaded files in GridFS, and both can be read with a few dozen lines of pymongo — no Tutor, no edX platform code, no version matching.

Which MongoDB collections hold course content in Open edX?

The split modulestore uses three: modulestore.active_versions maps org/course/run to the published structure id, modulestore.structures holds the block tree, and modulestore.definitions holds each block's actual content. Uploaded files like PDFs and spreadsheets live separately in the contentstore GridFS collections fs.files and fs.chunks.

Does mongorestore --nsInclude still read the whole archive?

Yes. A mongodump --archive is one sequential stream, so mongorestore has to read and decompress every byte even when it only writes a few collections. Budget wall time for a full pass over the archive, and pass every --nsInclude pattern you need in a single run so you only pay that cost once.

Where does edX store uploaded PDFs and other course files?

In the contentstore, which is MongoDB GridFS. fs.files holds one metadata document per file with its filename and exact byte length, and fs.chunks holds the binary content split into ordered pieces. Each file is keyed by an asset locator that embeds the org, course, and run, which is how you isolate one course's files in a multi-course dump.

Why does my extracted edX course seem to be missing files?

Probably because you inventoried the course from the student-interaction side — SQL tables or tracking logs — instead of the modulestore. Content that learners merely download, like a resources page full of files, leaves almost no interaction footprint. Walk the published structure in modulestore.structures and resolve every definition; that is the only complete inventory of the course.

Can an edX course be rebuilt on Moodle from this kind of extraction?

Yes. The structure walk gives you the exact chapter and unit hierarchy, the definitions give you the HTML and video references, and the GridFS pass gives you every uploaded file, all of which map onto Moodle sections, pages, and resources. It is manual assembly rather than an automated import, but nothing needs to be lost.