A rich API to your markdown files in seconds.
An open JS library to turn markdown files into structured, queryable data (SQL and JSON). Build rich markdown-powered sites fast and reliably.
Rich metadata extracted including frontmatter, links and more.
Lightweight and fast indexing 1000s of files in seconds.
Open source and extensible via plugin system.
Built with❤by
Datopian
Features
Key Advantages of MarkdownDB
- Power of plain text:
- Combination of unstructured content and structured data in simple Markdown files.
- Simplicity at core:
- Turn your Markdown files into a queryable, lightweight SQL database.
- Flexible and extendable:
- Bring your own document types, extend your frontmatter with computed fields and check for errors with with custom validations.
- Simple API:
- Get a list of all or some Markdown files, filter them by frontmatter fields, and more.
- Do one thing well:
- MarkdownDB just gives you a database and an API - a super-powerful and extensible way to create those from markdown.
- Open source:
- Your content isn't locked away in proprietary platforms. It's open, it's free, it's yours.
Features
Key Features of MarkdownDB
Task extraction
Extract tasks from markdown files easily
1- [x] do laundry1
2{
3 ...
4 "metadata": {
5 ...
6 "tasks": [{desription: "do laundry", checked: true}]
7 },
8 ...
9},
10 Frontmatter fields extraction
Extract structured markdown data from your markdown frontmatter.
1---
2title: Example Post
3date: 2023-01-01
4---
5
6# Content1
2{
3 ...
4 "metadata": {
5 ...
6 "title": "Example Post",
7 "date": "2023-01-01"
8 },
9 ...
10}
11 Tags extraction
Extract tags from markdown body and from `tags` frontmatter field.
1---
2tags: frontmatter_tag_1, frontmatter_tag_2
3---
4# Some heading
5#body_tag
6
7Lorem ipsum #tag1 #tag2 #tag31
2{
3 ...
4 "metadata": {
5 ...
6 "tags": ["frontmatter_tag_1", "frontmatter_tag_2"]
7 },
8 "tags": ["body_tag", "frontmatter_tag_1", "frontmatter_tag_2" ]
9 ...
10}
11 Computed Fields
Compute additional metadata fields on the fly with custom functions.
1
2 const addTitle = (fileInfo, ast) => {
3 // Find the first header node in the AST
4 const headerNode = ast.children.find((node) => node.type === "heading");
5
6 // Extract the text content from the header node
7 const title = headerNode
8 ? headerNode.children.map((child) => child.value).join("")
9 : "";
10
11 // Add the title property to the fileInfo
12 fileInfo.title = title;
13 };
14
15 client.indexFolder({
16 "folderPath": "PATH_TO_FOLDER",
17 "customConfig": {
18 "computedFields": [addTitle]
19 }
20 })
21 1
2{
3 ...
4 title: "Example Markdown File"
5 ...
6}
7 Our vision
Unified Content Management
Imagine a world where Markdown isn’t just text – it's a source of structured and unstructured data. With MarkdownDB, we aim to balance the simplicity and accessibility of writing in Markdown with the ability to query your collection of markdown files like a database – think get me all files "with type Blog" or "all documents created in the last week" or "all documents with 'hello world' in the title" or find "all tasks (i.e. - [ ]) in all documents with ⏭️ emoji in the task".
Quickstart
Start using MarkdownDB in just a few steps
You have a folder of markdown content
For example, your blog posts. Each file can have a YAML frontmatter header with metadata like title, date, tags, etc.
1---
2title: My first blog post
3date: 2021-01-01
4tags: [a, b, c]
5author: John Doe
6---
7
8# My first blog post
9
10This is my first blog post.
11I'm using MarkdownDB to manage my blog posts.
12 Index the files with MarkdownDB
Use the npm `mddb` package to index Markdown files into an SQLite database. This will create a `markdown.db` file in the current directory.
# npx mddb <path-to-folder-with-your-md-files>
npx mddb ./blogQuery your files with SQL...
E.g. get all the files with with tag `a`.
SELECT files.*
FROM files
INNER JOIN file_tags ON files._id = file_tags.file
WHERE file_tags.tag = 'a'...or using MarkdownDB Node.js API in a framework of your choice!
Use our Node API to query your data for your blog, wiki, docs, digital garden, or anything you want!
1// Next.js example
2import React from "react";
3import clientPromise from "@/lib/mddb.mjs";
4
5
6export default function Blog({ blogs }) {
7 return (
8 <div>
9 <h1>Blog</h1>
10 <ul>
11 {blogs.map((blog) => (
12 <li key={blog.id}>
13 <a href={blog.url_path}>{blog.title}</a>
14 </li>
15 ))}
16 </ul>
17 </div>
18 );
19}
20
21export const getStaticProps = async () => {
22 const mddb = await clientPromise;
23 // get all files that are not marked as draft in the frontmatter
24 const blogFiles = await mddb.getFiles({
25 frontmatter: {
26 draft: false
27 }
28 });
29
30 const blogsList = blogFiles.map(({ metadata, url_path }) => ({
31 ...metadata,
32 url_path,
33 }));
34
35 return {
36 props: {
37 blogs: blogsList,
38 },
39 };
40};Build your blog, wiki, docs, digital garden, or anything you want
And share it with the world!
Roadmap
What's new and what's coming
🎉 MarkdownDB released! 🎉
First version of the package released under `@flowershow/markdowndb` with basic functionalities, like indexing files into an SQLite database, extracting frontmatter data and basic JS API.
Version 0.1.0
We added support for extracting and querying forward links and backlinks.
Version 0.2.0
We fixed some annoying bugs and renamed package from `@flowershow/markdowndb` to just `mddb`.
Version 0.3.0
Support for querying files by frontmatter field values.
Version 0.4.0
Add Tags Extraction from Markdown Content.
Version 0.5.0
Add tasks extraction from files. e.g - [ ] task
Version 0.6.0
Implement JSON output to disk
Version 1.0.0
Support for custom document types and computed fields.
Start using MarkdownDB today.
Check out our tutorial and learn the basics of using MarkdownDB in the command line and in the Node.js project.