From e97ceec52280915915e16441be848fc3a2a45c78 Mon Sep 17 00:00:00 2001 From: Michele Mesiti Date: Thu, 19 Mar 2026 17:29:34 +0100 Subject: [PATCH 1/4] tools: table with comparison to be discussed. The table tries to summarise, but too much summarisation might just result in ambiguous/debatable/wrong statements --- content/tools.md | 196 ++++++++++++++++++++++++++++++++--------------- 1 file changed, 136 insertions(+), 60 deletions(-) diff --git a/content/tools.md b/content/tools.md index fce8a5a8..ac2b2d1a 100644 --- a/content/tools.md +++ b/content/tools.md @@ -11,7 +11,84 @@ --- -## In-code documentation +## Documentation Tools: comparison + +```{list-table} Comparison of the tools for documentation we have discussed so far +:widths: 20 10 10 10 10 10 10 15 +:header-rows: 1 +:stub-columns: 1 + +* - Type + - Convenient + - Easy + - Maintainabile + - Searchable + - Readable + - LLM-friendly + - Notes +* - in-code doc + - βœ…βœ… + - 🟨 + - βœ…πŸŸ¨ + - 🟨 + - ❌ + - βœ…πŸŸ¨ + - ❌for users +* - README + - βœ… + - βœ… + - βœ…πŸŸ¨ + - 🟨 + - βœ… + - βœ… + - typically enough +* - HTML Generators + - 🟨 + - ❌ + - βœ…πŸŸ¨ + - βœ… + - ❌ + - βœ…βœ… + - powerful +* - Wikis + - 🟨 + - βœ… + - ❌❌ + - βœ… + - βœ… + - ❌ + - βœ…for non-programmers +* - Latex + - 🟨(?) + - ❌ + - ❌🟨 + - 🟨 + - βœ… (?) + - ❌ + - βœ…Physics/Math, + ❌copy/paste +* - Jupyter + - 🟨 + - 🟨/❌ + - βœ…βœ… + - 🟨 (?) + - βœ… + - 🟨 + - βœ… validation tooling +``` + +What do we mean? +- **Convenience**: for programmers who live in code. +- **Easiness**: how easy is is to contribute and set up? +- **Maintainability** is good for those tools that can be version-controlled along with the code. + It is even better if it is easy to check automatically that the information is correct + (*does the output of a snippet of code match what is shown in the docs?*) +- **Searchability:** How easy is it to find the information we need? +- **Readability**: Can the documentation be rendered in a way that makes it easy to read? +- **LLM-friendliness**: how easy is to feed this documentation to an LLM? + + +### In-code documentation - Comments, function docstrings, ... - Advantages @@ -24,8 +101,7 @@ For a closer look at this see the {ref}`in-code-documentation` episode. --- - -## README files +### README files - Advantages - Versioned (goes with the code development) @@ -39,44 +115,31 @@ For a closer look at this see the {ref}`writing-readme-files` episode. --- -## Plain Text formats: reStructuredText and Markdown - -```markdown -# This is a section in Markdown This is a section in RST - ======================== - -## This is a subsection This is a subsection - -------------------- - -Nothing special needed for Nothing special needed for -a normal paragraph. a normal paragraph. +### Wikis - :: +- Popular solutions (but many others exist): + - [MediaWiki](https://www.mediawiki.org) + - [Dokuwiki](https://www.dokuwiki.org) +- Advantage + - Barrier to write and edit is low +- Disadvantages + - Typically disconnected from source code repository (**reproducibility**) + - Difficult to serve multiple versions + - Difficult to check out a specific old version + - Typically needs to be hosted and maintained - This is a code block This is a code block -**Bold** and *emphasized*. **Bold** and *emphasized*. -A list: A list: -- this is an item - this is an item -- another item - another item +--- -There is more: images, There is more: images, -tables, links, ... tables, links, ... -``` - -- Two of the most popular lightweight markup languages. -- reStructuredText (RST) has more features than Markdown but the choice is a matter of taste. -- There are (unfortunately) [many flavors of Markdown](https://github.com/jgm/CommonMark/wiki/Markdown-Flavors). -- Motivation to stick to a standard text-based format: **They make it easier to move the documentation to other tools - which also expect a standard format, as the project/organization grows**. -- We use [MyST](https://myst-parser.readthedocs.io/en/latest/) - flavored Markdown in the {ref}`sphinx` episode and the - {ref}`gh-pages` example. -- Nice resource to learn Markdown: [Learn Markdown in 60 seconds](https://commonmark.org/help/) -- [Pandoc](https://pandoc.org/) can convert between MD and RST (and many other formats). +### LaTeX/PDF +- Advantage + - Popular and familiar in the physics and mathematics community +- Disadvantages + - PDF format is not ideal for copy-pasting of examples + - Possible, but not trivial to automate rebuilding documentation after every Git push --- @@ -95,7 +158,7 @@ These tools offer some or all of these features: (if you can download ) - **Validation**: check that the code snipped in the documentation match the real behaviour of the code. -- **Continous checks**: regenerate automatically every time you save, so that you can catch errors early +- **Continuous checks**: regenerate automatically every time you save, so that you can catch errors early ````{tabs} ```{group-tab} Python @@ -129,7 +192,7 @@ These tools offer some or all of these features: - Uses RMarkdown and a LaTeX-like syntax - **Search:** - client-side (Javascript that runs in browser - fuse.js) - - also typically avaiable in RStudio + - also typically available in RStudio Long-Form Documentation for R is typically contained in [vignettes](https://r-pkgs.org/vignettes.html). @@ -192,12 +255,8 @@ These tools offer some or all of these features: ``` ```` -```{discussion} - -Do you know an awesome tool or feature that should be in this list? -Let us know! (Open a PR) +--- -``` ## Hosting Documentation on the Web @@ -210,37 +269,54 @@ GitHub, GitLab, and Bitbucket make it possible to serve HTML pages: and can be [connected](https://docs.readthedocs.com/platform/latest/reference/git-integration.htm) to common software forges. + + +```{discussion} + +Do you know an awesome tool or feature that should be in this list? +Let us know! (Open a PR) + +``` + --- -## Wikis +### Plain Text formats: reStructuredText and Markdown -- Popular solutions (but many others exist): - - [MediaWiki](https://www.mediawiki.org) - - [Dokuwiki](https://www.dokuwiki.org) -- Advantage - - Barrier to write and edit is low -- Disadvantages - - Typically disconnected from source code repository (**reproducibility**) - - Difficult to serve multiple versions - - Difficult to check out a specific old version - - Typically needs to be hosted and maintained +```markdown +# This is a section in Markdown This is a section in RST + ======================== +## This is a subsection This is a subsection + -------------------- +Nothing special needed for Nothing special needed for +a normal paragraph. a normal paragraph. + :: ---- + This is a code block This is a code block -## LaTeX/PDF -- Advantage - - Popular and familiar in the physics and mathematics community -- Disadvantages - - PDF format is not ideal for copy-pasting of examples - - Possible, but not trivial to automate rebuilding documentation after every Git push +**Bold** and *emphasized*. **Bold** and *emphasized*. +A list: A list: +- this is an item - this is an item +- another item - another item +There is more: images, There is more: images, +tables, links, ... tables, links, ... +``` ---- +- Two of the most popular lightweight markup languages. +- reStructuredText (RST) has more features than Markdown but the choice is a matter of taste. +- There are (unfortunately) [many flavors of Markdown](https://github.com/jgm/CommonMark/wiki/Markdown-Flavors). +- Motivation to stick to a standard text-based format: **They make it easier to move the documentation to other tools + which also expect a standard format, as the project/organization grows**. +- We use [MyST](https://myst-parser.readthedocs.io/en/latest/) + flavored Markdown in the {ref}`sphinx` episode and the + {ref}`gh-pages` example. +- Nice resource to learn Markdown: [Learn Markdown in 60 seconds](https://commonmark.org/help/) +- [Pandoc](https://pandoc.org/) can convert between MD and RST (and many other formats). ```{keypoints} From 6827c37bad741d2802124ec26a7e4a6f0e73957c Mon Sep 17 00:00:00 2001 From: Michele Mesiti Date: Wed, 13 May 2026 10:17:59 +0200 Subject: [PATCH 2/4] mention mkdocs drama in footnote, more keypoints --- content/tools.md | 76 +++++++++++++----------------------------------- 1 file changed, 20 insertions(+), 56 deletions(-) diff --git a/content/tools.md b/content/tools.md index ac2b2d1a..506284ac 100644 --- a/content/tools.md +++ b/content/tools.md @@ -11,6 +11,7 @@ --- + ## Documentation Tools: comparison ```{list-table} Comparison of the tools for documentation we have discussed so far @@ -41,7 +42,7 @@ - 🟨 - βœ… - βœ… - - typically enough + - typically enough * - HTML Generators - 🟨 - ❌ @@ -80,7 +81,8 @@ What do we mean? - **Convenience**: for programmers who live in code. - **Easiness**: how easy is is to contribute and set up? -- **Maintainability** is good for those tools that can be version-controlled along with the code. +- **Maintainability** is good for those tools that can be version-controlled + along with the code. It is even better if it is easy to check automatically that the information is correct (*does the output of a snippet of code match what is shown in the docs?*) - **Searchability:** How easy is it to find the information we need? @@ -88,62 +90,9 @@ What do we mean? - **LLM-friendliness**: how easy is to feed this documentation to an LLM? -### In-code documentation - -- Comments, function docstrings, ... -- Advantages - - Good for programmers - - Version controlled alongside code - - Can be used to auto-generate documentation for functions/classes -- Disadvantage - - Probably not enough for users of the code - -For a closer look at this see the {ref}`in-code-documentation` episode. - --- -### README files - -- Advantages - - Versioned (goes with the code development) - - It is often good enough to have a `README.md` or `README.rst` along with your code/script -- If you use README files, use either - [RST](https://docutils.sourceforge.net/rst.html) or - [Markdown](https://commonmark.org/help/) -- A great guide to README files: [MakeaREADME](https://www.makeareadme.com/) - -For a closer look at this see the {ref}`writing-readme-files` episode. - ---- - -### Wikis - -- Popular solutions (but many others exist): - - [MediaWiki](https://www.mediawiki.org) - - [Dokuwiki](https://www.dokuwiki.org) -- Advantage - - Barrier to write and edit is low -- Disadvantages - - Typically disconnected from source code repository (**reproducibility**) - - Difficult to serve multiple versions - - Difficult to check out a specific old version - - Typically needs to be hosted and maintained - - ---- - -### LaTeX/PDF - -- Advantage - - Popular and familiar in the physics and mathematics community -- Disadvantages - - PDF format is not ideal for copy-pasting of examples - - Possible, but not trivial to automate rebuilding documentation after every Git push - - ---- - ## HTML static site generators There are many tools generate documentation @@ -175,10 +124,21 @@ These tools offer some or all of these features: - Full-text server-side on [Read the docs](https://about.readthedocs.com) - **Validation**: via [doctest](https://docs.python.org/3/library/doctest.html) - - [MkDocs](https://www.mkdocs.org/): A Markdown-first static site generator. + - [MkDocs](https://www.mkdocs.org/): A Markdown-first static site generator + (with a vast system of plugins developed independently). - **API Reference generation**: via [mkdocstrings](https://mkdocstrings.github.io/) - **Search:** search plugin for client-side (Javascript that runs in the browser - lunr.js) + Project now (as of 2026) not maintained [^mkdocsdrama] + + [^mkdocsdrama]: After somewhat dramatic events (2026), + MkDocs 1.x is now superseded by [Zensical](https://zensical.org/), + which tries to keep compatibility wiht MkDocs 1.x. + (MkDocs 2.0 is also being developed + but projects and plugins based on 1.x + will break). + + - [Doxygen](https://www.doxygen.nl/): - **API Reference generation**: has also support for Python @@ -320,5 +280,9 @@ tables, links, ... tables, links, ... ```{keypoints} +- READMEs are typically a good starting point - Some popular solutions make reproducibility and maintenance of multiple code versions difficult. +- The landscape of tools is very diversified and every community has their own favourite. +- The basic functionality of all Static site generators is very similar, + but specific aspects (API ref generation, search, validation) differ. ``` From 8612eda7882fb709d0ccaeca24f112ede306beea Mon Sep 17 00:00:00 2001 From: Michele Mesiti Date: Wed, 23 Sep 2026 17:55:20 +0200 Subject: [PATCH 3/4] tools: some fixes and typos Co-authored-by: Michele Mesiti Co-authored-by: Radovan Bast --- content/tools.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/content/tools.md b/content/tools.md index 506284ac..70dad711 100644 --- a/content/tools.md +++ b/content/tools.md @@ -22,7 +22,7 @@ * - Type - Convenient - Easy - - Maintainabile + - Maintainable - Searchable - Readable - LLM-friendly @@ -48,8 +48,8 @@ - ❌ - βœ…πŸŸ¨ - βœ… - - ❌ - βœ…βœ… + - ❌ - powerful * - Wikis - 🟨 @@ -86,7 +86,7 @@ What do we mean? It is even better if it is easy to check automatically that the information is correct (*does the output of a snippet of code match what is shown in the docs?*) - **Searchability:** How easy is it to find the information we need? -- **Readability**: Can the documentation be rendered in a way that makes it easy to read? +- **Readability**: Can the documentation be rendered in a way that makes it easy to read? Can one copy-paste from it easily? - **LLM-friendliness**: how easy is to feed this documentation to an LLM? @@ -234,7 +234,7 @@ to common software forges. ```{discussion} Do you know an awesome tool or feature that should be in this list? -Let us know! (Open a PR) +Let us know! (click on "Edit on GitHub" at the top of this page) ``` From 350ce474213b8be2a346995325b2ad645a7c184e Mon Sep 17 00:00:00 2001 From: Michele Mesiti Date: Wed, 23 Sep 2026 18:51:38 +0200 Subject: [PATCH 4/4] tool comparison table: Tweaks/additions - merge Convenience and Easiness columns, added some notes regarding that - Maintainable -> Maintainable/Reproducible, updated explanation - Some tweaks to make columns slimmer - add comment about HTML generators being the only technique that allows readable API documentation - add llms.txt row Also fix sentence in HTML generator section about API reference. --- content/tools.md | 86 ++++++++++++++++++++++++++++-------------------- 1 file changed, 50 insertions(+), 36 deletions(-) diff --git a/content/tools.md b/content/tools.md index 70dad711..3316ec7f 100644 --- a/content/tools.md +++ b/content/tools.md @@ -11,84 +11,98 @@ --- - ## Documentation Tools: comparison -```{list-table} Comparison of the tools for documentation we have discussed so far -:widths: 20 10 10 10 10 10 10 15 +We try to compare documentation tools +in various dimensions: +- **Convenience**: how easy it is to contribute and set up, + for different kinds of people? +- **Maintainability and Reproducibility**: is it easy to retrieve the information for different versions of the code? + Good for tools that can be version-controlled + along with the code. + Even better if correctness of information can be checked automatically + (*does the output of a snippet of code match the docs?*) +- **Searchability:** is it easy to find the information we need? +- **Readability**: is it easy to read when rendered? Can one copy-paste from it easily? +- **LLM-friendliness**: is it convenient to feed to an LLM or an AI-agent? + +```{list-table} Comparison of tools for documentation +:widths: 20 20 10 10 10 10 15 :header-rows: 1 :stub-columns: 1 * - Type - - Convenient - - Easy - - Maintainable + - Convenient + - Maintainable/ + Reproducible - Searchable - Readable - - LLM-friendly + - LLM + friendly - Notes -* - in-code doc - - βœ…βœ… - - 🟨 +* - In-code + Documentation + - βœ…βœ… (P), + ❌ (U) - βœ…πŸŸ¨ - 🟨 - ❌ - βœ…πŸŸ¨ - - ❌for users + - * - README - - βœ… - βœ… - βœ…πŸŸ¨ - 🟨 - βœ… - βœ… - typically enough -* - HTML Generators - - 🟨 - - ❌ +* - HTML + Generators + - 🟨 - βœ…πŸŸ¨ - βœ… - βœ…βœ… - ❌ - - powerful + - powerful, + **only viable option** + **for readable API docs** * - Wikis - - 🟨 - - βœ… + - 🟨 (P) + βœ… (NP) - ❌❌ - βœ… - βœ… - ❌ - - βœ…for non-programmers + - * - Latex - - 🟨(?) - - ❌ + - 🟨 - ❌🟨 - 🟨 - βœ… (?) - ❌ - - βœ…Physics/Math, + - βœ…Physics/Math, ❌copy/paste * - Jupyter - 🟨 - - 🟨/❌ + 🟨/❌ - βœ…βœ… - 🟨 (?) - βœ… - 🟨 - βœ… validation tooling +* - [llms.txt](https://llmstxt.org/) + - 🟨 + - 🟨 + - N/A + - 🟨/❌ + - βœ…βœ… + - ``` +Abbreviations in the table: -What do we mean? -- **Convenience**: for programmers who live in code. -- **Easiness**: how easy is is to contribute and set up? -- **Maintainability** is good for those tools that can be version-controlled - along with the code. - It is even better if it is easy to check automatically that the information is correct - (*does the output of a snippet of code match what is shown in the docs?*) -- **Searchability:** How easy is it to find the information we need? -- **Readability**: Can the documentation be rendered in a way that makes it easy to read? Can one copy-paste from it easily? -- **LLM-friendliness**: how easy is to feed this documentation to an LLM? - +- **P**: For Programmers, who live in the code; +- **U**: For Users; +- **NP**: For Non-Programmers. --- @@ -102,7 +116,7 @@ or hosted on the web. Here are some HTML static site generators, relevant in our communities. These tools offer some or all of these features: -- **API Reference generation**: source code is read, scan for docstrings and render them +- **API Reference generation**: they read the source code scanning for docstrings, and render them to HTML - **Search**: they offer a "whole site" search feature (non trivial, when viewing only one page). (if you can download ) - **Validation**: check that the code snipped in the documentation @@ -133,7 +147,7 @@ These tools offer some or all of these features: [^mkdocsdrama]: After somewhat dramatic events (2026), MkDocs 1.x is now superseded by [Zensical](https://zensical.org/), - which tries to keep compatibility wiht MkDocs 1.x. + which tries to keep compatibility with MkDocs 1.x. (MkDocs 2.0 is also being developed but projects and plugins based on 1.x will break).