Rewrite readme and update contributor docs

Incorporates https://github.com/matrix-org/matrix-doc/pull/3025/

.gitignore

@@ -18,3 +18,4 @@
 _rendered.rst
 /.vscode/
 /.idea/
+/spec/

CONTRIBUTING.rst

@@ -29,9 +29,7 @@ some time to complete.
 Changes to the protocol (new endpoints, ideas, etc) need to go through the
 `proposals process <https://matrix.org/docs/spec/proposals>`_. Other changes,
 such as fixing bugs, typos, or clarifying existing behaviour do not need a proposal.
-If you're not sure, visit us at `#matrix-spec:matrix.org`_
+If you're not sure, visit us at `#matrix-spec:matrix.org`_ and ask.
-and ask.
-
 
 Other changes
 ~~~~~~~~~~~~~
@@ -64,12 +62,17 @@ following:
   to fix. On the other hand, introducing new behaviour is best represented by a
   proposal.
 
+* Design or aesthetic changes, such as improving accessibility, colour schemes,
+  etc. Please check in with us at `#matrix-docs:matrix.org`_ with your proposed
+  design change before opening a PR so we can work with you on it.
+
 For such changes, please do just open a `pull request`_. If you're not sure if
 your change is covered by the above, please visit `#matrix-spec:matrix.org` and
 ask.
 
 .. _`pull request`: https://help.github.com/articles/about-pull-requests
 .. _`#matrix-spec:matrix.org`: https://matrix.to/#/#matrix-spec:matrix.org
+.. _`#matrix-docs:matrix.org`: https://matrix.to/#/#matrix-docs:matrix.org
 
 
 Adding to the changelog
@@ -100,8 +103,7 @@ the ``newsfragments`` directory. The ``type`` can be one of the following:
 
 All news fragments must have a brief summary explaining the change in the
 contents of the file. The summary must end in a full stop to be in line with
-the style guide and and formatting must be done using `Restructured Text
+the style guide and and formatting must be done using Markdown.
-<http://www.sphinx-doc.org/en/master/usage/restructuredtext/basics.html>`_.
 
 Changes that do not change the spec, such as changes to the build script, formatting,
 CSS, etc should not get a news fragment.

README.md

@@ -0,0 +1,97 @@
+# Matrix Specification
+
+This repository contains the Matrix Specification, rendered at [spec.matrix.org](http://spec.matrix.org/).
+
+Developers looking to use Matrix should join [#matrix-dev:matrix.org](http://matrix.to/#/#matrix-dev:matrix.org)
+on Matrix for help.
+
+Spec authors and proposal writers are welcome to join [#matrix-spec:matrix.org](http://matrix.to/#/#matrix-spec:matrix.org). We welcome contributions! See [CONTRIBUTING.rst](./CONTRIBUTING.rst) for details.
+
+## Structure
+
+The Matrix spec is compiled with [Hugo](https://gohugo.io/) (a static site generator) with the following structure:
+
+* `/assets`: assets that need postprocessing using [Hugo Pipes](https://gohugo.io/hugo-pipes/introduction/). 
+  For example, Sass files would go here.
+
+* `/content`: files that will become pages in the site go here. Typically these are Markdown files with some YAML front 
+  matter indicating, [among other things](https://gohugo.io/content-management/front-matter/), what layout should be 
+  applied to this page. The organization of files under `/content` determines the organization of pages in the built 
+  site.
+
+* `/data`: this can contain TOML, YAML, or JSON files. Files kept here are directly available to template code as 
+  [data objects](https://gohugo.io/templates/data-templates/), so templates don't need to load them from a file and 
+  parse them. This is also where our 
+
+* `/layouts`: this contains [Hugo templates](https://gohugo.io/templates/). Some templates define the overall layout of 
+  a page: for example, whether it has header, footer, sidebar, and so on.
+    * `/layouts/partials`: these templates can be called from other templates, so they can be used to factor out 
+      template code that's used in more than one template. An obvious example here is something like a sidebar, where 
+      several different page layouts might all include the sidebar. But also, partial templates can return values: this 
+      means they can be used like functions, that can be called by multiple templates to do some common processing.
+    * `/layouts/shortcodes`: these templates can be called directly from files in `/content`.
+
+* `/static`: static files which don't need preprocessing. JS or CSS files could live here.
+
+* `/themes`: you can use just Hugo or use it with a theme. Themes primarily provide additional templates, which are 
+  supplied in a `/themes/$theme_name/layouts` directory. You can use a theme but customise it by providing your own 
+  versions of any of the them layouts in the base `/layouts` directory. That is, if a theme provides 
+  `/themes/$theme_name/layouts/sidebar.html` and you provide `/layouts/sidebar.html`, then your version of this 
+  template will be used.
+
+It also has the following top-level file:
+
+* `config.toml`: site-wide configuration settings. Some of these are built-in and you can add your own. Config settings 
+  defined here are available in templates. All these directories above are configurable via `config.toml` settings.
+
+Additionally, the following directories may be of interest:
+
+* `/attic`: Here contains historical sections of specification and legacy drafts for the specification.
+* `/changelogs`: Various bits of changelog for the specification areas.
+* `/event-schemas`: [JSON Schema](http://json-schema.org/) definitions for the spec.
+* `/data-definitions`: Bits of structured data consumable by Matrix implementations.
+* `/meta`: Documentation relating to the spec's processes that are otherwise untracked (release instructions, etc).
+* `/scripts`: Various scripts for generating the spec.
+* `/proposals`: Matrix Spec Change (MSC) proposals. See <https://spec.matrix.org/unstable/proposals/>.
+* `/api`: [OpenAPI](https://github.com/OAI/OpenAPI-Specification/blob/master/versions/2.0.md) / Swagger definitions for
+  the spec.
+
+## Authoring changes to the spec
+
+Please read [CONTRIBUTING.rst](./CONTRIBUTING.rst) before authoring a change to the spec. Note that spec authoring takes
+place after an MSC has been accepted, not as part of a proposal itself.
+
+1. Install the extended version (often the OS default) of Hugo: <https://gohugo.io/getting-started/installing>
+2. Run `git submodule update --init --recursive` for good measure.
+3. Run `npm i` to install the dependencies. Note that this will require NodeJS to be installed.
+4. Run `npm run get-proposals` to seed the proposals data. This is not required.
+5. Run `hugo serve` to run a local webserver which builds whenever a file change is detected. If watching doesn't appear
+   to be working for you, try `hugo serve --disableFastRender` instead.
+6. Edit the specification 🙂
+
+We use a highly customized [Docsy](https://www.docsy.dev/) theme for our generated site, which uses Bootstrap and Font 
+Awesome. If you're looking at making design-related changes to the spec site, please coordinate with us in 
+[#matrix-docs:matrix.org](https://matrix.to/#/#matrix-docs:matrix.org) before opening a PR.
+
+## Building the specification
+
+If for some reason you're not a CI/CD system and want to render the spec yourself, follow the above steps for authoring
+changes to the specification and instead of `hugo serve` run `hugo -d "spec"` - this will generate the spec to `/spec`.
+If you'd like to serve the spec off a path instead of a domain root (eg: `/unstable`), add `--baseURL "/unstable"` to
+the `hugo -d "spec"` command.
+
+For building the swagger definitions, create a python3 virtualenv and activate it. Then run `pip install -r ./scripts/requirements.txt` and finally `python ./scripts/dump-swagger.py` to generate it to `./scripts/swagger/api-docs.json`.
+To make use of the generated file, there are a number of options:
+
+* It can be uploaded from your filesystem to an online editor/viewer such as
+  http://editor.swagger.io/
+* You can run a local HTTP server by running `./scripts/swagger-http-server.py`, and then view the documentation via an
+  online viewer; for example, at <http://petstore.swagger.io/?url=http://localhost:8000/api-docs.json>
+* You can host the swagger UI yourself. See <https://github.com/swagger-api/swagger-ui#how-to-run> for advice on how to 
+  do so.
+
+## Issue tracking
+
+Specification issues are tracked on github at <https://github.com/matrix-org/matrix-doc/issues>.
+
+See [meta/github-labels.rst](./meta/github-labels.rst) for information on what the labels mean.

README.rst

@@ -1,143 +0,0 @@
-This repository contains the Matrix specification.
-
-If you want to ask more about the specification, join us on
-`#matrix-dev:matrix.org <http://matrix.to/#/#matrix-dev:matrix.org>`_.
-
-We welcome contributions to the spec! See the notes below on `Building the
-specification`_, and `<CONTRIBUTING.rst>`_ to get started making contributions.
-
-Note that the Matrix Project lists, which were previously kept in this
-repository, are now in https://github.com/matrix-org/matrix.org.
-
-Structure of this repository
-============================
-
-- ``api`` : `OpenAPI`_ (swagger) specifications for the the HTTP APIs.
-- ``attic``: historical sections of specification for reference
-  purposes.
-- ``changelogs``: change logs for the various parts of the
-  specification.
-- ``drafts``: Previously, contained documents which were under discussion for
-  future incusion into the specification and/or supporting documentation. This
-  is now historical, as we use separate discussion documents (see
-  `<CONTRIBUTING.rst>`_).
-- ``event-schemas``: the `JSON Schema`_ for all Matrix events
-  contained in the specification, along with example JSON files.
-- ``meta``: documents outlining the processes involved when writing
-  documents, e.g. documentation style, guidelines.
-- ``scripts``: scripts to generate formatted versions of the
-  documentation, typically HTML.
-- ``specification``: the specification split up into sections.
-
-.. _OpenAPI: https://github.com/OAI/OpenAPI-Specification/blob/master/versions/2.0.md
-.. _JSON Schema: http://json-schema.org/
-
-Building the specification
-==========================
-
-The Matrix Spec is generated by a set of scripts, from the RST documents, API
-specs and event schemas in this repository.
-
-Preparation
------------
-
-To use the scripts, it is best to create a Python 3.4+ virtualenv as follows::
-
-  virtualenv -p python3 env
-  env/bin/pip install -r scripts/requirements.txt
-
-(Benjamin Saunders has contributed a script for `Nix`_ users, which can be
-invoked with ``nix-shell scripts/contrib/shell.nix``.)
-
-.. TODO: Possibly we need some libs installed; should record what they are.
-
-.. _`Nix`: https://nixos.org/nix/
-
-Generating the specification
-----------------------------
-
-To rebuild the specification, use ``scripts/gendoc.py``::
-
-  source env/bin/activate
-  ./scripts/gendoc.py
-
-The above will write the rendered version of the specification to
-``scripts/gen``. To view it, point your browser at ``scripts/gen/index.html``.
-
-Windows users
-~~~~~~~~~~~~~
-The ``source`` program does not exist on Windows, so instead run one of the 
-``activate`` files in ``.\env\Scripts\`` to activate the virtual environment.
-
-If you're on Windows Vista or higher, be sure that the "Symbolic Links"
-option was selected when installing Git prior to cloning this repository. If
-you're still seeing errors about files not being found it is likely because
-the symlink at ``api/client-server/definitions/event-schemas`` looks like a
-file. To correct the problem, open an Administrative/Elevated Command Prompt in your
-cloned matrix-doc directory and run the following::
-
-  cd api\client-server\definitions
-  del event-schemas
-  mklink /D event-schemas "..\..\..\event-schemas"
-
-This will delete the file and replace it with a symlink. Git should not detect
-this as a change, and you should be able to go back to building the project.
-
-Generating the OpenAPI (Swagger) specs
---------------------------------------
-
-`Swagger`_ is a framework for representing RESTful APIs. We use it to generate
-interactive documentation for our APIs.
-
-Before the Swagger docs can be used in the Swagger UI (or other tool expecting
-a Swagger specs, they must be combined into a single json file. This can be
-done as follows::
-
-  source env/bin/activate
-  ./scripts/dump-swagger.py
-
-By default, ``dump-swagger`` will write to ``scripts/swagger/api-docs.json``.
-
-To make use of the generated file, there are a number of options:
-
-* It can be uploaded from your filesystem to an online editor/viewer such as
-  http://editor.swagger.io/
-* You can run a local HTTP server by running
-  ``./scripts/swagger-http-server.py``, and then view the documentation via an
-  online viewer; for example, at
-  http://petstore.swagger.io/?url=http://localhost:8000/api-docs.json
-* You can host the swagger UI yourself. See
-  https://github.com/swagger-api/swagger-ui#how-to-run for advice on how to do
-  so.
-
-.. _`Swagger`: http://swagger.io/
-
-Continuserv
------------
-
-Continuserv is a script which will rebuild the specification every time a file
-is changed, and will serve it to a browser over HTTP. It is intended for use by
-specification authors, so that they can quickly see the effects of their
-changes.
-
-It is written in Go, so you will need the ``go`` compiler installed on your
-computer. You will also need to install fsnotify by running::
-
-  go get gopkg.in/fsnotify/fsnotify.v1
-
-Then, create a virtualenv as described above under `Preparation`_,
-and::
-
-  source env/bin/activate
-  go run ./scripts/continuserv/main.go
-
-You will then be able to view the generated spec by visiting
-http://localhost:8000/index.html.
-
-Issue tracking
-==============
-
-Issues with the Matrix specification are tracked in `GitHub
-<https://github.com/matrix-org/matrix-doc/issues>`_.
-
-See `meta/github-labels.rst <meta/github-labels.rst>`_ for notes on what the labels mean.

meta/documentation_style.rst

@@ -8,52 +8,36 @@ in.
 Format
 ------
 
-Documentation is written either in github-flavored markdown or RST.
+Documentation is written either in github-flavored markdown.
 
 Sections
 --------
 
-RST support lots of different punctuation characters for underlines on sections.
+Markdown supports headings through the `#` prefix on text. Please avoid heavily
-Content in the specification MUST use the same characters in order for the
+nested titles (h6, or 6 `#` characters) and instead re-evaluate the document structure.
-complete specification to be merged correctly. These characters are:
-
-- ``=``
-- ``-``
-- ``~``
-- ``+``
-- ``^``
-- \ `````
-- ``@``
-- ``:``
-
-If you find yourself using ``^`` or beyond, you should rethink your document
-layout if possible.
 
 Correct capitalisation for long section names
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 
-Headings should start with a capital letter, and use lower-case otherwise.
+Headings should start with a capital letter, and use lower-case otherwise. This
-
+document is an example of what we mean.
 
 TODOs
 -----
 
-Any RST file in this repository may make it onto ``matrix.org``. We do not want
+Any file in this repository might make it onto the matrix.org site, and as such
-``TODO`` markers visible there. For internal comments, notes, TODOs, use standard
+we do not want ``TODO`` markers visible there. For internal comments, notes, TODOs,
-RST comments like so::
+etc please use standard markdown comments (`<!-- TODO TravisR: Fix this -->`). Please
-
+include your name in the TODO comment so we know who to ask about it in the future.
-  .. TODO-Bob
-    There is something to do here. This will not be rendered by something like
-    rst2html.py so it is safe to put internal comments here.
-
-You SHOULD put your username with the TODO so we know who to ask about it.
 
 Line widths
 -----------
 
-We use 80 characters for line widths. This is a guideline and can be flouted IF
+We use 80 characters for line widths. This is a guideline and can be ignored IF
 AND ONLY IF it makes reading more legible. Use common sense.
 
+For proposals, please use 120 characters as a guide.
+
 Stylistic notes
 ---------------