Merge branch 'main' into refresh_token_spec

changelogs/application_service/newsfragments/1094.feature

@@ -0,0 +1 @@
+Add timestamp massaging as per [MSC3316](https://github.com/matrix-org/matrix-spec-proposals/pull/3316).

changelogs/client_server/newsfragments/1003.clarification

@@ -1 +1 @@
-Adjust the OpenAPI specification so that the type `Flow information` is explicitly defined when the CS spec is rendered.
+Adjust the OpenAPI specification so that the type `Flow information` is explicitly defined when the client-server API is rendered.

changelogs/client_server/newsfragments/1054.clarification

@@ -0,0 +1 @@
+Fix various typos throughout the specification.

changelogs/client_server/newsfragments/1059.clarification

@@ -0,0 +1 @@
+Fix various typos throughout the specification.

changelogs/client_server/newsfragments/1081.clarification

@@ -0,0 +1 @@
+Fix various typos throughout the specification.

changelogs/client_server/newsfragments/1100.clarification

@@ -0,0 +1 @@
+Clarify that state keys starting with `@` are in fact reserved. Regressed from [#3658](https://github.com/matrix-org/matrix-spec-proposals/pull/3658).

changelogs/client_server/newsfragments/3681.clarification

@@ -1 +1 @@
-Fix broken syntax in Server Access Control Lists definition.
+Fix various typos throughout the specification.

changelogs/room_versions/newsfragments/1037.clarification

@@ -1,2 +1 @@
-Improve readability of definitions in the state resolution v2 algorithm.
+Improve readability and understanding of the state resolution algorithms.
-

changelogs/room_versions/newsfragments/1042.clarification

@@ -0,0 +1 @@
+Improve readability and understanding of the state resolution algorithms.

changelogs/room_versions/newsfragments/1043.clarification

@@ -0,0 +1 @@
+Improve readability and understanding of the state resolution algorithms.

changelogs/room_versions/newsfragments/1050.clarification

@@ -0,0 +1 @@
+Improve readability of the authorization rules.

changelogs/room_versions/newsfragments/1093.clarification

@@ -0,0 +1 @@
+For room versions 8, 9, and 10: clarify which homeserver is required to sign the join event.

changelogs/room_versions/newsfragments/3737.clarification

@@ -1 +1 @@
-Fix join membership auth rules when `join_rule` is `knock`.
+For room versions 7, 8, 9, and 10: fix join membership authorization rules when `join_rule` is `knock`.

changelogs/room_versions/newsfragments/3739.feature

@@ -1 +1 @@
-Update the default room version to 9.
+Update the default room version to 9 as per [MSC3589](https://github.com/matrix-org/matrix-spec-proposals/pull/3589).

changelogs/server_server/newsfragments/1038.clarification

@@ -1 +1 @@
-Fix origin server name in S2S Request Authentication example.
+Clarify the format for the Authorization header.

changelogs/server_server/newsfragments/1045.clarification

@@ -0,0 +1 @@
+Clarify what a "valid event" means when performing checks on a received PDU.

changelogs/server_server/newsfragments/1055.clarification

@@ -0,0 +1 @@
+Clarify that `valid_until_ts` is in milliseconds, like other timestamps used in Matrix.

changelogs/server_server/newsfragments/1067.clarification

@@ -0,0 +1 @@
+Clarify the format for the Authorization header.

changelogs/server_server/newsfragments/1067.feature

@@ -0,0 +1 @@
+Add a `destination` property to the Authorization header, as per [MSC3383](https://github.com/matrix-org/matrix-spec-proposals/pull/3383).

changelogs/server_server/newsfragments/1070.clarification

@@ -0,0 +1 @@
+Clarify that checks on PDUs should refer to the state *before* an event.

changelogs/server_server/newsfragments/998.clarification

@@ -1 +1 @@
-Remove `origin` field from PDUs which exists on many but not all PDUs in practice and doesn't serve an actual purpose.
+Remove largely unused `origin` field from PDUs.

content/application-service-api.md

@@ -300,13 +300,38 @@ An example request would be:
 
 #### Timestamp massaging
 
-Previous drafts of the Application Service API permitted application
+{{% added-in v="1.3" %}}
-services to alter the timestamp of their sent events by providing a `ts`
+
-query parameter when sending an event. This API has been excluded from
+Application services can alter the timestamp associated with an event, allowing
-the first release due to design concerns, however some servers may still
+the application service to better represent the "real" time an event was sent
-support the feature. Please visit [issue
+at. While this doesn't affect the server-side ordering of the event, it can allow
-\#1585](https://github.com/matrix-org/matrix-doc/issues/1585) for more
+an application service to better represent when an event would have been sent/received
-information.
+at, such as in the case of bridges where the remote network might have a slight
+delay and the application service wishes to bridge the proper time onto the message.
+
+When authenticating requests as an application service, the caller can append a `ts`
+query string argument to change the `origin_server_ts` of the resulting event. Attempting
+to set the timestamp to anything other than what is accepted by `origin_server_ts` should
+be rejected by the server as a bad request.
+
+When not present, the server's behaviour is unchanged: the local system time of the server
+will be used to provide a timestamp, representing "now".
+
+The `ts` query string argument is only valid on the following endpoints:
+
+* [`PUT /rooms/{roomId}/send/{eventType}/{txnId}`](/client-server-api/#put_matrixclientv3roomsroomidsendeventtypetxnid)
+* [`PUT /rooms/{roomId}/state/{eventType}/{stateKey}`](/client-server-api/#put_matrixclientv3roomsroomidstateeventtypestatekey)
+
+Other endpoints, such as `/kick`, do not support `ts`: instead, callers can use the
+`PUT /state` endpoint to mimic the behaviour of the other APIs.
+
+{{% boxes/warning %}}
+Changing the time of an event does not change the server-side (DAG) ordering for the
+event. The event will still be appended at the tip of the DAG as though the timestamp
+was set to "now". Future MSCs, like [MSC2716](https://github.com/matrix-org/matrix-spec-proposals/pull/2716),
+are expected to provide functionality which can allow DAG order manipulation (for history
+imports and similar behaviour).
+{{% /boxes/warning %}}
 
 #### Server admin style permissions
 

content/client-server-api/_index.md

@@ -1107,8 +1107,8 @@ as follows:
 }
 ```
 
-As with [token-based]() interactive login, the `token` must encode the
+The `token` must encode the user ID, since there is no other identifying
-user ID. In the case that the token is not valid, the homeserver must
+data in the request. In the case that the token is not valid, the homeserver must
 respond with `403 Forbidden` and an error code of `M_FORBIDDEN`.
 
 If the homeserver advertises `m.login.sso` as a viable flow, and the

content/rooms/fragments/v1-auth-rules.md

@@ -4,11 +4,11 @@ toc_hide: true
 
 The types of state events that affect authorization are:
 
--   `m.room.create`
+-   [`m.room.create`](/client-server-api#mroomcreate)
--   `m.room.member`
+-   [`m.room.member`](/client-server-api#mroommember)
--   `m.room.join_rules`
+-   [`m.room.join_rules`](/client-server-api#mroom)
--   `m.room.power_levels`
+-   [`m.room.power_levels`](/client-server-api#mroompower_levels)
--   `m.room.third_party_invite`
+-   [`m.room.third_party_invite`](/client-server-api#mroomthird_party_invite)
 
 {{% boxes/note %}}
 Power levels are inferred from defaults when not explicitly supplied.

content/rooms/fragments/v2-state-res.md

@@ -2,20 +2,19 @@
 toc_hide: true
 ---
 
-The room state *S*′(*E*) after an event *E* is defined in terms of the
+The room state *S′(E)* after an event *E* is defined in terms of the
-room state *S*(*E*) before *E*, and depends on whether *E* is a state
+room state *S(E)* before *E*, and depends on whether *E* is a state
 event or a message event:
 
--   If *E* is a message event, then *S*′(*E*) = *S*(*E*).
+-   If *E* is a message event, then *S′(E)* = *S(E)*.
--   If *E* is a state event, then *S*′(*E*) is *S*(*E*), except that its
+-   If *E* is a state event, then *S′(E)* is *S(E)*, except that its
-    entry corresponding to *E*'s `event_type` and `state_key` is
+    entry corresponding to the `event_type` and `state_key` of *E* is
-    replaced by *E*'s `event_id`.
+    replaced by the `event_id` of *E*.
 
-The room state *S*(*E*) before *E* is the *resolution* of the set of
+The room state *S(E)* before *E* is the *resolution* of the set of
-states {*S*′(*E*<sub>1</sub>), *S*′(*E*<sub>2</sub>), …} consisting of
+states {*S′(E*<sub>1</sub>*)*, *S′(E*<sub>2</sub>*)*, …}
-the states after each of *E*'s `prev_event`s
+after the `prev_event`s {*E*<sub>1</sub>, *E*<sub>2</sub>, …} of *E*.
-{*E*<sub>1</sub>, *E*<sub>2</sub>, …}, where the resolution of a set of
+The resolution of a set of states is given in the algorithm below.
-states is given in the algorithm below.
 
 #### Definitions
 
@@ -31,11 +30,22 @@ the `membership` is `leave` or `ban` and the `sender` does not match the
 might remove someone's ability to do something in the room.
 
 **Unconflicted state map and conflicted state set.**
-The *unconflicted state map* is the state where the value of each key
+The keys of the state maps *S<sub>i</sub>* are 2-tuples of strings of the form
-exists and is the same in each state *S*<sub>*i*</sub>. The *conflicted
+*K* = `(event_type, state_key)`. The values *V* are state events.
-state set* is the set of all other state events. Note that the
+The key-value pairs (*K*, *V*) across all state maps *S<sub>i</sub>* can be 
-unconflicted state map only has one event per `(event_type, state_key)`,
+divided into two collections.
-whereas the conflicted state set may have multiple events.
+If a given key *K* is present in every *S<sub>i</sub>* with the same value *V* 
+in each state map, then the pair (*K*, *V*) belongs to the *unconflicted state map*.
+Otherwise (*K*, *V*) belongs to the *conflicted state set*.
+
+Note that the unconflicted state map only has one event for each key *K*,
+whereas the conflicted state set may associate multiple events to the same key.
+
+**Auth chain.**
+The *auth chain* of an event *E* is the set containing all of *E*'s auth events,
+all of *their* auth events, and so on recursively, stretching back to the
+start of the room. Put differently, these are the events reachable by walking
+the graph induced by an event's `auth_events` links.
 
 **Auth difference.**
 The *auth difference* is calculated by first calculating the full auth
@@ -112,9 +122,9 @@ the auth event is not rejected.
 
 The *resolution* of a set of states is obtained as follows:
 
-1.  Take all *power events* and any events in their auth chains,
+1.  Select all *power events* that appear in the *full conflicted set*. Compute
-    recursively, that appear in the *full conflicted set* and order them
+    the union of their auth chains, including the power events themselves.
-    by the *reverse topological power ordering*.
+    Sort the union using the *reverse topological power ordering*.
 2.  Apply the *iterative auth checks algorithm*, starting from the
     *unconflicted state map*, to the list of events from the previous
     step to get a partially resolved state.

content/rooms/fragments/v3-auth-rules.md

@@ -11,11 +11,11 @@ however.
 
 The types of state events that affect authorization are:
 
--   `m.room.create`
+-   [`m.room.create`](/client-server-api#mroomcreate)
--   `m.room.member`
+-   [`m.room.member`](/client-server-api#mroommember)
--   `m.room.join_rules`
+-   [`m.room.join_rules`](/client-server-api#mroom)
--   `m.room.power_levels`
+-   [`m.room.power_levels`](/client-server-api#mroompower_levels)
--   `m.room.third_party_invite`
+-   [`m.room.third_party_invite`](/client-server-api#mroomthird_party_invite)
 
 {{% boxes/note %}}
 Power levels are inferred from defaults when not explicitly supplied.

content/rooms/fragments/v8-auth-rules.md

@@ -12,11 +12,11 @@ of receipt, they are authorized at a later stage: see the
 
 The types of state events that affect authorization are:
 
--   `m.room.create`
+-   [`m.room.create`](/client-server-api#mroomcreate)
--   `m.room.member`
+-   [`m.room.member`](/client-server-api#mroommember)
--   `m.room.join_rules`
+-   [`m.room.join_rules`](/client-server-api#mroom)
--   `m.room.power_levels`
+-   [`m.room.power_levels`](/client-server-api#mroompower_levels)
--   `m.room.third_party_invite`
+-   [`m.room.third_party_invite`](/client-server-api#mroomthird_party_invite)
 
 {{% boxes/note %}}
 Power levels are inferred from defaults when not explicitly supplied.
@@ -46,7 +46,7 @@ The rules are as follows:
     1.  If no `state_key` key or `membership` key in `content`, reject.
     2.  If `content` has a `join_authorised_via_users_server`
         key:
-        1.  If the event is not validly signed by the user ID denoted
+        1.  If the event is not validly signed by the homeserver of the user ID denoted
             by the key, reject.
     3.  If `membership` is `join`:
         1.  If the only previous event is an `m.room.create` and the

content/rooms/v1.md

@@ -66,18 +66,18 @@ This is fixed in the state resolution algorithm introduced in room
 version 2.
 {{% /boxes/warning %}}
 
-The room state *S*′(*E*) after an event *E* is defined in terms of the
+The room state *S′*(*E*) after an event *E* is defined in terms of the
 room state *S*(*E*) before *E*, and depends on whether *E* is a state
 event or a message event:
 
--   If *E* is a message event, then *S*′(*E*) = *S*(*E*).
+-   If *E* is a message event, then *S′(E)* = *S(E)*.
--   If *E* is a state event, then *S*′(*E*) is *S*(*E*), except that its
+-   If *E* is a state event, then *S′(E)* is *S(E)*, except that its
-    entry corresponding to *E*'s `event_type` and `state_key` is
+    entry corresponding to the `event_type` and `state_key` of *E* is
-    replaced by *E*'s `event_id`.
+    replaced by the `event_id` of *E*.
 
-The room state *S*(*E*) before *E* is the *resolution* of the set of
+The room state *S(E)* before *E* is the *resolution* of the set of
-states {*S*′(*E*′), *S*′(*E*″), …} consisting of the states after each
+states {*S′(E′)*, *S′(E″)*, …} after the `prev_events` {*E′*, *E″*, …}.
-of *E*'s `prev_event`s {*E*′, *E*″, …}.
+of *E*.
 
 The *resolution* of a set of states is defined as follows. The resolved
 state is built up in a number of passes; here we use *R* to refer to the

content/rooms/v6.md

@@ -59,11 +59,11 @@ Events must be signed by the server denoted by the `sender` key.
 
 The types of state events that affect authorization are:
 
--   `m.room.create`
+-   [`m.room.create`](/client-server-api#mroomcreate)
--   `m.room.member`
+-   [`m.room.member`](/client-server-api#mroommember)
--   `m.room.join_rules`
+-   [`m.room.join_rules`](/client-server-api#mroom)
--   `m.room.power_levels`
+-   [`m.room.power_levels`](/client-server-api#mroompower_levels)
--   `m.room.third_party_invite`
+-   [`m.room.third_party_invite`](/client-server-api#mroomthird_party_invite)
 
 {{% boxes/note %}}
 Power levels are inferred from defaults when not explicitly supplied.

content/rooms/v7.md

@@ -45,11 +45,11 @@ of receipt, they are authorized at a later stage: see the
 
 The types of state events that affect authorization are:
 
--   `m.room.create`
+-   [`m.room.create`](/client-server-api#mroomcreate)
--   `m.room.member`
+-   [`m.room.member`](/client-server-api#mroommember)
--   `m.room.join_rules`
+-   [`m.room.join_rules`](/client-server-api#mroom)
--   `m.room.power_levels`
+-   [`m.room.power_levels`](/client-server-api#mroompower_levels)
--   `m.room.third_party_invite`
+-   [`m.room.third_party_invite`](/client-server-api#mroomthird_party_invite)
 
 {{% boxes/note %}}
 Power levels are inferred from defaults when not explicitly supplied.

content/server-server-api.md

@@ -255,7 +255,7 @@ condition applies throughout the request signing process.
 Step 2 add Authorization header:
 
     GET /target HTTP/1.1
-    Authorization: X-Matrix origin=origin.hs.example.com,key="ed25519:key1",sig="ABCDEF..."
+    Authorization: X-Matrix origin="origin.hs.example.com",destination="destination.hs.example.com",key="ed25519:key1",sig="ABCDEF..."
     Content-Type: application/json
 
     <JSON-encoded request body>
@@ -283,14 +283,52 @@ def authorization_headers(origin_name, origin_signing_key,
 
     for key, sig in signed_json["signatures"][origin_name].items():
         authorization_headers.append(bytes(
-            "X-Matrix origin=%s,key=\"%s\",sig=\"%s\"" % (
+            "X-Matrix origin=\"%s\",destination=\"%s\",key=\"%s\",sig=\"%s\"" % (
-                origin_name, key, sig,
+                origin_name, destination_name, key, sig,
             )
         ))
 
-    return ("Authorization", authorization_headers)
+    return ("Authorization", authorization_headers[0])
 ```
 
+The format of the Authorization header is given in
+[RFC 7235](https://datatracker.ietf.org/doc/html/rfc7235#section-2.1). In
+summary, the header begins with authorization scheme `X-Matrix`, followed by
+one or more spaces, followed by a comma-separated list of parameters written as
+name=value pairs. The names are case insensitive and order does not matter. The
+values must be enclosed in quotes if they contain characters that are not
+allowed in `token`s, as defined in
+[RFC 7230](https://datatracker.ietf.org/doc/html/rfc7230#section-3.2.6); if a
+value is a valid `token`, it may or may not be enclosed in quotes. Quoted
+values may include backslash-escaped characters. When parsing the header, the
+recipient must unescape the characters. That is, a backslash-character pair is
+replaced by the character that follows the backslash.
+
+For compatibility with older servers, the sender should
+- only include one space after `X-Matrix`,
+- only use lower-case names, and
+- avoid using backslashes in parameter values.
+
+For compatibility with older servers, the recipient should allow colons to be
+included in values without requiring the value to be enclosed in quotes.
+
+The authorization parameters to include are:
+
+- `origin`: the server name of the sending server. This is the same as the
+  `origin` field from JSON described in step 1.
+- `destination`: {{< added-in v="1.3" >}} the server name of the receiving
+  sender. This is the same as the `destination` field from the JSON described
+  in step 1. For compatibility with older servers, recipients should accept
+  requests without this parameter, but MUST always send it. If this property
+  is included, but the value does not match the receiving server's name, the
+  receiving server must deny the request with an HTTP status code 401
+  Unauthorized.
+- `key`: the ID, including the algorithm name, of the sending server's key used
+  to sign the request.
+- `signature`: the signature of the JSON as calculated in step 1.
+
+Unknown parameters are ignored.
+
 ### Response Authentication
 
 Responses are authenticated by the TLS server certificate. A homeserver
@@ -356,13 +394,15 @@ specification](/rooms).
 Whenever a server receives an event from a remote server, the receiving
 server must ensure that the event:
 
-1.  Is a valid event, otherwise it is dropped.
+1.  Is a valid event, otherwise it is dropped. For an event to be valid, it
+    must contain a `room_id`, and it must comply with the event format of
+    that [room version](/rooms).
 2.  Passes signature checks, otherwise it is dropped.
 3.  Passes hash checks, otherwise it is redacted before being processed
     further.
 4.  Passes authorization rules based on the event's auth events,
     otherwise it is rejected.
-5.  Passes authorization rules based on the state at the event,
+5.  Passes authorization rules based on the state before the event,
     otherwise it is rejected.
 6.  Passes authorization rules based on the current state of the room,
     otherwise it is "soft failed".

data/api/client-server/definitions/client_event_without_room_id.yaml

@@ -38,6 +38,10 @@ properties:
       Present if, and only if, this event is a *state* event. The key making
       this piece of state unique in the room. Note that it is often an empty
       string.
+
+      State keys starting with an `@` are reserved for referencing user IDs, such
+      as room members. With the exception of a few events, state events set with a
+      given user's ID as the state key MUST only be set by that user.
     type: string
     example: '@user:example.org'
   sender:

data/api/client-server/room_state.yaml

@@ -114,7 +114,7 @@ paths:
 
             Some example error codes include:
 
-            * `M_INVALID_PARAMETER`: One or more aliases within the `m.room.canonical_alias`
+            * `M_INVALID_PARAM`: One or more aliases within the `m.room.canonical_alias`
               event have invalid syntax.
 
             * `M_BAD_ALIAS`: One or more aliases within the `m.room.canonical_alias` event

data/api/server-server/definitions/keys.yaml

@@ -88,9 +88,10 @@ properties:
     type: integer
     format: int64
     description: |-
-      POSIX timestamp when the list of valid keys should be refreshed. This field MUST
+      POSIX timestamp in milliseconds when the list of valid keys should be refreshed.
-      be ignored in room versions 1, 2, 3, and 4. Keys used beyond this timestamp MUST
+      This field MUST be ignored in room versions 1, 2, 3, and 4. Keys used beyond this
-      be considered invalid, depending on the [room version specification](/rooms).
+      timestamp MUST be considered invalid, depending on the
+      [room version specification](/rooms).
 
       Servers MUST use the lesser of this field and 7 days into the future when
       determining if a key is valid. This is to avoid a situation where an attacker

data/event-schemas/examples/m.call.answer.yaml

@@ -4,7 +4,6 @@
   "content": {
     "version" : 0,
     "call_id": "12345",
-    "lifetime": 60000,
     "answer": {
       "type" : "answer",
       "sdp" : "v=0\r\no=- 6584580628695956864 2 IN IP4 127.0.0.1[...]"

layouts/shortcodes/added-in.html

@@ -2,7 +2,7 @@
 {{ $this := .Params.this }}
 
 {{ if $this }}
-  <span>**[New in this version]**</span>
+  <span><strong>[New in this version]</strong></span>
 {{ else }}
-  <span>**[Added in `v{{ $ver }}`]**</span>
+  <span><strong>[Added in <code>v{{ $ver }}</code>]</strong></span>
 {{ end }} {{/* Do not leave an empty line at the end of this file otherwise the inline behaviour breaks. */}}

layouts/shortcodes/changed-in.html

@@ -2,7 +2,7 @@
 {{ $this := .Params.this }}
 
 {{ if $this }}
-  <span>**[Changed in this version]**</span>
+  <span><strong>[Changed in this version]</strong></span>
 {{ else }}
-  <span>**[Changed in `v{{ $ver }}`]**</span>
+  <span><strong>[Changed in <code>v{{ $ver }}</code>]</strong></span>
 {{ end }} {{/* Do not leave an empty line at the end of this file otherwise the inline behaviour breaks. */}}