From ec37fe44270b6dfaa6ad6e143a26bf428d99f3d6 Mon Sep 17 00:00:00 2001 From: Luca Becker Date: Thu, 1 Aug 2019 15:13:04 +0200 Subject: [PATCH 001/114] first draft of MSC2209 --- ...rules-other-keys-in-m.room.power.levels.md | 45 +++++++++++++++++++ 1 file changed, 45 insertions(+) create mode 100644 proposals/2209-auth-rules-other-keys-in-m.room.power.levels.md diff --git a/proposals/2209-auth-rules-other-keys-in-m.room.power.levels.md b/proposals/2209-auth-rules-other-keys-in-m.room.power.levels.md new file mode 100644 index 00000000..93d9158a --- /dev/null +++ b/proposals/2209-auth-rules-other-keys-in-m.room.power.levels.md @@ -0,0 +1,45 @@ +# Update auth rules to check notifications key in m.room.power_levels + +## Introduction + +The key `notifications` was added to the `m.room.power_levels` event after the finalisation of the auth rules specified in room version 1. This leads to the fact, that this dictionary is not subject to the same validation as other dictionaries in the event, such as `users` or `events`. This especially means that Alice is able to alter any entry within the dictionary including ones, that are above her own power level, which is inconsistent with the behaviour for the other two dictionaries. + +[m.room.power_levels](https://matrix.org/docs/spec/client_server/r0.5.0#m-room-power-levels) +[room version 1](https://matrix.org/docs/spec/rooms/v1) + +## Proposal + +The auth rules regarding `m.room.power_levels` have been established in room version 1. The general idea here was that creators of a new `m.room.power_levels` event are only able to make changes to something that is equal to or lower than their own power level. +So, assuming a room with Alice (50), Bob(100), `m.room.power_levels` at 50, `m.room.name` at 75 and `m.room.topic` at 25 would mean the following: + +* Alice CAN alter `m.room.topic` to any power level up to her own, in this case 50 +* Alice is NOT able to alter `m.room.name` since the current value is higher than her own (75 > 50) +* Alice is NOT able to alter the power level of Bob, since his current value is higher than her own (100 > 50) +* Alice is free to set the level for anything that has not been defined such as `org.alice.message` up to a maximum of 50 + +Later on the key `notifications` was added to the `m.room.power_levels` event. It contains a mapping of notification keys to a power level, that is required for a user to trigger the specific notification. The most famous notification type is the `@room` notification. + +Going back to the original example because this key was added later on, the auth rules make no mention of it, which enables the following behaviour. *It is assumed that `@room` is at 75* + +* Alice can add any key to the dictionary and set the value to any value she wants, including ones higher than her own. +* Alice can alter the value for `@room` to any value she wants, including ones that are higher than her own, even though her own value is lower. + +The proposed solution is to alter the auth rules and add the `notification` dictionary to the same rules that handle `events` and `users`. + +So the rule [10.d](https://matrix.org/docs/spec/rooms/v1.html) of the auth rules in room version 1 would be altered to: + + +For each entry being added, changed or removed in events, users __and notifications__ keys: + + +## Tradeoffs + +The proposed solution would be a breaking change with current room versions and the alternative would be to leave the `notifications` key without any checks. + +## Security considerations + +This is likely to improve security because it prevents malicious users that were only given the right to emit `m.room.power_levels` so that they could alter a very specific key, such as, `invite` from altering the rules established for triggering notifications. + +## Conclusion + +The current spec conform behaviour regarding `notifications` is inconsistent with behaviour shown by the other dictionaries and room administrators are very likely expecting the `notifications` to work the same as them. The required change is minimal is and also in line with the general spirit of the auth rules regarding the `m.room.power_levels` event. From 42dbeee7d356e96f22c4b64bda4a3c0594dfd76f Mon Sep 17 00:00:00 2001 From: Luca Becker Date: Thu, 1 Aug 2019 18:32:11 +0200 Subject: [PATCH 002/114] MSC2209: 2nd draft @uhoregs changes --- ...rules-other-keys-in-m.room.power.levels.md | 64 ++++++++++++++----- 1 file changed, 48 insertions(+), 16 deletions(-) diff --git a/proposals/2209-auth-rules-other-keys-in-m.room.power.levels.md b/proposals/2209-auth-rules-other-keys-in-m.room.power.levels.md index 93d9158a..db4d9bee 100644 --- a/proposals/2209-auth-rules-other-keys-in-m.room.power.levels.md +++ b/proposals/2209-auth-rules-other-keys-in-m.room.power.levels.md @@ -2,44 +2,76 @@ ## Introduction -The key `notifications` was added to the `m.room.power_levels` event after the finalisation of the auth rules specified in room version 1. This leads to the fact, that this dictionary is not subject to the same validation as other dictionaries in the event, such as `users` or `events`. This especially means that Alice is able to alter any entry within the dictionary including ones, that are above her own power level, which is inconsistent with the behaviour for the other two dictionaries. +The key `notifications` was added to the `m.room.power_levels` event after the +finalisation of the auth rules specified in room version 1. This leads to the +fact, that this dictionary is not subject to the same validation as other +dictionaries in the event, such as `users` or `events`. This especially means +that Alice is able to alter any entry within the dictionary including ones, +that are above her own power level, which is inconsistent with the behaviour +for the other two dictionaries. [m.room.power_levels](https://matrix.org/docs/spec/client_server/r0.5.0#m-room-power-levels) [room version 1](https://matrix.org/docs/spec/rooms/v1) ## Proposal -The auth rules regarding `m.room.power_levels` have been established in room version 1. The general idea here was that creators of a new `m.room.power_levels` event are only able to make changes to something that is equal to or lower than their own power level. -So, assuming a room with Alice (50), Bob(100), `m.room.power_levels` at 50, `m.room.name` at 75 and `m.room.topic` at 25 would mean the following: +The auth rules regarding `m.room.power_levels` have been established in room +version 1. The general idea here was that creators of a new `m.room.power_levels` +event are only able to make changes to something that is equal to or lower than +their own power level. +So, assuming a room with Alice (50), Bob(100), `m.room.power_levels` at 50, +`m.room.name` at 75 and `m.room.topic` at 25 would mean the following: * Alice CAN alter `m.room.topic` to any power level up to her own, in this case 50 -* Alice is NOT able to alter `m.room.name` since the current value is higher than her own (75 > 50) -* Alice is NOT able to alter the power level of Bob, since his current value is higher than her own (100 > 50) -* Alice is free to set the level for anything that has not been defined such as `org.alice.message` up to a maximum of 50 +* Alice is NOT able to alter `m.room.name` since the current value is higher than +her own (75 > 50) +* Alice is NOT able to alter the power level of Bob, since his current value is +higher than her own (100 > 50) +* Alice is free to set the level for anything that has not been defined such as +`org.alice.message` up to a maximum of 50 -Later on the key `notifications` was added to the `m.room.power_levels` event. It contains a mapping of notification keys to a power level, that is required for a user to trigger the specific notification. The most famous notification type is the `@room` notification. +Later on the key `notifications` was added to the `m.room.power_levels` event. +It contains a mapping of notification keys to a power level, that is required +for a user to trigger the specific notification. The most famous notification +type is the `@room` notification. -Going back to the original example because this key was added later on, the auth rules make no mention of it, which enables the following behaviour. *It is assumed that `@room` is at 75* +Going back to the original example because this key was added later on, the auth +rules make no mention of it, which enables the following behaviour. *It is assumed +that `@room` is at 75* -* Alice can add any key to the dictionary and set the value to any value she wants, including ones higher than her own. -* Alice can alter the value for `@room` to any value she wants, including ones that are higher than her own, even though her own value is lower. +* Alice can add any key to the dictionary and set the value to any value she wants, +including ones higher than her own. +* Alice can alter the value for `@room` to any value she wants, including ones that +are higher than her own, even though her own value is lower. -The proposed solution is to alter the auth rules and add the `notification` dictionary to the same rules that handle `events` and `users`. +The proposed solution is to alter the auth rules and add the `notifications` dictionary +to the same rules that handle `events` and `users`. -So the rule [10.d](https://matrix.org/docs/spec/rooms/v1.html) of the auth rules in room version 1 would be altered to: +So the rule [10.d](https://matrix.org/docs/spec/rooms/v1.html) of the auth rules in +room version 1 would be updated in a new room version to: -For each entry being added, changed or removed in events, users __and notifications__ keys: +For each entry being added, changed or removed in events, users __and notifications__ +keys: ## Tradeoffs -The proposed solution would be a breaking change with current room versions and the alternative would be to leave the `notifications` key without any checks. +The proposed solution would be a breaking change with current room versions and +the alternative would be to leave the `notifications` key without any checks. ## Security considerations -This is likely to improve security because it prevents malicious users that were only given the right to emit `m.room.power_levels` so that they could alter a very specific key, such as, `invite` from altering the rules established for triggering notifications. +This is likely to improve security because it prevents malicious users that were +only given the right to emit `m.room.power_levels` so that they could alter a very +specific key, such as `invite`, from altering the rules established for triggering +notifications. ## Conclusion -The current spec conform behaviour regarding `notifications` is inconsistent with behaviour shown by the other dictionaries and room administrators are very likely expecting the `notifications` to work the same as them. The required change is minimal is and also in line with the general spirit of the auth rules regarding the `m.room.power_levels` event. +The current spec conform behaviour regarding `notifications` is inconsistent with +behaviour shown by the other dictionaries and room administrators are very likely +expecting the `notifications` to work the same as them. The required change is minimal +is and also in line with the general spirit of the auth rules regarding the +`m.room.power_levels` event. A new room version is, however, required. This can be +done with other pending changes. From 25e3602bbe0f76285f75abd4277959ee8add4948 Mon Sep 17 00:00:00 2001 From: Luca Becker Date: Sat, 3 Aug 2019 09:15:32 +0200 Subject: [PATCH 003/114] 3rd draft of MSC2209 --- .../2209-auth-rules-other-keys-in-m.room.power.levels.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/proposals/2209-auth-rules-other-keys-in-m.room.power.levels.md b/proposals/2209-auth-rules-other-keys-in-m.room.power.levels.md index db4d9bee..594c0b99 100644 --- a/proposals/2209-auth-rules-other-keys-in-m.room.power.levels.md +++ b/proposals/2209-auth-rules-other-keys-in-m.room.power.levels.md @@ -19,7 +19,7 @@ The auth rules regarding `m.room.power_levels` have been established in room version 1. The general idea here was that creators of a new `m.room.power_levels` event are only able to make changes to something that is equal to or lower than their own power level. -So, assuming a room with Alice (50), Bob(100), `m.room.power_levels` at 50, +So, assuming a room with Alice (50), Bob (100), `m.room.power_levels` at 50, `m.room.name` at 75 and `m.room.topic` at 25 would mean the following: * Alice CAN alter `m.room.topic` to any power level up to her own, in this case 50 @@ -51,8 +51,8 @@ So the rule [10.d](https://matrix.org/docs/spec/rooms/v1.html) of the auth rules room version 1 would be updated in a new room version to: -For each entry being added, changed or removed in events, users __and notifications__ -keys: +> For each entry being added, changed or removed in events, users __and notifications__ +>keys: ## Tradeoffs From b0f873785d338922876990d30e542079888b11d8 Mon Sep 17 00:00:00 2001 From: Travis Ralston Date: Wed, 21 Aug 2019 18:03:17 -0600 Subject: [PATCH 004/114] Proposal for room version 6 --- proposals/2240-rooms-v6.md | 21 +++++++++++++++++++++ 1 file changed, 21 insertions(+) create mode 100644 proposals/2240-rooms-v6.md diff --git a/proposals/2240-rooms-v6.md b/proposals/2240-rooms-v6.md new file mode 100644 index 00000000..a67a1ad7 --- /dev/null +++ b/proposals/2240-rooms-v6.md @@ -0,0 +1,21 @@ +# MSC2240: Room Version 6 + +A new room version, `6`, is proposed using [room version 5](https://matrix.org/docs/spec/rooms/v5.html) +as a starting point and incorporating the following MSCs: + +* [MSC2174](https://github.com/matrix-org/matrix-doc/pull/2174) - Moving the `redacts` key. +* [MSC2175](https://github.com/matrix-org/matrix-doc/pull/2175) - Removing the `creator` from create events. +* [MSC2176](https://github.com/matrix-org/matrix-doc/pull/2176) - Modern redaction rules. +* [MSC2209](https://github.com/matrix-org/matrix-doc/pull/2209) - Including notifications in power level auth rules. +* [MSC2212](https://github.com/matrix-org/matrix-doc/pull/2212) - Third party user power levels. +* [MSC2213](https://github.com/matrix-org/matrix-doc/pull/2213) - Rejoinability of invite-only rooms. +* [MSC1849](https://github.com/matrix-org/matrix-doc/pull/1849) - Event aggregations/relationships. + +MSCs which require a new room version and are excluded from v6 are: +* [MSC2214](https://github.com/matrix-org/matrix-doc/pull/2214) - Joining upgraded private rooms. + This MSC is excluded due to the solution not being fully worked out. As this v6 proposal progresses, + it is possible for it to be included in the future. + + +Room version 6 upon being added to the specification shall be considered stable. No other room versions +are affected by this MSC. From 3ee9a7d654050219f09487ce7b6bd9584827a6c2 Mon Sep 17 00:00:00 2001 From: Travis Ralston Date: Wed, 12 Feb 2020 21:03:11 -0700 Subject: [PATCH 005/114] Attempt to convert MSC2324 to reality We've been using this in practice for a while now, so we should document it more officially. MSC: https://github.com/matrix-org/matrix-doc/pull/2324 --- specification/proposals_intro.rst | 102 +++++++++++++++++++++++++++++- 1 file changed, 101 insertions(+), 1 deletion(-) diff --git a/specification/proposals_intro.rst b/specification/proposals_intro.rst index 82a4225b..3f70345f 100644 --- a/specification/proposals_intro.rst +++ b/specification/proposals_intro.rst @@ -230,7 +230,8 @@ follows: comments and in relevant rooms on Matrix. Discussion outside of GitHub should be summarised in a comment on the PR. - When a member of the Spec Core Team believes that no new discussion points are - being made, they will propose a motion for a final comment period (FCP), + being made, and the proposal has suitable evidence of working (see `implementing a + proposal`_ below), they will propose a motion for a final comment period (FCP), along with a *disposition* of either merge, close or postpone. This FCP is provided to allow a short period of time for any invested party to provide a final objection before a major decision is made. If sufficient reasoning is @@ -341,6 +342,105 @@ Closed proposal-closed A proposal which Obsolete obsolete A proposal which has been made obsolete by another proposal or decision elsewhere. =============================== ============================= ==================================== +Implementing a proposal +----------------------- + +As part of the proposal process the spec core team will require evidence of the MSC +working in order for it to move into FCP. This can usually be a branch/pull request +to whichever implementation of choice that proves the MSC works in practice, though +in some cases the MSC itself will be small enough to be considered proven. Where it's +unclear if a MSC will require an implementation proof, ask in `#matrix-spec:matrix.org +`_. + +Early adoption of a MSC/idea +~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +To help facilitate early releases of software dependent on a spec release, implementations +are required to use the following process to ensure that the official Matrix namespace +is not cluttered with development or testing data. + +.. Note:: + Proof of concept implementations that are solely left on a branch/unreleased and for + demonstrating that a particular MSC works do not have to follow this. Note that the + ``develop`` branch adopted by some projects is still subject to this process. + +The process for an implementation willing to be ahead of a spec release is: + +1. Have an idea for a feature. +2. Implement the feature using unstable endpoints, vendor prefixes, and unstable + feature flags as appropriate. + + * When using unstable endpoints, they MUST include a vendor prefix. For example: + `/_matrix/client/unstable/com.example/login`. Vendor prefixes throughout Matrix + always use the Java package naming convention. The MSC for the feature should + identify which preferred vendor prefix is to be used by early adopters. + * Unstable endpoints **do not** inherit from stable (e.g. `/r0`) APIs. Implementations + cannot assume that a particular endpoint will exist in the unstable namespace + even if the server advertises support for the feature. + * If the client needs to be sure the server supports the feature, an unstable + feature flag that MUST be vendor prefixed is to be used. This kind of flag shows + up in the ``unstable_features`` section of ``/versions`` as, for example, + ``com.example.new_login``. The MSC for the feature should identify which preferred + feature flag is to be used by early adopters. + * When using this approach correctly, the implementation can ship/release the + feature at any time, so long as the implementation is able to accept the technical + debt that results from needing to provide adequate backwards and forwards + compatibility. The implementation MUST support the flag disappearing and be + generally safe for users. Note that implementations early in the MSC review + process may also be required to provide backwards compatibility with earlier + editions of the proposal. + * If the implementation cannot support the technical debt (or if it's impossible + to provide forwards/backwards compatibility - e.g. a user authentication change + which can't be safely rolled back), the implementation should not attempt to + implement the feature and instead wait for a spec release. + * If at any point the idea changes, the feature flag should also change so that + implementations can adapt as needed. + +3. In parallel, or ahead of implementation, open an MSC and solicit review per above. +4. Before FCP can be called, the Spec Core Team will require evidence of the MSC + working as proposed. A typical example of this is an implementation of the MSC, + though the implementation does not need to be shipped anywhere and can therefore + avoid the forwards/backwards compatibility concerns mentioned here. +5. FCP is gone through, and assuming nothing is flagged the MSC lands. +6. A spec PR is written to incorporate the changes into Matrix. +7. A spec release happens. +8. Implementations switch to using stable prefixes (e.g.: ``/r0``) if the server + supports the specification version released. If the server doesn't advertise the + specification version, but does have the feature flag, unstable prefixes should + still be used. +9. A transition period of about 2 months starts immediately after the spec release, + before implementations start to loudly encourage other implementations to switch + to stable endpoints. For example, a server implementation should start asking + client implementations to support the stable endpoints 2 months after the spec + release, if they haven't already. The same applies in the reverse: if clients + cannot switch to stable prefixes because server implementations haven't started + supporting the new spec release, some noise should be raised in the general direction + of the implementation. + + * Please be considerate when being noisy to implementations. A gentle reminder + in their issue tracker is generally good enough. + +.. Note:: + MSCs MUST still describe what the stable endpoints/feature looks like with a note + towards the bottom for what the unstable feature flag/prefixes are. For example, + a MSC would propose `/_matrix/client/r0/new/endpoint`, not `/_matrix/client/unstable/ + com.example/new/endpoint`. + +In summary: + +* Implementations MUST NOT use stable endpoints before the MSC is in the spec. This + includes NOT using stable endpoints before a spec release happens, but after FCP has + passed. +* Implementations are able to ship features that are exposed to users by default before + an MSC has been merged to the spec, provided they follow the process above. +* Implementations SHOULD be wary of the technical debt they are incurring by moving faster + than the spec. +* The vendor prefix is chosen by the developer of the feature, using the Java package + naming convention. The foundation's preferred vendor prefix is `org.matrix`. +* The vendor prefixes, unstable feature flags, and unstable endpoints should be included + in the MSC, though the MSC MUST be written in a way that proposes new stable endpoints. + Typically this is solved by a small table at the bottom mapping the various values + from stable to unstable. Proposal Tracking ----------------- From da795e582478da464defd8f68d2635ea1c41eabc Mon Sep 17 00:00:00 2001 From: Travis Ralston Date: Thu, 5 Mar 2020 17:24:29 -0700 Subject: [PATCH 006/114] Apply suggestions from code review Co-Authored-By: Richard van der Hoff <1389908+richvdh@users.noreply.github.com> --- specification/proposals_intro.rst | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/specification/proposals_intro.rst b/specification/proposals_intro.rst index 3f70345f..c3b312ce 100644 --- a/specification/proposals_intro.rst +++ b/specification/proposals_intro.rst @@ -385,14 +385,14 @@ The process for an implementation willing to be ahead of a spec release is: * When using this approach correctly, the implementation can ship/release the feature at any time, so long as the implementation is able to accept the technical debt that results from needing to provide adequate backwards and forwards - compatibility. The implementation MUST support the flag disappearing and be + compatibility. The implementation MUST support the flag (and server-side implementation) disappearing and be generally safe for users. Note that implementations early in the MSC review process may also be required to provide backwards compatibility with earlier editions of the proposal. * If the implementation cannot support the technical debt (or if it's impossible to provide forwards/backwards compatibility - e.g. a user authentication change which can't be safely rolled back), the implementation should not attempt to - implement the feature and instead wait for a spec release. + implement the feature and should instead wait for a spec release. * If at any point the idea changes, the feature flag should also change so that implementations can adapt as needed. @@ -429,7 +429,7 @@ The process for an implementation willing to be ahead of a spec release is: In summary: * Implementations MUST NOT use stable endpoints before the MSC is in the spec. This - includes NOT using stable endpoints before a spec release happens, but after FCP has + includes NOT using stable endpoints in the period between completion of FCP and release of the spec. passed. * Implementations are able to ship features that are exposed to users by default before an MSC has been merged to the spec, provided they follow the process above. From 79b2e5263d296a735d3346bf4eb8882841671fb6 Mon Sep 17 00:00:00 2001 From: Travis Ralston Date: Sat, 7 Mar 2020 14:05:05 -0700 Subject: [PATCH 007/114] Update specification/proposals_intro.rst Co-Authored-By: Richard van der Hoff <1389908+richvdh@users.noreply.github.com> --- specification/proposals_intro.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/specification/proposals_intro.rst b/specification/proposals_intro.rst index c3b312ce..5ee82dc3 100644 --- a/specification/proposals_intro.rst +++ b/specification/proposals_intro.rst @@ -393,7 +393,7 @@ The process for an implementation willing to be ahead of a spec release is: to provide forwards/backwards compatibility - e.g. a user authentication change which can't be safely rolled back), the implementation should not attempt to implement the feature and should instead wait for a spec release. - * If at any point the idea changes, the feature flag should also change so that + * If at any point after early release, the idea changes in a backwards-incompatible way, the feature flag should also change so that implementations can adapt as needed. 3. In parallel, or ahead of implementation, open an MSC and solicit review per above. From c69a747bae52296f8a7fe930090c64c98227cd73 Mon Sep 17 00:00:00 2001 From: Patrick Cloke Date: Mon, 9 Mar 2020 12:31:06 -0400 Subject: [PATCH 008/114] Convert the initial Google Doc to Markdown. --- proposals/xxxx-ui-interactive-auth-for-sso.md | 262 ++++++++++++++++++ 1 file changed, 262 insertions(+) create mode 100644 proposals/xxxx-ui-interactive-auth-for-sso.md diff --git a/proposals/xxxx-ui-interactive-auth-for-sso.md b/proposals/xxxx-ui-interactive-auth-for-sso.md new file mode 100644 index 00000000..fc7413fe --- /dev/null +++ b/proposals/xxxx-ui-interactive-auth-for-sso.md @@ -0,0 +1,262 @@ +# User-Interactive Auth for SSO-backed homeservers + +## Background + +Certain endpoints, such as `DELETE /_matrix/client/r0/devices/{deviceId}` and +`POST /_matrix/client/r0/account/3pid/add`, require the user to reconfirm their +identity, as a guard against a leaked access token being used to take over an +entire account. + +On a normal homeserver, this is done by prompting the user to enter their +password. However, on a homeserver where users authenticate via a single-sign-on +system, the user doesn't have a password, so this doesn't work. Instead we need +to delegate that check to the SSO system. + +At the protocol level, this means adding support for SSO to +[user-interactive auth](https://matrix.org/docs/spec/client_server/r0.6.0#user-interactive-authentication-api). + +### The current implementation + +Or, "how UI Auth works, in practice": + +When the client calls one of the protected endpoints, it initially returns a 401 +response. For example: + +``` +POST /_matrix/client/r0/delete_devices HTTP/1.1 +Content-Type: application/json + +{ + "devices": ["FSVVTZRRAA"] +} + +HTTP/1.1 401 Unauthorized +Content-Type: application/json + +{ + "flows": [ + { + "stages": [ + "m.login.password" + ] + } + ], + "params": {}, + "session": "dTKfsLHSAJeAhqfxUsvrIVJd" +} +``` + +The client: + +* inspects the "flows" list +* discovers there is a flow it knows how to follow +* carries out the first "stage" of that flow (m.login.password) + +ie, the client prompts the user to enter a password. + +The client then resubmits with an additional 'auth' param, with "type" giving +the name of the authentication type it has just carried out. That completes the +authentication flow, so the server is now happy, and returns a 200: + +``` +POST /_matrix/client/r0/delete_devices HTTP/1.1 +Content-Type: application/json + +{ + "devices": ["FSVVTZRRAA"], + "auth": { + "session":"dTKfsLHSAJeAhqfxUsvrIVJd", + "type":"m.login.password", + "identifier":{"type":"m.id.user", "user":"@userid:matrix.org"}, + "password":"" + } +} + + +HTTP/1.1 200 OK +Content-Type: application/json +Content-Length: 179 + +{} +``` + +## Proposal + +We add an `m.login.sso` authentication type to the UI auth spec. There are no +additional params as part of this auth type. + +1. If the client wants to complete that authentication type, it opens a browser + window for `/_matrix/client/r0/auth/m.login.sso/fallback/web?session=<...>` + with session set to the UI-Auth session id (from the "auth" dict). + + The homeserver returns a page which says words to the effect of "A client is + trying to remove a device/add an email address/take over your account. To + confirm this action, **re-authenticate with single sign-on**. If you did not + expect this, your account may be compromised!" + + See security section below. +2. The link, once the user clicks on it, goes to the SSO provider's page. +3. The SSO provider validates the user, and redirects the browser back to the + homeserver. +4. The homeserver validates the response from the SSO provider, updates the + user-interactive auth session to show that the SSO has completed, serves the + fallback auth completion page as specced: + https://matrix.org/docs/spec/client_server/r0.6.0#fallback +5. The client resubmits its original request, with its original session id, + which now hopefully completes. + +Note that the post-SSO URL on the homeserver is left up to the homeserver +implementation rather than forming part of the spec. + +* SAML2 servers typically only support one URL per service provider, so in + practice it will need to be the same as that already used for the login flow + (for synapse, it's /_matrix/saml2/authn_response) - and the server needs to + be able to figure out if it's doing SSO for a login attempt or an SSO + attempt. +* CAS doesn't have the same restriction. + +### Example flow: + +0. Client submits the request, which the server says requires SSO: + + ``` + POST /_matrix/client/r0/delete_devices HTTP/1.1 + Content-Type: application/json + Authorization: Bearer xyzzy + + { + "devices": ["FSVVTZRRAA"] + } + + HTTP/1.1 401 Unauthorized + Content-Type: application/json + + { + "flows": [ + { + "stages": [ + "m.login.sso" + ] + } + ], + "params": {}, + "session": "dTKfsLHSAJeAhqfxUsvrIVJd" + } + ``` + +1. Client opens a browser window for the fallback endpoint: + + ``` + GET /_matrix/client/r0/auth/m.login.sso/fallback/web + ?session=dTKfsLHSAJeAhqfxUsvrIVJd HTTP/1.1 + + HTTP/1.1 200 OK + + + can delete device pls? + clicky + + ``` + +2. The user clicks the confirmation link which goes to the SSO provider's site: + + ``` + GET https://sso_provider/validate?SAMLRequest= HTTP/1.1 + + ... SAML/CAS stuff + ``` + +3. The SSO provider validates the user and ends up redirecting the browser back + to the homeserver. (The example below shows a 302 for simplicity but SAML normally uses a 200 with an html
, with javascript to automatically submit it.) + + ``` + HTTP/1.1 302 Moved + Location: https://homeserver/_matrix/saml2/authn_response? + SAMLResponse= + ``` + +4. The browser sends the SSO response to the homeserver, which validates it and + shows the fallback auth completion page: + + ``` + GET /_matrix/saml2/authn_response?SAMLResponse= + + + HTTP/1.1 200 OK + + + +

Thank you.

+

You may now close this window and return to the application.

+ ``` + +5. The client closes the browser popup if it is still open, and resubmits its + original request, which now succeeds: + + ``` + POST /_matrix/client/r0/delete_devices HTTP/1.1 + Content-Type: application/json + Authorization: Bearer xyzzy + + { + "auth": { + "session": "dTKfsLHSAJeAhqfxUsvrIVJd" + } + } + + HTTP/1.1 200 OK + Content-Type: application/json + + {} + ``` + +A few notes: + +* The security of this relies on UI-Auth sessions only being used for the same + request as they were initiated for. I don't think that's currently enforced. +* We might consider an alternative client flow where the fallback auth ends up + redirecting to a given URI, instead of doing javascript postMessage foo. I + think that's an orthogonal change to the fallback auth though. +* In theory, any clients that already implement the fallback process for + unknown authentication types will work fine without modification. + Unfortunately, I don't think Riot (on any platform) is among them. + +## Security considerations + +Recall that the thing we are trying to guard against here is a single leaked +access-token being used to take over an entire account. So let's assume the +attacker has got hold of an access token for your account. What happens if we + skip the confirmation step? + +The attacker, who has your access token, starts a UI-Auth session to add his +email address to your account. + +He then sends you a link "hey, check out this cool video!"; the link leads (via +an innocent-looking url shortener) to +`/_matrix/client/r0/auth/m.login.sso/fallback/web?session=<...>`, with the ID of +the session that he just created. + +Recall that we're skipping the confirmation step, so the server redirects +straight to the SSO provider. + +It's common for SSO providers to redirect straight back to the app if you've +recently authenticated with them; even in the best case, the SSO provider shows +some innocent message along the lines of "Confirm that you want to sign in to +". + +So the SSO completes, and the attacker's session is validated. + +By telling the user what's about to happen as clearly as we can, and making them +confirm, we can mitigate this problem. + +## Unstable prefix + +We should use a vendor prefix here until this hits the spec. + +`org.matrix.login.sso` ? \ No newline at end of file From f48bbd32788ec8734fbaaa85728197632aeffd61 Mon Sep 17 00:00:00 2001 From: Patrick Cloke Date: Mon, 9 Mar 2020 14:00:28 -0400 Subject: [PATCH 009/114] Attempt to clarify the MSC. --- proposals/xxxx-ui-interactive-auth-for-sso.md | 90 ++++++++++--------- 1 file changed, 47 insertions(+), 43 deletions(-) diff --git a/proposals/xxxx-ui-interactive-auth-for-sso.md b/proposals/xxxx-ui-interactive-auth-for-sso.md index fc7413fe..5b6f396d 100644 --- a/proposals/xxxx-ui-interactive-auth-for-sso.md +++ b/proposals/xxxx-ui-interactive-auth-for-sso.md @@ -1,6 +1,4 @@ -# User-Interactive Auth for SSO-backed homeservers - -## Background +# User-Interactive Auth for SSO-backed homeserver Certain endpoints, such as `DELETE /_matrix/client/r0/devices/{deviceId}` and `POST /_matrix/client/r0/account/3pid/add`, require the user to reconfirm their @@ -9,15 +7,17 @@ entire account. On a normal homeserver, this is done by prompting the user to enter their password. However, on a homeserver where users authenticate via a single-sign-on -system, the user doesn't have a password, so this doesn't work. Instead we need -to delegate that check to the SSO system. +system, the user doesn't have a password registered with the homeserver. Instead +we need to delegate that check to the SSO system. At the protocol level, this means adding support for SSO to [user-interactive auth](https://matrix.org/docs/spec/client_server/r0.6.0#user-interactive-authentication-api). -### The current implementation +In theory, any clients that already implement the fallback process for unknown +authentication types will work fine without modification. It is unknown whether +this is widely supported among clients. -Or, "how UI Auth works, in practice": +### UI Auth Overview When the client calls one of the protected endpoints, it initially returns a 401 response. For example: @@ -52,7 +52,7 @@ The client: * discovers there is a flow it knows how to follow * carries out the first "stage" of that flow (m.login.password) -ie, the client prompts the user to enter a password. +In this example, the client prompts the user to enter a password. The client then resubmits with an additional 'auth' param, with "type" giving the name of the authentication type it has just carried out. That completes the @@ -89,24 +89,25 @@ additional params as part of this auth type. window for `/_matrix/client/r0/auth/m.login.sso/fallback/web?session=<...>` with session set to the UI-Auth session id (from the "auth" dict). - The homeserver returns a page which says words to the effect of "A client is - trying to remove a device/add an email address/take over your account. To - confirm this action, **re-authenticate with single sign-on**. If you did not - expect this, your account may be compromised!" + The homeserver returns a page which asks for the user's confirmation before + proceeding. See the security considerations section below for why this is + necessary. For example, the page could say words to the effect of: - See security section below. + > A client is trying to remove a device/add an email address/take over your + > account. To confirm this action, **re-authenticate with single sign-on**. + > If you did not expect this, your account may be compromised! 2. The link, once the user clicks on it, goes to the SSO provider's page. 3. The SSO provider validates the user, and redirects the browser back to the homeserver. 4. The homeserver validates the response from the SSO provider, updates the - user-interactive auth session to show that the SSO has completed, serves the - fallback auth completion page as specced: - https://matrix.org/docs/spec/client_server/r0.6.0#fallback + user-interactive auth session to show that the SSO has completed, + [serves the fallback auth completion page as specced](https://matrix.org/docs/spec/client_server/r0.6.0#fallback). 5. The client resubmits its original request, with its original session id, - which now hopefully completes. + which now should complete. Note that the post-SSO URL on the homeserver is left up to the homeserver -implementation rather than forming part of the spec. +implementation rather than forming part of the specification, choices might be +limited by the chosen SSO implementation, for example: * SAML2 servers typically only support one URL per service provider, so in practice it will need to be the same as that already used for the login flow @@ -216,47 +217,50 @@ implementation rather than forming part of the spec. {} ``` -A few notes: +## Alternatives -* The security of this relies on UI-Auth sessions only being used for the same - request as they were initiated for. I don't think that's currently enforced. -* We might consider an alternative client flow where the fallback auth ends up - redirecting to a given URI, instead of doing javascript postMessage foo. I - think that's an orthogonal change to the fallback auth though. -* In theory, any clients that already implement the fallback process for - unknown authentication types will work fine without modification. - Unfortunately, I don't think Riot (on any platform) is among them. +An alternative client flow where the fallback auth ends up redirecting to a +given URI, instead of doing JavaScript postMessage foo could be considered. +This is probably an orthogonal change to the fallback auth though. ## Security considerations +### Why we need user to confirm before the SSO flow + Recall that the thing we are trying to guard against here is a single leaked access-token being used to take over an entire account. So let's assume the -attacker has got hold of an access token for your account. What happens if we - skip the confirmation step? +attacker has got hold of an access token for your account. What happens if the +confirmation step is skipped? -The attacker, who has your access token, starts a UI-Auth session to add his -email address to your account. +The attacker, who has your access token, starts a UI Authentication session to +add their email address to your account. -He then sends you a link "hey, check out this cool video!"; the link leads (via -an innocent-looking url shortener) to +They then sends you a link "hey, check out this cool video!"; the link leads (via +an innocent-looking URL shortener or some other phishing technique) to `/_matrix/client/r0/auth/m.login.sso/fallback/web?session=<...>`, with the ID of the session that he just created. -Recall that we're skipping the confirmation step, so the server redirects -straight to the SSO provider. +Since there is no confirmation step, the server redirects directly to the SSO +provider. It's common for SSO providers to redirect straight back to the app if you've recently authenticated with them; even in the best case, the SSO provider shows -some innocent message along the lines of "Confirm that you want to sign in to -". +an innocent message along the lines of "Confirm that you want to sign in to +". -So the SSO completes, and the attacker's session is validated. +After redirecting back to the homeserver, the SSO is completed and the +attacker's session is validated. They are now able to make their malicious +change to your account. -By telling the user what's about to happen as clearly as we can, and making them -confirm, we can mitigate this problem. +This problem can be mitigated by clearly telling the user what is about to happen. + +### Reusing UI-Auth sessions + +The security of this relies on UI-Auth sessions only being used for the same +request as they were initiated for. It is not believed that this is currently +enforced. ## Unstable prefix -We should use a vendor prefix here until this hits the spec. - -`org.matrix.login.sso` ? \ No newline at end of file +A vendor prefix of `org.matrix.login.sso` (instead of `m.login.sso` is proposed +until this is part of the specification. From 4d177753e1e8043b92f9bcec0c4501b9a2802a63 Mon Sep 17 00:00:00 2001 From: Patrick Cloke Date: Mon, 9 Mar 2020 15:05:37 -0400 Subject: [PATCH 010/114] Move proposal to proper proposal number. --- ...active-auth-for-sso.md => 2454-ui-interactive-auth-for-sso.md} | 0 1 file changed, 0 insertions(+), 0 deletions(-) rename proposals/{xxxx-ui-interactive-auth-for-sso.md => 2454-ui-interactive-auth-for-sso.md} (100%) diff --git a/proposals/xxxx-ui-interactive-auth-for-sso.md b/proposals/2454-ui-interactive-auth-for-sso.md similarity index 100% rename from proposals/xxxx-ui-interactive-auth-for-sso.md rename to proposals/2454-ui-interactive-auth-for-sso.md From 570398e045476fcf4e289367d4856d44b0882b67 Mon Sep 17 00:00:00 2001 From: Patrick Cloke Date: Tue, 10 Mar 2020 10:00:58 -0400 Subject: [PATCH 011/114] Remove section on the how the authentication currently works and just reference the specification. --- proposals/2454-ui-interactive-auth-for-sso.md | 65 +------------------ 1 file changed, 1 insertion(+), 64 deletions(-) diff --git a/proposals/2454-ui-interactive-auth-for-sso.md b/proposals/2454-ui-interactive-auth-for-sso.md index 5b6f396d..bfad25a9 100644 --- a/proposals/2454-ui-interactive-auth-for-sso.md +++ b/proposals/2454-ui-interactive-auth-for-sso.md @@ -11,75 +11,12 @@ system, the user doesn't have a password registered with the homeserver. Instead we need to delegate that check to the SSO system. At the protocol level, this means adding support for SSO to -[user-interactive auth](https://matrix.org/docs/spec/client_server/r0.6.0#user-interactive-authentication-api). +[user-interactive authentication API](https://matrix.org/docs/spec/client_server/r0.6.0#user-interactive-authentication-api). In theory, any clients that already implement the fallback process for unknown authentication types will work fine without modification. It is unknown whether this is widely supported among clients. -### UI Auth Overview - -When the client calls one of the protected endpoints, it initially returns a 401 -response. For example: - -``` -POST /_matrix/client/r0/delete_devices HTTP/1.1 -Content-Type: application/json - -{ - "devices": ["FSVVTZRRAA"] -} - -HTTP/1.1 401 Unauthorized -Content-Type: application/json - -{ - "flows": [ - { - "stages": [ - "m.login.password" - ] - } - ], - "params": {}, - "session": "dTKfsLHSAJeAhqfxUsvrIVJd" -} -``` - -The client: - -* inspects the "flows" list -* discovers there is a flow it knows how to follow -* carries out the first "stage" of that flow (m.login.password) - -In this example, the client prompts the user to enter a password. - -The client then resubmits with an additional 'auth' param, with "type" giving -the name of the authentication type it has just carried out. That completes the -authentication flow, so the server is now happy, and returns a 200: - -``` -POST /_matrix/client/r0/delete_devices HTTP/1.1 -Content-Type: application/json - -{ - "devices": ["FSVVTZRRAA"], - "auth": { - "session":"dTKfsLHSAJeAhqfxUsvrIVJd", - "type":"m.login.password", - "identifier":{"type":"m.id.user", "user":"@userid:matrix.org"}, - "password":"" - } -} - - -HTTP/1.1 200 OK -Content-Type: application/json -Content-Length: 179 - -{} -``` - ## Proposal We add an `m.login.sso` authentication type to the UI auth spec. There are no From 78e08c1987460cd14eba27f503b95b6dc4083a29 Mon Sep 17 00:00:00 2001 From: Patrick Cloke Date: Tue, 10 Mar 2020 10:11:08 -0400 Subject: [PATCH 012/114] Attempt to clarify the proposed changes. --- proposals/2454-ui-interactive-auth-for-sso.md | 38 +++++++++++-------- 1 file changed, 22 insertions(+), 16 deletions(-) diff --git a/proposals/2454-ui-interactive-auth-for-sso.md b/proposals/2454-ui-interactive-auth-for-sso.md index bfad25a9..d3cab51b 100644 --- a/proposals/2454-ui-interactive-auth-for-sso.md +++ b/proposals/2454-ui-interactive-auth-for-sso.md @@ -1,4 +1,4 @@ -# User-Interactive Auth for SSO-backed homeserver +# User-Interactive Authentication for SSO-backed homeserver Certain endpoints, such as `DELETE /_matrix/client/r0/devices/{deviceId}` and `POST /_matrix/client/r0/account/3pid/add`, require the user to reconfirm their @@ -10,17 +10,21 @@ password. However, on a homeserver where users authenticate via a single-sign-on system, the user doesn't have a password registered with the homeserver. Instead we need to delegate that check to the SSO system. -At the protocol level, this means adding support for SSO to +At the protocol level, this means adding support for SSO to the [user-interactive authentication API](https://matrix.org/docs/spec/client_server/r0.6.0#user-interactive-authentication-api). -In theory, any clients that already implement the fallback process for unknown -authentication types will work fine without modification. It is unknown whether -this is widely supported among clients. +In theory, once SSO is added as a possible flow for authentication any clients +that already implement the [fallback process for unknown authentication types](https://matrix.org/docs/spec/client_server/r0.6.0#fallback) +will work fine without modification. It is unknown whether this is widely +supported among clients. ## Proposal -We add an `m.login.sso` authentication type to the UI auth spec. There are no -additional params as part of this auth type. +An [additional authentication type](https://matrix.org/docs/spec/client_server/r0.6.0#authentication-types) +of `m.login.sso` is added to the user-interactive authentication specification. +There are no additional parameters as part of this authentication type. + +When choosing this authentication flow, the following should occur: 1. If the client wants to complete that authentication type, it opens a browser window for `/_matrix/client/r0/auth/m.login.sso/fallback/web?session=<...>` @@ -91,8 +95,9 @@ limited by the chosen SSO implementation, for example: HTTP/1.1 200 OK - can delete device pls? - clicky + A client is trying to remove a device from your account. To confirm this + action, re-authenticate with single sign-on. + If you did not expect this, your account may be compromised! ``` @@ -101,11 +106,12 @@ limited by the chosen SSO implementation, for example: ``` GET https://sso_provider/validate?SAMLRequest= HTTP/1.1 - ... SAML/CAS stuff + ``` 3. The SSO provider validates the user and ends up redirecting the browser back - to the homeserver. (The example below shows a 302 for simplicity but SAML normally uses a 200 with an html , with javascript to automatically submit it.) + to the homeserver. The example below shows a 302 for simplicity, this might + vary based on SSO implementation. ``` HTTP/1.1 302 Moved @@ -191,13 +197,13 @@ change to your account. This problem can be mitigated by clearly telling the user what is about to happen. -### Reusing UI-Auth sessions +### Reusing User Interactive Authentication sessions -The security of this relies on UI-Auth sessions only being used for the same -request as they were initiated for. It is not believed that this is currently -enforced. +The security of this relies on User Interactive Authentication sessions only +being used for the same request as they were initiated for. It is not believed +that this is currently enforced. ## Unstable prefix -A vendor prefix of `org.matrix.login.sso` (instead of `m.login.sso` is proposed +A vendor prefix of `org.matrix.login.sso` is proposed (instead of `m.login.sso`) until this is part of the specification. From 1facf7ff12834ad06fcaff9ee5e2cdf9d1302837 Mon Sep 17 00:00:00 2001 From: Patrick Cloke Date: Thu, 12 Mar 2020 14:14:20 -0400 Subject: [PATCH 013/114] Add an MSC proposal for how to handle sessions when modifying a password. --- ...sword-modification-invalidating-devices.md | 53 +++++++++++++++++++ 1 file changed, 53 insertions(+) create mode 100644 proposals/2457-password-modification-invalidating-devices.md diff --git a/proposals/2457-password-modification-invalidating-devices.md b/proposals/2457-password-modification-invalidating-devices.md new file mode 100644 index 00000000..910e7b62 --- /dev/null +++ b/proposals/2457-password-modification-invalidating-devices.md @@ -0,0 +1,53 @@ +# Invalidating devices during password modification + +There are multiple use cases for why a user might want to modify their password: + +* Adopting a password manager (to use a unique password or more secure password). +* Password rotation. +* Re-secure a compromised account. +* ... probably tons of others ... + +These can be summarized into two groups: + +1. "My account has been compromised and I need to re-secure it." +2. "I just want to change my password." + +The [current Matrix specification](https://matrix.org/docs/spec/client_server/r0.6.0#post-matrix-client-r0-account-password) +does not provide a way to differentiate between these use cases. It gives no +guidance into what should happen to other sessions / devices when a password is +modified and leaves it up to the implementation. + +It is reasonable for a client to want to specify this behavior to offer two +different workflows: + +1. Modify a password and log all other devices out (for use when an account has + been compromised). +2. Modify a password and do not touch any session data (for use in a + non-malicious situations). + +Alternately a client may default to whichever workflow is best for their users. + +## Proposal + +An optional field is added to the JSON body body of the [password reset endpoint](https://matrix.org/docs/spec/client_server/r0.6.0#post-matrix-client-r0-account-password) +called `logout_devices`. This is a boolean flag (defaulting to `true`) that +signals to whether other devices and sessions should be invalidated after +modifying the password. + +## Potential issues + +While Synapse defaults to the behavior of invalidating other devices and +sessions this may not be true of other implementations. Thus, a default of +`true` may not be backwards compatible. It might be more prudent to specify that +the behavior of not specifying the `logout_devices` flag is undefined. + +## Alternatives + +Provide a new endpoint in a future version that supports an additional field (as +above), using a new endpoint would avoid backwards compatibility issues. + +## Security considerations + +By defaulting to invalidating devices and sessions the security considerations +of this endpoint should remain intact. A client will need to be modified to +choose to keep other devices active. From 61b306f8a8a63005f2ab131d32cbf1d5409ddada Mon Sep 17 00:00:00 2001 From: Patrick Cloke Date: Fri, 13 Mar 2020 07:12:57 -0400 Subject: [PATCH 014/114] Remove concerns about backwards compatibility. --- ...-password-modification-invalidating-devices.md | 15 +++++++++------ 1 file changed, 9 insertions(+), 6 deletions(-) diff --git a/proposals/2457-password-modification-invalidating-devices.md b/proposals/2457-password-modification-invalidating-devices.md index 910e7b62..9af6e229 100644 --- a/proposals/2457-password-modification-invalidating-devices.md +++ b/proposals/2457-password-modification-invalidating-devices.md @@ -36,15 +36,18 @@ modifying the password. ## Potential issues -While Synapse defaults to the behavior of invalidating other devices and -sessions this may not be true of other implementations. Thus, a default of -`true` may not be backwards compatible. It might be more prudent to specify that -the behavior of not specifying the `logout_devices` flag is undefined. +The specification states: + +> The homeserver SHOULD NOT revoke the access token provided in the request, +> however all other access tokens for the user should be revoked if the request +> succeeds. + +Defaulting `logout_devices` to `true` should be backwards compatible. ## Alternatives -Provide a new endpoint in a future version that supports an additional field (as -above), using a new endpoint would avoid backwards compatibility issues. +A new endpoint could be provided in a future version of the specification that +supports an additional field (as described above). ## Security considerations From 2d2731efff99ddeb67bdb16a716ed1eba8660189 Mon Sep 17 00:00:00 2001 From: Patrick Cloke Date: Mon, 16 Mar 2020 10:42:17 -0400 Subject: [PATCH 015/114] Remove duplicated word. Co-Authored-By: Andrew Morgan <1342360+anoadragon453@users.noreply.github.com> --- proposals/2457-password-modification-invalidating-devices.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/proposals/2457-password-modification-invalidating-devices.md b/proposals/2457-password-modification-invalidating-devices.md index 9af6e229..9b59156c 100644 --- a/proposals/2457-password-modification-invalidating-devices.md +++ b/proposals/2457-password-modification-invalidating-devices.md @@ -29,7 +29,7 @@ Alternately a client may default to whichever workflow is best for their users. ## Proposal -An optional field is added to the JSON body body of the [password reset endpoint](https://matrix.org/docs/spec/client_server/r0.6.0#post-matrix-client-r0-account-password) +An optional field is added to the JSON body of the [password reset endpoint](https://matrix.org/docs/spec/client_server/r0.6.0#post-matrix-client-r0-account-password) called `logout_devices`. This is a boolean flag (defaulting to `true`) that signals to whether other devices and sessions should be invalidated after modifying the password. From eb17e5d826008a82ca197866a03761ddb6fe9480 Mon Sep 17 00:00:00 2001 From: Hubert Chathi Date: Fri, 14 Feb 2020 16:20:10 -0500 Subject: [PATCH 016/114] switch to symmetric algorithm --- proposals/1946-secure_server-side_storage.md | 52 ++++++++++---------- 1 file changed, 25 insertions(+), 27 deletions(-) diff --git a/proposals/1946-secure_server-side_storage.md b/proposals/1946-secure_server-side_storage.md index fd907e53..59f02d8b 100644 --- a/proposals/1946-secure_server-side_storage.md +++ b/proposals/1946-secure_server-side_storage.md @@ -123,38 +123,33 @@ and the key descriptions for the keys would be: #### Encryption algorithms -##### `m.secret_storage.v1.curve25519-aes-sha2` +##### `m.secret_storage.v1.aes-hmac-sha2` -The public key is stored in the `pubkey` property of the `m.secret_storage.key.[key -ID]` account_data as a base64-encoded string. +Secrets are encrypted using AES-CTR-256 and MACed using HMAC-SHA-256. The data +is encrypted and MACed as follows: -The data is encrypted and MACed as follows: +1. Given the secret storage key, generate 64 bytes by performing an HKDF with + SHA-256 as the hash, a salt of 32 bytes of 0, and with the secret name as + the info. The first 32 bytes are used as the AES key, and the next 32 bytes + are used as the MAC key +2. Generate 16 random bytes, set bit 63 to 0 (in order to work around + differences in AES-CTR implementations), and use this as the AES + initialization vector. This becomes the `iv` property, encoded using base64. +3. Encrypt the data using AES-CTR-256 using the AES key generated above. This + encrypted data, encoded using base64, becomes the `ciphertext` property. +4. Pass the raw encrypted data (prior to base64 encoding) through HMAC-SHA-256 + using the MAC key generated above. The resulting MAC is base64-encoded and + becomes the `mac` property. -1. Generate an ephemeral curve25519 key, and perform an ECDH with the ephemeral - key and the public key to generate a shared secret. The public half of the - ephemeral key, encoded using base64, becomes the `ephemeral` property. -2. Using the shared secret, generate 80 bytes by performing an HKDF using - SHA-256 as the hash, with a salt of 32 bytes of 0, and with the empty string - as the info. The first 32 bytes are used as the AES key, the next 32 bytes - are used as the MAC key, and the last 16 bytes are used as the AES - initialization vector. -4. Encrypt the data using AES-CBC-256 with PKCS#7 padding. This encrypted - data, encoded using base64, becomes the `ciphertext` property. -5. Pass the raw encrypted data (prior to base64 encoding) through HMAC-SHA-256 - using the MAC key generated above. The first 8 bytes of the resulting MAC - are base64-encoded, and become the `mac` property. +(We use AES-CTR to match file encryption and key exports.) -(The key HKDF, AES, and HMAC steps are the same as what are used for encryption -in olm and megolm.) - -For example, the `m.secret_storage.key.[key ID]` for a key using this algorithm +For example, the `m.secret_storage.key.key_id` for a key using this algorithm could look like: ```json { "name": "m.default", - "algorithm": "m.secret_storage.v1.curve25519-aes-sha2", - "pubkey": "base64+public+key" + "algorithm": "m.secret_storage.v1.aes-hmac-sha2" } ``` @@ -164,8 +159,8 @@ and data encrypted using this algorithm could look like this: { "encrypted": { "key_id": { + "iv": "16+bytes+base64", "ciphertext": "base64+encoded+encrypted+data", - "ephemeral": "base64+ephemeral+key", "mac": "base64+encoded+mac" } } @@ -174,7 +169,7 @@ and data encrypted using this algorithm could look like this: ###### Keys -When a user is given a raw key for `m.secret_storage.v1.curve25519-aes-sha2`, +When a user is given a raw key for `m.secret_storage.v1.aes-hmac-sha2`, it will be encoded as follows (this is the same as what is proposed in MSC1703): * prepend the two bytes 0x8b and 0x01 to the key @@ -200,7 +195,8 @@ ID]` account-data: "passphrase": { "algorithm": "m.pbkdf2", "salt": "MmMsAlty", - "iterations": 100000 + "iterations": 100000, + "bits": 256 }, ... } @@ -209,7 +205,9 @@ ID]` account-data: **`m.pbkdf2`** The key is generated using PBKDF2 using the salt given in the `salt` parameter, -and the number of iterations given in the `iterations` parameter. +and the number of iterations given in the `iterations` parameter. The key size +that is generated is given by the `bits` parameter, or 256 bits if no `bits` +parameter is given. ### Sharing From f7ebda040e4c9d476da81cd6bd48e56b3affa805 Mon Sep 17 00:00:00 2001 From: Hubert Chathi Date: Tue, 24 Mar 2020 15:35:32 -0400 Subject: [PATCH 017/114] update some examples to use new algorithm name --- proposals/1946-secure_server-side_storage.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/proposals/1946-secure_server-side_storage.md b/proposals/1946-secure_server-side_storage.md index 59f02d8b..61950b5f 100644 --- a/proposals/1946-secure_server-side_storage.md +++ b/proposals/1946-secure_server-side_storage.md @@ -43,7 +43,7 @@ A key with ID `abcdefg` is stored in `m.secret_storage.key.abcdefg` ```json { "name": "Some key", - "algorithm": "m.secret_storage.v1.curve25519-aes-sha2", + "algorithm": "m.secret_storage.v1.aes-hmac-sha2", // ... other properties according to algorithm } ``` @@ -106,7 +106,7 @@ and the key descriptions for the keys would be: ```json { "name": "Some key", - "algorithm": "m.secret_storage.v1.curve25519-aes-sha2", + "algorithm": "m.secret_storage.v1.aes-hmac-sha2", // ... other properties according to algorithm } ``` @@ -116,7 +116,7 @@ and the key descriptions for the keys would be: ```json { "name": "Some other key", - "algorithm": "m.secret_storage.v1.curve25519-aes-sha2", + "algorithm": "m.secret_storage.v1.aes-hmac-sha2", // ... other properties according to algorithm } ``` From 76dd16b854b8b0d94fcced863fa0751d2fff4bcd Mon Sep 17 00:00:00 2001 From: Hubert Chathi Date: Tue, 24 Mar 2020 16:18:44 -0400 Subject: [PATCH 018/114] add proposal to switch to symmetric crypto --- proposals/1946-secure_server-side_storage.md | 6 ++ proposals/xxxx-symmetric-ssss.md | 76 ++++++++++++++++++++ 2 files changed, 82 insertions(+) create mode 100644 proposals/xxxx-symmetric-ssss.md diff --git a/proposals/1946-secure_server-side_storage.md b/proposals/1946-secure_server-side_storage.md index 61950b5f..5263e8c7 100644 --- a/proposals/1946-secure_server-side_storage.md +++ b/proposals/1946-secure_server-side_storage.md @@ -8,6 +8,12 @@ decryption key for the backups on the server, or cross-signing ([MSC1756](https://github.com/matrix-org/matrix-doc/pull/1756)) can store the signing keys. This proposal presents a standardized way of storing such data. +## Changes + +- [MSCxxxx](https://github.com/matrix-org/matrix-doc/pull/xxxx) changed the + encryption algorithm used from an asymmetric algorithm (Curve25519) to a + symmetric algorithm (AES). + ## Proposal Secrets are data that clients need to use and that are sent through or stored diff --git a/proposals/xxxx-symmetric-ssss.md b/proposals/xxxx-symmetric-ssss.md new file mode 100644 index 00000000..bde9fa95 --- /dev/null +++ b/proposals/xxxx-symmetric-ssss.md @@ -0,0 +1,76 @@ +# Symmetric SSSS + +[MSC1946 (Secure Secret Storage and +Sharing)](https://github.com/matrix-org/matrix-doc/pull/1946) proposed a way of +storing encrypted secrets on the server. In the proposal, secrets were +encrypted using a Curve25519 key, which was chosen to allow easier migration +from key backups that we created before the backup key was stored using it. + +However this does not provide any guarantees that data stored using the +proposal came from a trusted source. To remedy this, we propose to change the +encryption to use AES with a MAC to ensure that only someone who knows the key +is able to store data. + +## Proposal + +* The `m.secret_storage.v1.curve25519-aes-sha2` method proposed in MSC1946 is + removed. + +* A new method, `m.secret_storage.v1.aes-hmac-sha2`, is added. With this + method, the Secret Storage key may be any size (though 256 bits is + recommended), and data is encrypted as follows: + + 1. Given the secret storage key, generate 64 bytes by performing an HKDF with + SHA-256 as the hash, a salt of 32 bytes of 0, and with the secret name as + the info. The first 32 bytes are used as the AES key, and the next 32 bytes + are used as the MAC key + 2. Generate 16 random bytes, set bit 63 to 0 (in order to work around + differences in AES-CTR implementations), and use this as the AES + initialization vector. This becomes the `iv` property, encoded using base64. + 3. Encrypt the data using AES-CTR-256 using the AES key generated above. This + encrypted data, encoded using base64, becomes the `ciphertext` property. + 4. Pass the raw encrypted data (prior to base64 encoding) through HMAC-SHA-256 + using the MAC key generated above. The resulting MAC is base64-encoded and + becomes the `mac` property. + + (We use AES-CTR to match file encryption and key exports.) + + If the key Secret Storage key is generated from a passphrase, information + about how to generate the key is stored in the `passphrase` property of the + key's account-data in a similar manner to what was done with the original + `m.secret_storage.v1.curve25519-aes-sha2` method, except that there is an + optional `bits` parameter that defaults to 256, and indicates the number of + bits that should be generated from PBKDF2 (in other words, the size of the + key). + +* The `passthrough` property specified in the "Enconding the recovery key for + server-side storage via MSC1946" section of MSC1219 is removed. The primary + purpose of that property was to allow easy migration of pre-MSC1946 backups, + so that users could reuse the backup recovery key as the Secret Storage key + without needing to re-enter the recovery key. However, since we are now + using a symmetric encryption algorithm, the client needs to know the key that + is used to encrypt, so the purpose of the field cannot be fulfilled. + + +## Potential issues + +Users who have data stored using the old encryption algorithm will need their +data migrated. Clients that support the old algorithm but not the new +algorithm will not be able to use the migrated secrets until they are updated +with the new algorithms. This should not be a major problem because the only +clients that are known to have implemented the old algorithm are Riot +Web/Android/iOS, and they have been upgraded to implement the new algorithm. + + +## Alternatives + +Rather than switching to a symmetric encryption algorithm, we could stay with +an asymmetric encryption algorithm, and add on a method to authenticate the +data. However, it is much safer to use well-known cryptographic methods rather +than trying to invent something new. Since the main reason for using an +asymmetric scheme was to ease migration from older key backups without +requiring the user to re-enter the key, but this is no longer possible due to +the need to authenticate the data using the Secret Storage key, there is no +reason to stay with an asymmetric algorithm. It is also better to use +cryptographic methods already used in Matrix where possible, rather than +introducing something new. From 4b21e44e5f1f4dd52afa9555ba509f4755e7de37 Mon Sep 17 00:00:00 2001 From: Hubert Chathi Date: Tue, 24 Mar 2020 16:24:09 -0400 Subject: [PATCH 019/114] use the MSC number --- proposals/1946-secure_server-side_storage.md | 2 +- proposals/{xxxx-symmetric-ssss.md => 2472-symmetric-ssss.md} | 0 2 files changed, 1 insertion(+), 1 deletion(-) rename proposals/{xxxx-symmetric-ssss.md => 2472-symmetric-ssss.md} (100%) diff --git a/proposals/1946-secure_server-side_storage.md b/proposals/1946-secure_server-side_storage.md index 5263e8c7..a73fee4c 100644 --- a/proposals/1946-secure_server-side_storage.md +++ b/proposals/1946-secure_server-side_storage.md @@ -10,7 +10,7 @@ signing keys. This proposal presents a standardized way of storing such data. ## Changes -- [MSCxxxx](https://github.com/matrix-org/matrix-doc/pull/xxxx) changed the +- [MSC2472](https://github.com/matrix-org/matrix-doc/pull/2472) changed the encryption algorithm used from an asymmetric algorithm (Curve25519) to a symmetric algorithm (AES). diff --git a/proposals/xxxx-symmetric-ssss.md b/proposals/2472-symmetric-ssss.md similarity index 100% rename from proposals/xxxx-symmetric-ssss.md rename to proposals/2472-symmetric-ssss.md From eb48863c4044105b86d62997d88a086eda22e690 Mon Sep 17 00:00:00 2001 From: Patrick Cloke Date: Thu, 26 Mar 2020 11:47:32 -0400 Subject: [PATCH 020/114] Markdown formatting. Co-Authored-By: Hubert Chathi --- proposals/2454-ui-interactive-auth-for-sso.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/proposals/2454-ui-interactive-auth-for-sso.md b/proposals/2454-ui-interactive-auth-for-sso.md index d3cab51b..3532a2fc 100644 --- a/proposals/2454-ui-interactive-auth-for-sso.md +++ b/proposals/2454-ui-interactive-auth-for-sso.md @@ -52,7 +52,7 @@ limited by the chosen SSO implementation, for example: * SAML2 servers typically only support one URL per service provider, so in practice it will need to be the same as that already used for the login flow - (for synapse, it's /_matrix/saml2/authn_response) - and the server needs to + (for synapse, it's `/_matrix/saml2/authn_response`) - and the server needs to be able to figure out if it's doing SSO for a login attempt or an SSO attempt. * CAS doesn't have the same restriction. From 745f8c09df6f948f9083449b2b0848444776855e Mon Sep 17 00:00:00 2001 From: Patrick Cloke Date: Fri, 27 Mar 2020 13:59:37 -0400 Subject: [PATCH 021/114] Fix incorrect statement about the current spec's guidance. --- .../2457-password-modification-invalidating-devices.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/proposals/2457-password-modification-invalidating-devices.md b/proposals/2457-password-modification-invalidating-devices.md index 9b59156c..ccde2ac6 100644 --- a/proposals/2457-password-modification-invalidating-devices.md +++ b/proposals/2457-password-modification-invalidating-devices.md @@ -13,9 +13,9 @@ These can be summarized into two groups: 2. "I just want to change my password." The [current Matrix specification](https://matrix.org/docs/spec/client_server/r0.6.0#post-matrix-client-r0-account-password) -does not provide a way to differentiate between these use cases. It gives no -guidance into what should happen to other sessions / devices when a password is -modified and leaves it up to the implementation. +does not provide a way to differentiate between these use cases. It currently +specifies behavior that fits well into the first use-case above: that the +sessions except the current session should be revoked. It is reasonable for a client to want to specify this behavior to offer two different workflows: From 943d9386bb3d7e87e439eed10b5fd5c0057e360b Mon Sep 17 00:00:00 2001 From: Hubert Chathi Date: Mon, 30 Mar 2020 17:43:46 -0400 Subject: [PATCH 022/114] remove requirement to sign the key, since it is no longer used to trust it --- proposals/1946-secure_server-side_storage.md | 12 ++---------- proposals/2472-symmetric-ssss.md | 4 ++++ 2 files changed, 6 insertions(+), 10 deletions(-) diff --git a/proposals/1946-secure_server-side_storage.md b/proposals/1946-secure_server-side_storage.md index a73fee4c..ca4230de 100644 --- a/proposals/1946-secure_server-side_storage.md +++ b/proposals/1946-secure_server-side_storage.md @@ -38,9 +38,8 @@ Each key has an ID, and the description of the key is stored in the user's account_data using the event type `m.secret_storage.key.[key ID]`. The contents of the account data for the key will include an `algorithm` property, which indicates the encryption algorithm used, as well as a `name` property, which is -a human-readable name. The contents will be signed as signed JSON using the -user's master cross-signing key. Other properties depend on the encryption -algorithm, and are described below. +a human-readable name. Other properties depend on the encryption algorithm, +and are described below. Example: @@ -61,13 +60,6 @@ secrets that the user would expect to be available on all their clients. Unless the user specifies otherwise, clients will try to use the default key to decrypt secrets. -Clients MUST ensure that the key is trusted before using it to encrypt secrets. -One way to do that is to have the client that creates the key sign the key -description (as signed JSON) using the user's master cross-signing key. -Another way to do that is to prompt the user to enter the passphrase used to -generate the encryption key and ensure that the generated private key -corresponds to the public key. - #### Secret storage Encrypted data is stored in the user's account_data using the event type diff --git a/proposals/2472-symmetric-ssss.md b/proposals/2472-symmetric-ssss.md index bde9fa95..4827e2cd 100644 --- a/proposals/2472-symmetric-ssss.md +++ b/proposals/2472-symmetric-ssss.md @@ -51,6 +51,10 @@ is able to store data. using a symmetric encryption algorithm, the client needs to know the key that is used to encrypt, so the purpose of the field cannot be fulfilled. +* Signing the Secret Storage key with the user's master cross-signing key is no + longer required. The key is trusted on the basis of the user entering the + key/passphrase. + ## Potential issues From 4a3f9ec63ade930eaafad5b62d9ccb912893ba9a Mon Sep 17 00:00:00 2001 From: Hubert Chathi Date: Mon, 30 Mar 2020 17:44:06 -0400 Subject: [PATCH 023/114] add information to check the key --- proposals/1946-secure_server-side_storage.md | 12 +++++++++++- proposals/2472-symmetric-ssss.md | 8 ++++++++ 2 files changed, 19 insertions(+), 1 deletion(-) diff --git a/proposals/1946-secure_server-side_storage.md b/proposals/1946-secure_server-side_storage.md index ca4230de..5e24b3ae 100644 --- a/proposals/1946-secure_server-side_storage.md +++ b/proposals/1946-secure_server-side_storage.md @@ -141,13 +141,23 @@ is encrypted and MACed as follows: (We use AES-CTR to match file encryption and key exports.) +For the purposes of allowing clients to check whether a user has correctly +entered the key, clients should: + + 1. encrypt and MAC a message consisting of 32 bytes of 0 as described above, + using the empty string as the info parameter to the HKDF in step 1. + 2. store the `iv` and `mac` in the `m.secret_storage.key.[key ID]` + account-data. + For example, the `m.secret_storage.key.key_id` for a key using this algorithm could look like: ```json { "name": "m.default", - "algorithm": "m.secret_storage.v1.aes-hmac-sha2" + "algorithm": "m.secret_storage.v1.aes-hmac-sha2", + "iv": "random+data", + "mac": "mac+of+encrypted+zeros" } ``` diff --git a/proposals/2472-symmetric-ssss.md b/proposals/2472-symmetric-ssss.md index 4827e2cd..7fe9f179 100644 --- a/proposals/2472-symmetric-ssss.md +++ b/proposals/2472-symmetric-ssss.md @@ -43,6 +43,14 @@ is able to store data. bits that should be generated from PBKDF2 (in other words, the size of the key). +* For the purposes of allowing clients to check whether a user has correctly + entered the key, clients should: + + 1. encrypt and MAC a message consisting of 32 bytes of 0 as described above, + using the empty string as the info parameter to the HKDF in step 1. + 2. store the `iv` and `mac` in the `m.secret_storage.key.[key ID]` + account-data. + * The `passthrough` property specified in the "Enconding the recovery key for server-side storage via MSC1946" section of MSC1219 is removed. The primary purpose of that property was to allow easy migration of pre-MSC1946 backups, From 4cfcda57fb3676d6de6145bb9bd1f9de60645559 Mon Sep 17 00:00:00 2001 From: Patrick Cloke Date: Wed, 1 Apr 2020 10:48:05 -0400 Subject: [PATCH 024/114] Clarify that the only new item here is the new authentication type. --- proposals/2454-ui-interactive-auth-for-sso.md | 31 ++++++++++++++++--- 1 file changed, 26 insertions(+), 5 deletions(-) diff --git a/proposals/2454-ui-interactive-auth-for-sso.md b/proposals/2454-ui-interactive-auth-for-sso.md index 3532a2fc..fff96dcf 100644 --- a/proposals/2454-ui-interactive-auth-for-sso.md +++ b/proposals/2454-ui-interactive-auth-for-sso.md @@ -22,11 +22,27 @@ supported among clients. An [additional authentication type](https://matrix.org/docs/spec/client_server/r0.6.0#authentication-types) of `m.login.sso` is added to the user-interactive authentication specification. -There are no additional parameters as part of this authentication type. + +There are no additional parameters as part of this authentication type. As per +the user-interactive authentication specification the only parameter include in +the `auth` dictionary should be the session ID from the homeserver, e.g.: + +```json +{ + "auth": { + "session": "" + } +} +``` + +### Detailed fallback authentication flow: + +The following is a re-iteration of the [fallback authentication flow](https://matrix.org/docs/spec/client_server/r0.6.0#fallback), +but with details filled in for the proposed new authentication type. When choosing this authentication flow, the following should occur: -1. If the client wants to complete that authentication type, it opens a browser +1. If the client wants to complete authentication using SSO, it opens a browser window for `/_matrix/client/r0/auth/m.login.sso/fallback/web?session=<...>` with session set to the UI-Auth session id (from the "auth" dict). @@ -59,6 +75,10 @@ limited by the chosen SSO implementation, for example: ### Example flow: +A more complete example is provided below in which a user attempts to delete +a device and is pushed into the user interactive authentication process with +SSO being the only possible flow. + 0. Client submits the request, which the server says requires SSO: ``` @@ -163,7 +183,7 @@ limited by the chosen SSO implementation, for example: ## Alternatives An alternative client flow where the fallback auth ends up redirecting to a -given URI, instead of doing JavaScript postMessage foo could be considered. +given URI, instead of doing JavaScript `postMessage` foo could be considered. This is probably an orthogonal change to the fallback auth though. ## Security considerations @@ -200,8 +220,9 @@ This problem can be mitigated by clearly telling the user what is about to happe ### Reusing User Interactive Authentication sessions The security of this relies on User Interactive Authentication sessions only -being used for the same request as they were initiated for. It is not believed -that this is currently enforced. +being used for the same request as they were initiated for. This security is not +only a concern for the proposed SSO authentication type. It is not believed +that this is currently enforced in implementations. ## Unstable prefix From f6879c897bace91ccfe0cb102f490e9dc5619cde Mon Sep 17 00:00:00 2001 From: Patrick Cloke Date: Wed, 15 Apr 2020 07:39:00 -0400 Subject: [PATCH 025/114] Fix minor grammatical fixes. Co-Authored-By: Hubert Chathi --- proposals/2454-ui-interactive-auth-for-sso.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/proposals/2454-ui-interactive-auth-for-sso.md b/proposals/2454-ui-interactive-auth-for-sso.md index fff96dcf..e39facfb 100644 --- a/proposals/2454-ui-interactive-auth-for-sso.md +++ b/proposals/2454-ui-interactive-auth-for-sso.md @@ -13,7 +13,7 @@ we need to delegate that check to the SSO system. At the protocol level, this means adding support for SSO to the [user-interactive authentication API](https://matrix.org/docs/spec/client_server/r0.6.0#user-interactive-authentication-api). -In theory, once SSO is added as a possible flow for authentication any clients +In theory, once SSO is added as a possible flow for authentication, any clients that already implement the [fallback process for unknown authentication types](https://matrix.org/docs/spec/client_server/r0.6.0#fallback) will work fine without modification. It is unknown whether this is widely supported among clients. @@ -24,7 +24,7 @@ An [additional authentication type](https://matrix.org/docs/spec/client_server/r of `m.login.sso` is added to the user-interactive authentication specification. There are no additional parameters as part of this authentication type. As per -the user-interactive authentication specification the only parameter include in +the user-interactive authentication specification, the only parameter included in the `auth` dictionary should be the session ID from the homeserver, e.g.: ```json @@ -57,7 +57,7 @@ When choosing this authentication flow, the following should occur: 3. The SSO provider validates the user, and redirects the browser back to the homeserver. 4. The homeserver validates the response from the SSO provider, updates the - user-interactive auth session to show that the SSO has completed, + user-interactive auth session to show that the SSO has completed, and [serves the fallback auth completion page as specced](https://matrix.org/docs/spec/client_server/r0.6.0#fallback). 5. The client resubmits its original request, with its original session id, which now should complete. From 958e1b4a2e2ffd7dd4210818daf88385325ac0c4 Mon Sep 17 00:00:00 2001 From: Patrick Cloke Date: Mon, 4 May 2020 14:36:52 -0400 Subject: [PATCH 026/114] Make the spec changes for MSC 2457. --- api/client-server/registration.yaml | 8 ++++++-- changelogs/client_server/newsfragments/2523.feature | 1 + 2 files changed, 7 insertions(+), 2 deletions(-) create mode 100644 changelogs/client_server/newsfragments/2523.feature diff --git a/api/client-server/registration.yaml b/api/client-server/registration.yaml index 8114299e..ecc7a6fa 100644 --- a/api/client-server/registration.yaml +++ b/api/client-server/registration.yaml @@ -328,8 +328,7 @@ paths: The homeserver may change the flows available depending on whether a valid access token is provided. The homeserver SHOULD NOT revoke the - access token provided in the request, however all other access tokens - for the user should be revoked if the request succeeds. + access token provided in the request. security: - accessToken: [] operationId: changePassword @@ -343,6 +342,11 @@ paths: type: string description: The new password for the account. example: "ihatebananas" + logout_devices: + type: boolean + description: |- + Whether other access tokens should be revoked if the request succeeds. Defaults to true. + example: true auth: description: |- Additional authentication information for the user-interactive authentication API. diff --git a/changelogs/client_server/newsfragments/2523.feature b/changelogs/client_server/newsfragments/2523.feature new file mode 100644 index 00000000..6f690ea4 --- /dev/null +++ b/changelogs/client_server/newsfragments/2523.feature @@ -0,0 +1 @@ +Optionally invalidate other access tokens during password modification per `MSC 2457 `_. From 0c582ea8c71658b71e76210d30f2ece5c3ea8644 Mon Sep 17 00:00:00 2001 From: Patrick Cloke Date: Mon, 4 May 2020 16:21:19 -0400 Subject: [PATCH 027/114] Apply suggestions from code review Co-authored-by: Travis Ralston --- api/client-server/registration.yaml | 3 ++- changelogs/client_server/newsfragments/2523.feature | 2 +- 2 files changed, 3 insertions(+), 2 deletions(-) diff --git a/api/client-server/registration.yaml b/api/client-server/registration.yaml index ecc7a6fa..c3f544c1 100644 --- a/api/client-server/registration.yaml +++ b/api/client-server/registration.yaml @@ -345,7 +345,8 @@ paths: logout_devices: type: boolean description: |- - Whether other access tokens should be revoked if the request succeeds. Defaults to true. + Whether the other access tokens, and their associated devices, for the user should be + revoked if the request succeeds. Defaults to true. example: true auth: description: |- diff --git a/changelogs/client_server/newsfragments/2523.feature b/changelogs/client_server/newsfragments/2523.feature index 6f690ea4..e45d1c2f 100644 --- a/changelogs/client_server/newsfragments/2523.feature +++ b/changelogs/client_server/newsfragments/2523.feature @@ -1 +1 @@ -Optionally invalidate other access tokens during password modification per `MSC 2457 `_. +Optionally invalidate other access tokens during password modification per `MSC2457 `_. From 1e330c942307259ffdf076940ca70b7ba33da83e Mon Sep 17 00:00:00 2001 From: Patrick Cloke Date: Mon, 4 May 2020 16:23:31 -0400 Subject: [PATCH 028/114] Clarify revocation behavior. --- api/client-server/registration.yaml | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/api/client-server/registration.yaml b/api/client-server/registration.yaml index c3f544c1..50ce4a96 100644 --- a/api/client-server/registration.yaml +++ b/api/client-server/registration.yaml @@ -328,7 +328,8 @@ paths: The homeserver may change the flows available depending on whether a valid access token is provided. The homeserver SHOULD NOT revoke the - access token provided in the request. + access token provided in the request. Whether other access tokens for + the user are revoked depends on the request parameters. security: - accessToken: [] operationId: changePassword From e89521d1951976cb2a3bca9509e5e501b7fd6325 Mon Sep 17 00:00:00 2001 From: Hubert Chathi Date: Mon, 4 May 2020 22:08:22 -0400 Subject: [PATCH 029/114] some fixes (spelling, RST, and naming) --- api/client-server/keys.yaml | 2 +- specification/modules/end_to_end_encryption.rst | 16 ++++++++-------- 2 files changed, 9 insertions(+), 9 deletions(-) diff --git a/api/client-server/keys.yaml b/api/client-server/keys.yaml index 69e39def..e172ea8a 100644 --- a/api/client-server/keys.yaml +++ b/api/client-server/keys.yaml @@ -101,7 +101,7 @@ paths: responses: 200: description: - The provided keys were sucessfully uploaded. + The provided keys were successfully uploaded. schema: type: object properties: diff --git a/specification/modules/end_to_end_encryption.rst b/specification/modules/end_to_end_encryption.rst index 4b433f17..0223d9f5 100644 --- a/specification/modules/end_to_end_encryption.rst +++ b/specification/modules/end_to_end_encryption.rst @@ -565,9 +565,9 @@ The process between Alice and Bob verifying each other would be: they match or not. #. Assuming they match, Alice and Bob's devices calculate the HMAC of their own device keys and a comma-separated sorted list of of the key IDs that they wish the other user - to verify, using SHA-256 as the hash function. HMAC is defined in [RFC 2104](https://tools.ietf.org/html/rfc2104). + to verify, using SHA-256 as the hash function. HMAC is defined in `RFC 2104 `_. The key for the HMAC is different for each item and is calculated by generating - 32 bytes (256 bits) using `the key verification HKDF <#SAS-HKDF>`_. + 32 bytes (256 bits) using `the key verification HKDF <#sas-hkdf>`_. #. Alice's device sends Bob's device a ``m.key.verification.mac`` message containing the MAC of Alice's device keys and the MAC of her key IDs to be verified. Bob's device does the same for Bob's device keys and key IDs concurrently with Alice. @@ -653,14 +653,14 @@ are used in addition to those already specified: {{m_key_verification_mac_event}} -.. _`SAS-HKDF`: +.. _sas-hkdf: HKDF calculation <<<<<<<<<<<<<<<< -In all of the SAS methods, HKDF is as defined in [RFC 5869](https://tools.ietf.org/html/rfc5869) +In all of the SAS methods, HKDF is as defined in `RFC 5869 `_ and uses the previously agreed-upon hash function for the hash function. The shared -secret is supplied as the input keying material. No salt is used, and the input +secret is supplied as the input keying material. No salt is used, and the info parameter is the concatenation of: * The string ``MATRIX_KEY_VERIFICATION_SAS``. @@ -677,7 +677,7 @@ parameter is the concatenation of: For verification of each party's device keys, HKDF is as defined in RFC 5869 and uses SHA-256 as the hash function. The shared secret is supplied as the input keying -material. No salt is used, and in the input parameter is the concatenation of: +material. No salt is used, and in the info parameter is the concatenation of: * The string ``MATRIX_KEY_VERIFICATION_MAC``. * The Matrix ID of the user whose key is being MAC-ed. @@ -691,7 +691,7 @@ material. No salt is used, and in the input parameter is the concatenation of: SAS method: ``decimal`` <<<<<<<<<<<<<<<<<<<<<<< -Generate 5 bytes using `HKDF <#SAS-HKDF>`_ then take sequences of 13 bits to +Generate 5 bytes using `HKDF <#sas-hkdf>`_ then take sequences of 13 bits to convert to decimal numbers (resulting in 3 numbers between 0 and 8191 inclusive each). Add 1000 to each calculated number. @@ -708,7 +708,7 @@ such as dashes, or with the numbers on individual lines. SAS method: ``emoji`` <<<<<<<<<<<<<<<<<<<<< -Generate 6 bytes using `HKDF <#SAS-HKDF>`_ then split the first 42 bits into +Generate 6 bytes using `HKDF <#sas-hkdf>`_ then split the first 42 bits into 7 groups of 6 bits, similar to how one would base64 encode something. Convert each group of 6 bits to a number and use the following table to get the corresponding emoji: From 6e339542562dfc00c77b2abc91d18b0476f48854 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Damir=20Jeli=C4=87?= Date: Tue, 5 May 2020 12:11:51 +0200 Subject: [PATCH 030/114] client-server: Mark the event_id when putting room events as required. --- api/client-server/room_send.yaml | 2 ++ api/client-server/room_state.yaml | 2 ++ 2 files changed, 4 insertions(+) diff --git a/api/client-server/room_send.yaml b/api/client-server/room_send.yaml index 6963f76c..fc8f3339 100644 --- a/api/client-server/room_send.yaml +++ b/api/client-server/room_send.yaml @@ -85,5 +85,7 @@ paths: type: string description: |- A unique identifier for the event. + required: + - event_id tags: - Room participation diff --git a/api/client-server/room_state.yaml b/api/client-server/room_state.yaml index 62168f26..640e2009 100644 --- a/api/client-server/room_state.yaml +++ b/api/client-server/room_state.yaml @@ -92,6 +92,8 @@ paths: type: string description: |- A unique identifier for the event. + required: + - event_id 403: description: |- The sender doesn't have permission to send the event into the room. From 3054ac572fb518499dd53eb21334788d9d3def68 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Damir=20Jeli=C4=87?= Date: Tue, 5 May 2020 12:17:54 +0200 Subject: [PATCH 031/114] changelogs: Fragment for the event id when putting room events clarification. --- changelogs/client_server/newsfragments/2525.clarification | 2 ++ 1 file changed, 2 insertions(+) create mode 100644 changelogs/client_server/newsfragments/2525.clarification diff --git a/changelogs/client_server/newsfragments/2525.clarification b/changelogs/client_server/newsfragments/2525.clarification new file mode 100644 index 00000000..e93444bf --- /dev/null +++ b/changelogs/client_server/newsfragments/2525.clarification @@ -0,0 +1,2 @@ +Clarification that the event ids are required in the response when putting room +events. From b0e8fd648df7fd9119a0e472710be3e46d3c6483 Mon Sep 17 00:00:00 2001 From: Travis Ralston Date: Tue, 5 May 2020 09:21:58 -0600 Subject: [PATCH 032/114] Apply suggestions from code review --- changelogs/client_server/newsfragments/2525.clarification | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/changelogs/client_server/newsfragments/2525.clarification b/changelogs/client_server/newsfragments/2525.clarification index e93444bf..df1fc538 100644 --- a/changelogs/client_server/newsfragments/2525.clarification +++ b/changelogs/client_server/newsfragments/2525.clarification @@ -1,2 +1 @@ -Clarification that the event ids are required in the response when putting room -events. +Clarify that an ``event_id`` is returned when sending events. From 9980b83dd4bd120a055e8544240121b38cbb628f Mon Sep 17 00:00:00 2001 From: Hubert Chathi Date: Tue, 5 May 2020 15:14:24 -0400 Subject: [PATCH 033/114] add changelog --- changelogs/client_server/newsfragments/2524.clarification | 1 + 1 file changed, 1 insertion(+) create mode 100644 changelogs/client_server/newsfragments/2524.clarification diff --git a/changelogs/client_server/newsfragments/2524.clarification b/changelogs/client_server/newsfragments/2524.clarification new file mode 100644 index 00000000..cf36c3ba --- /dev/null +++ b/changelogs/client_server/newsfragments/2524.clarification @@ -0,0 +1 @@ +Fix various wording and markup issues in the end-to-end encryption section. From 6ede023b35e735f262886d8f02dc3523527443e9 Mon Sep 17 00:00:00 2001 From: Hubert Chathi Date: Tue, 5 May 2020 16:06:21 -0400 Subject: [PATCH 034/114] make the changelog match others so they get combined --- changelogs/client_server/newsfragments/2524.clarification | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/changelogs/client_server/newsfragments/2524.clarification b/changelogs/client_server/newsfragments/2524.clarification index cf36c3ba..902a9c3f 100644 --- a/changelogs/client_server/newsfragments/2524.clarification +++ b/changelogs/client_server/newsfragments/2524.clarification @@ -1 +1 @@ -Fix various wording and markup issues in the end-to-end encryption section. +Fix various spelling errors throughout the specification. From fffed943bcdfc3b016a5c03ce4fa66c35160220b Mon Sep 17 00:00:00 2001 From: Hubert Chathi Date: Tue, 5 May 2020 19:18:04 -0400 Subject: [PATCH 035/114] proposal to add endpoint to delete backups --- proposals/xxxx-add-delete-backup.md | 14 ++++++++++++++ 1 file changed, 14 insertions(+) create mode 100644 proposals/xxxx-add-delete-backup.md diff --git a/proposals/xxxx-add-delete-backup.md b/proposals/xxxx-add-delete-backup.md new file mode 100644 index 00000000..b84de399 --- /dev/null +++ b/proposals/xxxx-add-delete-backup.md @@ -0,0 +1,14 @@ +# Add ability to delete key backups + +[MSC1219](https://github.com/matrix-org/matrix-doc/issues/1219) defined a +mechanism for key backups. However, it inadvertently omitted the endpoint to +delete an entire key backup. This proposal adds the endpoint. + +## Proposal + +An endpoint is added, `DELETE /room_keys/version/{version}`, that deletes a +backup version. Both the information about the key backup, as well as all keys +associated with the backup should be deleted. Like `POST +/room_keys/version/{version}`, and unlike `GET /room_keys/version/{version}`, +`{version}` cannot be empty, to ensure that the wrong backup is not +accidentally deleted. From 0b39308a139276a3ed02297bac4e9a5f99032a3e Mon Sep 17 00:00:00 2001 From: Hubert Chathi Date: Tue, 5 May 2020 19:20:20 -0400 Subject: [PATCH 036/114] rename to use msc number --- .../{xxxx-add-delete-backup.md => 2526-add-delete-backup.md} | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) rename proposals/{xxxx-add-delete-backup.md => 2526-add-delete-backup.md} (93%) diff --git a/proposals/xxxx-add-delete-backup.md b/proposals/2526-add-delete-backup.md similarity index 93% rename from proposals/xxxx-add-delete-backup.md rename to proposals/2526-add-delete-backup.md index b84de399..e4d589e3 100644 --- a/proposals/xxxx-add-delete-backup.md +++ b/proposals/2526-add-delete-backup.md @@ -1,4 +1,4 @@ -# Add ability to delete key backups +# MSC2526: Add ability to delete key backups [MSC1219](https://github.com/matrix-org/matrix-doc/issues/1219) defined a mechanism for key backups. However, it inadvertently omitted the endpoint to From 6b8b31ce638d82fed94957bb1b6c33bc71fd92cb Mon Sep 17 00:00:00 2001 From: Gnuxie <50846879+Gnuxie@users.noreply.github.com> Date: Wed, 6 May 2020 17:10:39 +0100 Subject: [PATCH 037/114] correct use of required annotation in json-schema https://tools.ietf.org/html/draft-handrews-json-schema-validation-02#section-6.5.3 --- api/server-server/definitions/keys.yaml | 8 +++----- 1 file changed, 3 insertions(+), 5 deletions(-) diff --git a/api/server-server/definitions/keys.yaml b/api/server-server/definitions/keys.yaml index 8bc6c563..1e025a52 100644 --- a/api/server-server/definitions/keys.yaml +++ b/api/server-server/definitions/keys.yaml @@ -20,7 +20,6 @@ properties: server_name: type: string description: DNS name of the homeserver. - required: true example: "example.org" verify_keys: type: object @@ -31,7 +30,6 @@ properties: algorithm and ``abc123`` being the version in the example below). Together, this forms the Key ID. The version must have characters matching the regular expression ``[a-zA-Z0-9_]``. - required: true additionalProperties: type: object title: Verify Key @@ -44,8 +42,8 @@ properties: key: type: string description: The `Unpadded Base64`_ encoded key. - required: true example: "VGhpcyBzaG91bGQgYmUgYSByZWFsIGVkMjU1MTkgcGF5bG9hZA" + required: ["key"] old_verify_keys: type: object description: |- @@ -69,13 +67,12 @@ properties: type: integer format: int64 description: POSIX timestamp in milliseconds for when this key expired. - required: true example: 1532645052628 key: type: string description: The `Unpadded Base64`_ encoded key. - required: true example: "VGhpcyBzaG91bGQgYmUgeW91ciBvbGQga2V5J3MgZWQyNTUxOSBwYXlsb2FkLg" + required: ["expired_ts", "key"] signatures: type: object description: Digital signatures for this object signed using the ``verify_keys``. @@ -103,3 +100,4 @@ properties: publishes a key which is valid for a significant amount of time without a way for the homeserver owner to revoke it. example: 1052262000000 +required: ["server_name", "verify_keys"] From 6613cd89a668f400a82a8a9f36b961768d77090f Mon Sep 17 00:00:00 2001 From: Richard van der Hoff <1389908+richvdh@users.noreply.github.com> Date: Thu, 7 May 2020 15:42:04 +0100 Subject: [PATCH 038/114] 2454-ui-interactive-auth-for-sso.md: markup fix --- proposals/2454-ui-interactive-auth-for-sso.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/proposals/2454-ui-interactive-auth-for-sso.md b/proposals/2454-ui-interactive-auth-for-sso.md index e39facfb..18112c5f 100644 --- a/proposals/2454-ui-interactive-auth-for-sso.md +++ b/proposals/2454-ui-interactive-auth-for-sso.md @@ -209,7 +209,7 @@ provider. It's common for SSO providers to redirect straight back to the app if you've recently authenticated with them; even in the best case, the SSO provider shows an innocent message along the lines of "Confirm that you want to sign in to -". +\". After redirecting back to the homeserver, the SSO is completed and the attacker's session is validated. They are now able to make their malicious From 9065d1aa143e06a42fd36e59d923d0756ae6225d Mon Sep 17 00:00:00 2001 From: Ben Parsons Date: Thu, 7 May 2020 15:45:33 +0100 Subject: [PATCH 039/114] proposal to use existing events as captions for images --- proposals/2529-text-messages-as-captions.md | 37 +++++++++++++++++++++ 1 file changed, 37 insertions(+) create mode 100644 proposals/2529-text-messages-as-captions.md diff --git a/proposals/2529-text-messages-as-captions.md b/proposals/2529-text-messages-as-captions.md new file mode 100644 index 00000000..df76e325 --- /dev/null +++ b/proposals/2529-text-messages-as-captions.md @@ -0,0 +1,37 @@ +# Use existing m.room.message/m.text events as captions for images + +## Background + +There is a demand to be able to apply a text caption to an image, as is possible in other chat platforms. In Matrix this is not possible, so people will generally send two events: one `m.image`, then a `m.text` event immediately afterward to simulate a caption. + +Better would be to able to explicitly mark an event as a caption. + +## Proposal + +Allow an optional `m.relates_to` field in the `content` field of a text message event. + +Example: + +``` +... + "content": { + "body": "Caption text", + "msgtype": "m.text", + "m.relates_to": { + "event_id": "$(some image event)", + "rel_type": "m.caption" + } + }, +``` + +If a client recognises the `rel_type`, they can render the caption with the image rather than as a separate message in the timeline. + +The benefit of this is that if a client doesn't support or recognise the `m.caption`, it can ignore the relation and just render the message inline. + +This would not require aggregation from the server since there will always be a need to send the event separately anyway. + +## Potential issues + +* Not sure how this relates to the broader questions discussed in MSC1849 +* This is catering to a narrow use-case requirement. There may be a more general solution available +* Would MSC1767 (extensible events) obsolete this? From c704da1449eb18127a4fa4b8028f6dbfb3e1fd9d Mon Sep 17 00:00:00 2001 From: Ben Parsons Date: Thu, 7 May 2020 15:49:18 +0100 Subject: [PATCH 040/114] remove proposal --- proposals/2529-text-messages-as-captions.md | 37 --------------------- 1 file changed, 37 deletions(-) delete mode 100644 proposals/2529-text-messages-as-captions.md diff --git a/proposals/2529-text-messages-as-captions.md b/proposals/2529-text-messages-as-captions.md deleted file mode 100644 index df76e325..00000000 --- a/proposals/2529-text-messages-as-captions.md +++ /dev/null @@ -1,37 +0,0 @@ -# Use existing m.room.message/m.text events as captions for images - -## Background - -There is a demand to be able to apply a text caption to an image, as is possible in other chat platforms. In Matrix this is not possible, so people will generally send two events: one `m.image`, then a `m.text` event immediately afterward to simulate a caption. - -Better would be to able to explicitly mark an event as a caption. - -## Proposal - -Allow an optional `m.relates_to` field in the `content` field of a text message event. - -Example: - -``` -... - "content": { - "body": "Caption text", - "msgtype": "m.text", - "m.relates_to": { - "event_id": "$(some image event)", - "rel_type": "m.caption" - } - }, -``` - -If a client recognises the `rel_type`, they can render the caption with the image rather than as a separate message in the timeline. - -The benefit of this is that if a client doesn't support or recognise the `m.caption`, it can ignore the relation and just render the message inline. - -This would not require aggregation from the server since there will always be a need to send the event separately anyway. - -## Potential issues - -* Not sure how this relates to the broader questions discussed in MSC1849 -* This is catering to a narrow use-case requirement. There may be a more general solution available -* Would MSC1767 (extensible events) obsolete this? From 78f8d1322f220018d51281885ca237a724831366 Mon Sep 17 00:00:00 2001 From: Patrick Cloke Date: Fri, 8 May 2020 11:27:11 -0400 Subject: [PATCH 041/114] Add MSC2454 to the specification. --- .../client_server/newsfragments/2532.feature | 1 + specification/client_server_api.rst | 33 +++++++++++++++++++ 2 files changed, 34 insertions(+) create mode 100644 changelogs/client_server/newsfragments/2532.feature diff --git a/changelogs/client_server/newsfragments/2532.feature b/changelogs/client_server/newsfragments/2532.feature new file mode 100644 index 00000000..cf74a289 --- /dev/null +++ b/changelogs/client_server/newsfragments/2532.feature @@ -0,0 +1 @@ +Add User-Interactive Authentication for SSO-backed homeserver per `MSC2454 `_. diff --git a/specification/client_server_api.rst b/specification/client_server_api.rst index eb32d3b4..581f0c28 100644 --- a/specification/client_server_api.rst +++ b/specification/client_server_api.rst @@ -643,6 +643,7 @@ This specification defines the following auth types: - ``m.login.password`` - ``m.login.recaptcha`` - ``m.login.oauth2`` + - ``m.login.sso`` - ``m.login.email.identity`` - ``m.login.msisdn`` - ``m.login.token`` @@ -782,6 +783,38 @@ the auth code. Homeservers can choose any path for the ``redirect URI``. Once the OAuth flow has completed, the client retries the request with the session only, as above. +Single Sign-On +<<<<<<<<<<<<<< +:Type: + ``m.login.sso`` +:Description: + Authentication is supported by authorising with an external single sign-on + provider. + +A client wanting to complete authentication using SSO should use the +`Fallback`_ authentication flow by opening a browser window for +``/_matrix/client/r0/auth/m.login.sso/fallback/web?session=<...>`` with the +session parameter set to the session ID provied by the server. + +The homeserver should return a page which asks for the user's confirmation +before proceeding. For example, the page could say words to the effect of: + + A client is trying to remove a device/add an email address/take over your + account. To confirm this action, re-authenticate with single sign-on. If you + did not expect this, your account may be compromised! + +Once the user has confirmed they should be redirected to the single sign-on +provider's login page. Once the provider has validated the user, the browser is +redirected back to the homeserver. + +The homeserver then validates the response from the single sign-on provider and +updates the user-interactive authentication session to mark the single sign-on +stage has been completed. The browser is shown the fallback authentication +completion page. + +Once the flow has completed, the client retries the request with the session +only, as above. + Email-based (identity / homeserver) <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<< :Type: From e85f6c31a3f6d2f93de402347edae4500d95b383 Mon Sep 17 00:00:00 2001 From: Patrick Cloke Date: Mon, 11 May 2020 15:14:11 -0400 Subject: [PATCH 042/114] Fix a typo found in review. Co-authored-by: Andrew Morgan <1342360+anoadragon453@users.noreply.github.com> --- specification/client_server_api.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/specification/client_server_api.rst b/specification/client_server_api.rst index 581f0c28..fce879a2 100644 --- a/specification/client_server_api.rst +++ b/specification/client_server_api.rst @@ -794,7 +794,7 @@ Single Sign-On A client wanting to complete authentication using SSO should use the `Fallback`_ authentication flow by opening a browser window for ``/_matrix/client/r0/auth/m.login.sso/fallback/web?session=<...>`` with the -session parameter set to the session ID provied by the server. +session parameter set to the session ID provided by the server. The homeserver should return a page which asks for the user's confirmation before proceeding. For example, the page could say words to the effect of: From 3556b8457f5bde619c6e81c9b996c43fc86fb78d Mon Sep 17 00:00:00 2001 From: Travis Ralston Date: Mon, 11 May 2020 21:05:26 -0600 Subject: [PATCH 043/114] Add changelog for 2527 --- changelogs/server_server/newsfragments/2527.clarification | 1 + 1 file changed, 1 insertion(+) create mode 100644 changelogs/server_server/newsfragments/2527.clarification diff --git a/changelogs/server_server/newsfragments/2527.clarification b/changelogs/server_server/newsfragments/2527.clarification new file mode 100644 index 00000000..329d5da2 --- /dev/null +++ b/changelogs/server_server/newsfragments/2527.clarification @@ -0,0 +1 @@ +Clarify which fields are required on the key server endpoints. From f156938821cba90bb555af45dac28c01b460524f Mon Sep 17 00:00:00 2001 From: Travis Ralston Date: Tue, 12 May 2020 16:10:14 -0600 Subject: [PATCH 044/114] Update v6 room proposal for modern activity --- proposals/2240-rooms-v6.md | 23 ++++++++--------------- 1 file changed, 8 insertions(+), 15 deletions(-) diff --git a/proposals/2240-rooms-v6.md b/proposals/2240-rooms-v6.md index a67a1ad7..fddbdc9c 100644 --- a/proposals/2240-rooms-v6.md +++ b/proposals/2240-rooms-v6.md @@ -1,21 +1,14 @@ # MSC2240: Room Version 6 -A new room version, `6`, is proposed using [room version 5](https://matrix.org/docs/spec/rooms/v5.html) -as a starting point and incorporating the following MSCs: +A new room version, `6`, is proposed using [room version 5](https://matrix.org/docs/spec/rooms/v5.html) as a base +and incorporating the following MSCs: -* [MSC2174](https://github.com/matrix-org/matrix-doc/pull/2174) - Moving the `redacts` key. -* [MSC2175](https://github.com/matrix-org/matrix-doc/pull/2175) - Removing the `creator` from create events. -* [MSC2176](https://github.com/matrix-org/matrix-doc/pull/2176) - Modern redaction rules. * [MSC2209](https://github.com/matrix-org/matrix-doc/pull/2209) - Including notifications in power level auth rules. -* [MSC2212](https://github.com/matrix-org/matrix-doc/pull/2212) - Third party user power levels. -* [MSC2213](https://github.com/matrix-org/matrix-doc/pull/2213) - Rejoinability of invite-only rooms. -* [MSC1849](https://github.com/matrix-org/matrix-doc/pull/1849) - Event aggregations/relationships. +* [MSC2212](https://github.com/matrix-org/matrix-doc/pull/2432) - Alias event authorisation and redaction. +* PLACEHOLDER - Something to do with integers. -MSCs which require a new room version and are excluded from v6 are: -* [MSC2214](https://github.com/matrix-org/matrix-doc/pull/2214) - Joining upgraded private rooms. - This MSC is excluded due to the solution not being fully worked out. As this v6 proposal progresses, - it is possible for it to be included in the future. +Though other MSCs are capable of being included in this version, they do not have sufficient implementation to be +considered stable enough for v6 rooms. A future room version may still include them. - -Room version 6 upon being added to the specification shall be considered stable. No other room versions -are affected by this MSC. +Room version 6 upon being added to the specification shall be considered stable. No other room versions are affected +by this MSC. From 42ae5c966a1256e3ff2ec1945c45529dc92fae68 Mon Sep 17 00:00:00 2001 From: Travis Ralston Date: Tue, 12 May 2020 19:02:39 -0600 Subject: [PATCH 045/114] Clarify the limits of prev_events and auth_events Fixes https://github.com/matrix-org/matrix-doc/issues/2307 These restrictions are due to a Synapse change which never made it to the spec prior to the MSC process. This is therefore documenting an omission. Synapse's numbers are verified here: https://github.com/matrix-org/synapse/blob/207b1737ee0acd226359d59ce3b7f7d46111b1c8/synapse/handlers/federation.py#L1183-L1213 ... and Synapse's handling of the backfill endpoint is described here: https://github.com/matrix-org/synapse/blob/207b1737ee0acd226359d59ce3b7f7d46111b1c8/synapse/handlers/federation.py#L829-L837 This also includes a mention that https://github.com/matrix-org/matrix-doc/pull/1594 should render the `auth_events` limitation unnecessary. No changes are required to the the "checks on receipt of a PDU" section of the server-server spec as it starts with "must be a valid event", to which an event is invalid if it contains too many prev or auth events. --- api/server-server/backfill.yaml | 8 ++++++++ api/server-server/definitions/pdu_v3.yaml | 6 ++++++ api/server-server/definitions/pdu_v4.yaml | 6 ++++++ api/server-server/definitions/unsigned_pdu_base.yaml | 6 ++++++ 4 files changed, 26 insertions(+) diff --git a/api/server-server/backfill.yaml b/api/server-server/backfill.yaml index 2ed6298c..d53de6c0 100644 --- a/api/server-server/backfill.yaml +++ b/api/server-server/backfill.yaml @@ -63,6 +63,14 @@ paths: description: |- A transaction containing the PDUs that preceded the given event(s), including the given event(s), up to the given limit. + + .. Note:: + Though the PDU definitions require that ``prev_events`` and ``auth_events`` be limited + in number, the response of backfill MUST NOT be validated on these specific restrictions. + + Due to historical reasons, it is possible that events which were previously accepted + would now be rejected by these limitations. The events should be rejected per usual by + the ``/send``, ``/get_missing_events``, and remaining endpoints. schema: $ref: "definitions/unlimited_pdu_transaction.yaml" "/get_missing_events/{roomId}": diff --git a/api/server-server/definitions/pdu_v3.yaml b/api/server-server/definitions/pdu_v3.yaml index 38105098..3e69d941 100644 --- a/api/server-server/definitions/pdu_v3.yaml +++ b/api/server-server/definitions/pdu_v3.yaml @@ -32,6 +32,10 @@ allOf: description: |- Event IDs for the authorization events that would allow this event to be in the room. + + Must contain less than or equal to 10 events. Note that if the relevant + auth event selection rules are used, this restriction should never be + encountered. example: ["$base64EncodedHash", "$AnotherEvent"] prev_events: type: array @@ -41,6 +45,8 @@ allOf: description: |- Event IDs for the most recent events in the room that the homeserver was aware of when it made this event. + + Must contain less than or equal to 20 events. example: ["$base64EncodedHash", "$AnotherEvent"] hashes: type: object diff --git a/api/server-server/definitions/pdu_v4.yaml b/api/server-server/definitions/pdu_v4.yaml index a045e657..d22956c6 100644 --- a/api/server-server/definitions/pdu_v4.yaml +++ b/api/server-server/definitions/pdu_v4.yaml @@ -32,6 +32,10 @@ allOf: description: |- Event IDs for the authorization events that would allow this event to be in the room. + + Must contain less than or equal to 10 events. Note that if the relevant + auth event selection rules are used, this restriction should never be + encountered. example: ["$URLsafe-base64EncodedHash", "$Another_Event"] prev_events: type: array @@ -41,6 +45,8 @@ allOf: description: |- Event IDs for the most recent events in the room that the homeserver was aware of when it made this event. + + Must contain less than or equal to 20 events. example: ["$URLsafe-base64EncodedHash", "$Another_Event"] required: - auth_events diff --git a/api/server-server/definitions/unsigned_pdu_base.yaml b/api/server-server/definitions/unsigned_pdu_base.yaml index 283e4fed..f1485b55 100644 --- a/api/server-server/definitions/unsigned_pdu_base.yaml +++ b/api/server-server/definitions/unsigned_pdu_base.yaml @@ -53,6 +53,8 @@ properties: description: |- Event IDs and reference hashes for the most recent events in the room that the homeserver was aware of when it made this event. + + Must contain less than or equal to 20 events. items: type: array maxItems: 2 @@ -84,6 +86,10 @@ properties: description: |- Event IDs and reference hashes for the authorization events that would allow this event to be in the room. + + Must contain less than or equal to 10 events. Note that if the relevant + auth event selection rules are used, this restriction should never be + encountered. items: type: array maxItems: 2 From 9599c3bc99bd1e7a2914d868f7349da4e9707b6f Mon Sep 17 00:00:00 2001 From: Travis Ralston Date: Tue, 12 May 2020 19:03:53 -0600 Subject: [PATCH 046/114] changelog --- changelogs/server_server/newsfragments/2538.clarification | 1 + 1 file changed, 1 insertion(+) create mode 100644 changelogs/server_server/newsfragments/2538.clarification diff --git a/changelogs/server_server/newsfragments/2538.clarification b/changelogs/server_server/newsfragments/2538.clarification new file mode 100644 index 00000000..be6f769e --- /dev/null +++ b/changelogs/server_server/newsfragments/2538.clarification @@ -0,0 +1 @@ +Clarify the limits of ``prev_event``s and ``auth_events`` for PDUs. From 05228bf40be9eb89b225924f74b497119d1dac62 Mon Sep 17 00:00:00 2001 From: Travis Ralston Date: Tue, 12 May 2020 19:13:16 -0600 Subject: [PATCH 047/114] ``s -> s`` --- changelogs/server_server/newsfragments/2538.clarification | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/changelogs/server_server/newsfragments/2538.clarification b/changelogs/server_server/newsfragments/2538.clarification index be6f769e..4b709a8d 100644 --- a/changelogs/server_server/newsfragments/2538.clarification +++ b/changelogs/server_server/newsfragments/2538.clarification @@ -1 +1 @@ -Clarify the limits of ``prev_event``s and ``auth_events`` for PDUs. +Clarify the limits of ``prev_events`` and ``auth_events`` for PDUs. From 7797c187275c5e340b2d4ef54aa8d4ef331a5012 Mon Sep 17 00:00:00 2001 From: Patrick Cloke Date: Wed, 13 May 2020 11:22:51 -0400 Subject: [PATCH 048/114] Add MSC2540 for stricter validation of event JSON. --- proposals/2540-stricter-event-validation.md | 55 +++++++++++++++++++++ 1 file changed, 55 insertions(+) create mode 100644 proposals/2540-stricter-event-validation.md diff --git a/proposals/2540-stricter-event-validation.md b/proposals/2540-stricter-event-validation.md new file mode 100644 index 00000000..9763b4f4 --- /dev/null +++ b/proposals/2540-stricter-event-validation.md @@ -0,0 +1,55 @@ +# MSC2540: Stricter event validation: JSON compliance + +## Background + +There has been [prior discussions](https://github.com/matrix-org/matrix-doc/issues/1646) +about validating events more strictly. This MSC proposes fixing a small piece of +this: JSON compliance. + +The [Canonical JSON](https://matrix.org/docs/spec/appendices#canonical-json) +specification requires that numbers that are serialized in JSON are integers in +the range of [-2 ^ 53 + 1, 2 ^ 53 - 1], which matches the requirements of +[section 6 of RFC 7159](https://tools.ietf.org/html/rfc7159). Note that it is +not explicit, but all floats are invalid. + +It is worth mentioning that there are common extensions to JSON which produce +invalid JSON according to the Matrix specification, some programming langauges +even support these by default. One common additional feature is handling +"special" float values: `Infinity`, `-Infinity`, and `NaN`. + + +## Proposal + +In a future room version, Matrix server implementations should strictly enforce +the Canonical JSON specification for events. + +The rationale for doing this in a future room version is to avoid a split brain +room -- where some federated servers believe an event is valid and others reject +it as invalid. Rooms will be able to opt into this behavior as part of a room +version upgrade. + + +## Potential issues + +N/A + + +## Alternatives + +It could be argued that this MSC is unnecessary since it does not add any new +requirements for handling of JSON data. Unfortunately starting to enforce these +requirements in current rooms could cause federation to break as homeservers +will disagree on whether events are valid. + + +## Security considerations + +N/A + + +## Unstable prefix + +A room versions of `org.matrix.strict_canonicaljson` until a future room version +is available. This room version will use +[room version 5](https://matrix.org/docs/spec/rooms/v5) as base and include the +above modifications. From 11587a53849b4e898fd13b8051a8417619590879 Mon Sep 17 00:00:00 2001 From: Patrick Cloke Date: Wed, 13 May 2020 11:25:31 -0400 Subject: [PATCH 049/114] Small clarification. --- proposals/2540-stricter-event-validation.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/proposals/2540-stricter-event-validation.md b/proposals/2540-stricter-event-validation.md index 9763b4f4..81acd970 100644 --- a/proposals/2540-stricter-event-validation.md +++ b/proposals/2540-stricter-event-validation.md @@ -21,7 +21,7 @@ even support these by default. One common additional feature is handling ## Proposal In a future room version, Matrix server implementations should strictly enforce -the Canonical JSON specification for events. +the JSON compliance of the Canonical JSON specification for events. The rationale for doing this in a future room version is to avoid a split brain room -- where some federated servers believe an event is valid and others reject From d8e6e17bd378481e7f411f83c1fe30b001b9fb18 Mon Sep 17 00:00:00 2001 From: Travis Ralston Date: Wed, 13 May 2020 09:42:14 -0600 Subject: [PATCH 050/114] Incorporate integers MSC --- proposals/2240-rooms-v6.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/proposals/2240-rooms-v6.md b/proposals/2240-rooms-v6.md index fddbdc9c..9555f97c 100644 --- a/proposals/2240-rooms-v6.md +++ b/proposals/2240-rooms-v6.md @@ -5,7 +5,7 @@ and incorporating the following MSCs: * [MSC2209](https://github.com/matrix-org/matrix-doc/pull/2209) - Including notifications in power level auth rules. * [MSC2212](https://github.com/matrix-org/matrix-doc/pull/2432) - Alias event authorisation and redaction. -* PLACEHOLDER - Something to do with integers. +* [MSC2540](https://github.com/matrix-org/matrix-doc/pull/2540) - Integers in canonical JSON compliance. Though other MSCs are capable of being included in this version, they do not have sufficient implementation to be considered stable enough for v6 rooms. A future room version may still include them. From bbbd9c4c33a5641da7bf247d9f7967278ef0802c Mon Sep 17 00:00:00 2001 From: Patrick Cloke Date: Wed, 13 May 2020 12:50:09 -0400 Subject: [PATCH 051/114] Update wording to avoid demand-style language. Co-authored-by: Travis Ralston --- proposals/2540-stricter-event-validation.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/proposals/2540-stricter-event-validation.md b/proposals/2540-stricter-event-validation.md index 81acd970..d5d31b46 100644 --- a/proposals/2540-stricter-event-validation.md +++ b/proposals/2540-stricter-event-validation.md @@ -20,7 +20,7 @@ even support these by default. One common additional feature is handling ## Proposal -In a future room version, Matrix server implementations should strictly enforce +In a future room version, Matrix server implementations are to strictly enforce the JSON compliance of the Canonical JSON specification for events. The rationale for doing this in a future room version is to avoid a split brain From 03588cbf0dfe47b8eb6bcc0dbb53a1c39115044a Mon Sep 17 00:00:00 2001 From: Patrick Cloke Date: Wed, 13 May 2020 14:02:50 -0400 Subject: [PATCH 052/114] Be clearer about errors and what to do with current room versions. --- proposals/2540-stricter-event-validation.md | 14 +++++++++++--- 1 file changed, 11 insertions(+), 3 deletions(-) diff --git a/proposals/2540-stricter-event-validation.md b/proposals/2540-stricter-event-validation.md index d5d31b46..12996333 100644 --- a/proposals/2540-stricter-event-validation.md +++ b/proposals/2540-stricter-event-validation.md @@ -20,18 +20,26 @@ even support these by default. One common additional feature is handling ## Proposal -In a future room version, Matrix server implementations are to strictly enforce -the JSON compliance of the Canonical JSON specification for events. +In a future room version, homeserver implementations are to strictly enforce +the JSON compliance of the Canonical JSON specification for events. Events that +do not abide by these rules should be rejected with the error code `M_NOT_JSON`. The rationale for doing this in a future room version is to avoid a split brain room -- where some federated servers believe an event is valid and others reject it as invalid. Rooms will be able to opt into this behavior as part of a room version upgrade. +Homeserver implementations are not to strictly enforce this JSON compliance in +[stable room versions](https://matrix.org/docs/spec/#complete-list-of-room-versions). +The rationale is essentially the same as why a future room version is necessary, +this ensures that all federated servers treat the same events as valid. + ## Potential issues -N/A +Homeserver implementations might include JSON parsers which are stricter than +others. It may not be worthwhile or reasonable to loosen those restrictions for +stable room versions. ## Alternatives From 6757c60d35cba656972c8e979f83ba3514b17350 Mon Sep 17 00:00:00 2001 From: Patrick Cloke Date: Thu, 14 May 2020 07:22:48 -0400 Subject: [PATCH 053/114] Specify the current stable room versions. Co-authored-by: Travis Ralston --- proposals/2540-stricter-event-validation.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/proposals/2540-stricter-event-validation.md b/proposals/2540-stricter-event-validation.md index 12996333..64a64dde 100644 --- a/proposals/2540-stricter-event-validation.md +++ b/proposals/2540-stricter-event-validation.md @@ -30,7 +30,7 @@ it as invalid. Rooms will be able to opt into this behavior as part of a room version upgrade. Homeserver implementations are not to strictly enforce this JSON compliance in -[stable room versions](https://matrix.org/docs/spec/#complete-list-of-room-versions). +[room versions 1, 2, 3, 4, and 5](https://matrix.org/docs/spec/#complete-list-of-room-versions). The rationale is essentially the same as why a future room version is necessary, this ensures that all federated servers treat the same events as valid. From 560eb6df71d974f71038c67f004092e9beca3076 Mon Sep 17 00:00:00 2001 From: Travis Ralston Date: Thu, 14 May 2020 10:44:44 -0600 Subject: [PATCH 054/114] Fix MSC reference --- proposals/2240-rooms-v6.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/proposals/2240-rooms-v6.md b/proposals/2240-rooms-v6.md index 9555f97c..2b3f1163 100644 --- a/proposals/2240-rooms-v6.md +++ b/proposals/2240-rooms-v6.md @@ -4,7 +4,7 @@ A new room version, `6`, is proposed using [room version 5](https://matrix.org/d and incorporating the following MSCs: * [MSC2209](https://github.com/matrix-org/matrix-doc/pull/2209) - Including notifications in power level auth rules. -* [MSC2212](https://github.com/matrix-org/matrix-doc/pull/2432) - Alias event authorisation and redaction. +* [MSC2432](https://github.com/matrix-org/matrix-doc/pull/2432) - Alias event authorisation and redaction. * [MSC2540](https://github.com/matrix-org/matrix-doc/pull/2540) - Integers in canonical JSON compliance. Though other MSCs are capable of being included in this version, they do not have sufficient implementation to be From 0b6301ac80b95473ddaad44b644bbf7d5867ae92 Mon Sep 17 00:00:00 2001 From: Patrick Cloke Date: Fri, 15 May 2020 10:51:41 -0400 Subject: [PATCH 055/114] Fix grammar and typos from review. Co-authored-by: Richard van der Hoff <1389908+richvdh@users.noreply.github.com> --- proposals/2540-stricter-event-validation.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/proposals/2540-stricter-event-validation.md b/proposals/2540-stricter-event-validation.md index 64a64dde..94ec481e 100644 --- a/proposals/2540-stricter-event-validation.md +++ b/proposals/2540-stricter-event-validation.md @@ -13,7 +13,7 @@ the range of [-2 ^ 53 + 1, 2 ^ 53 - 1], which matches the requirements of not explicit, but all floats are invalid. It is worth mentioning that there are common extensions to JSON which produce -invalid JSON according to the Matrix specification, some programming langauges +invalid JSON according to the Matrix specification; some programming langauges even support these by default. One common additional feature is handling "special" float values: `Infinity`, `-Infinity`, and `NaN`. @@ -31,7 +31,7 @@ version upgrade. Homeserver implementations are not to strictly enforce this JSON compliance in [room versions 1, 2, 3, 4, and 5](https://matrix.org/docs/spec/#complete-list-of-room-versions). -The rationale is essentially the same as why a future room version is necessary, +The rationale is essentially the same as why a future room version is necessary: this ensures that all federated servers treat the same events as valid. @@ -57,7 +57,7 @@ N/A ## Unstable prefix -A room versions of `org.matrix.strict_canonicaljson` until a future room version +A room version of `org.matrix.strict_canonicaljson` until a future room version is available. This room version will use [room version 5](https://matrix.org/docs/spec/rooms/v5) as base and include the above modifications. From 007c8b57992137c19e2ded990e141bf0b9157eb1 Mon Sep 17 00:00:00 2001 From: Patrick Cloke Date: Fri, 15 May 2020 10:52:29 -0400 Subject: [PATCH 056/114] Clarify the range of valid values. --- proposals/2540-stricter-event-validation.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/proposals/2540-stricter-event-validation.md b/proposals/2540-stricter-event-validation.md index 94ec481e..3027be56 100644 --- a/proposals/2540-stricter-event-validation.md +++ b/proposals/2540-stricter-event-validation.md @@ -8,7 +8,7 @@ this: JSON compliance. The [Canonical JSON](https://matrix.org/docs/spec/appendices#canonical-json) specification requires that numbers that are serialized in JSON are integers in -the range of [-2 ^ 53 + 1, 2 ^ 53 - 1], which matches the requirements of +the inclusive range of `[-2 ^ 53 + 1, 2 ^ 53 - 1]`, which matches the requirements of [section 6 of RFC 7159](https://tools.ietf.org/html/rfc7159). Note that it is not explicit, but all floats are invalid. From e5fa76c8edc526bad233000019e9da8da17ce232 Mon Sep 17 00:00:00 2001 From: Patrick Cloke Date: Fri, 15 May 2020 10:53:26 -0400 Subject: [PATCH 057/114] Clarify range a bit more. --- proposals/2540-stricter-event-validation.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/proposals/2540-stricter-event-validation.md b/proposals/2540-stricter-event-validation.md index 3027be56..3d58411d 100644 --- a/proposals/2540-stricter-event-validation.md +++ b/proposals/2540-stricter-event-validation.md @@ -8,7 +8,7 @@ this: JSON compliance. The [Canonical JSON](https://matrix.org/docs/spec/appendices#canonical-json) specification requires that numbers that are serialized in JSON are integers in -the inclusive range of `[-2 ^ 53 + 1, 2 ^ 53 - 1]`, which matches the requirements of +the inclusive range of `[-(2^53) + 1, (2^53) - 1]`, which matches the requirements of [section 6 of RFC 7159](https://tools.ietf.org/html/rfc7159). Note that it is not explicit, but all floats are invalid. From bac9afa124b47ed722a74c00ef7c65d83308beb4 Mon Sep 17 00:00:00 2001 From: Hubert Chathi Date: Fri, 15 May 2020 12:14:46 -0400 Subject: [PATCH 058/114] document error codes and remove a statement that isn't true --- proposals/2526-add-delete-backup.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/proposals/2526-add-delete-backup.md b/proposals/2526-add-delete-backup.md index e4d589e3..b41fe5a3 100644 --- a/proposals/2526-add-delete-backup.md +++ b/proposals/2526-add-delete-backup.md @@ -8,7 +8,7 @@ delete an entire key backup. This proposal adds the endpoint. An endpoint is added, `DELETE /room_keys/version/{version}`, that deletes a backup version. Both the information about the key backup, as well as all keys -associated with the backup should be deleted. Like `POST -/room_keys/version/{version}`, and unlike `GET /room_keys/version/{version}`, -`{version}` cannot be empty, to ensure that the wrong backup is not -accidentally deleted. +associated with the backup should be deleted. If the specified version was +previously deleted, the endpoint succeeds, returning an HTTP code of 200. If +the specified version never existed, the endpoint returns an HTTP code of 404 +with a Matrix `errcode` of `M_NOT_FOUND`. From d24f15a3a91b19429d65dd516e8cded1c7a2b44f Mon Sep 17 00:00:00 2001 From: Travis Ralston Date: Fri, 15 May 2020 13:41:05 -0600 Subject: [PATCH 059/114] Spec soft-logout per MSC1466 MSC: https://github.com/matrix-org/matrix-doc/issues/1466 --- api/client-server/registration.yaml | 5 ++++- .../client_server/newsfragments/2545.feature | 1 + specification/client_server_api.rst | 22 +++++++++++++++++++ 3 files changed, 27 insertions(+), 1 deletion(-) create mode 100644 changelogs/client_server/newsfragments/2545.feature diff --git a/api/client-server/registration.yaml b/api/client-server/registration.yaml index 50ce4a96..a6d5d275 100644 --- a/api/client-server/registration.yaml +++ b/api/client-server/registration.yaml @@ -346,8 +346,11 @@ paths: logout_devices: type: boolean description: |- - Whether the other access tokens, and their associated devices, for the user should be + Whether the other access tokens, and their associated devices, for the user should be revoked if the request succeeds. Defaults to true. + + When ``false``, the server can still take advantage of `the soft logout method <#soft-logout>`_ + for the user's remaining devices. example: true auth: description: |- diff --git a/changelogs/client_server/newsfragments/2545.feature b/changelogs/client_server/newsfragments/2545.feature new file mode 100644 index 00000000..40c066f4 --- /dev/null +++ b/changelogs/client_server/newsfragments/2545.feature @@ -0,0 +1 @@ +Add soft-logout support per `MSC1466 `_. diff --git a/specification/client_server_api.rst b/specification/client_server_api.rst index fce879a2..3c99d305 100644 --- a/specification/client_server_api.rst +++ b/specification/client_server_api.rst @@ -123,6 +123,10 @@ The common error codes are: :``M_UNKNOWN_TOKEN``: The access token specified was not recognised. + An additional response parameter, ``soft_logout``, might be present on the response + for 401 HTTP status codes. See `the soft logout section <#soft-logout>`_ for more + information. + :``M_MISSING_TOKEN``: No access token was specified for the request. @@ -404,6 +408,24 @@ should pass the ``device_id`` in the request body. If the client sets the to that device. There is therefore at most one active access token assigned to each device at any one time. +Soft logout +~~~~~~~~~~~ + + + +When a requests fail due to a 401 status code per above, the server can +include an extra response parameter, ``soft_logout``, to indicate if the +device information has been retained by the server. This defaults to ``false``, +implying the server has deleted the device alongside the access token. + +When ``soft_logout`` is true, the client can acquire a new access token by +specifying the device ID it is already using to the login API. In most cases +a ``soft_logout: true`` response indicates that the user's session has expired +on the server-side and the user simply needs to provide their credentials again. + +If ``soft_logout`` is ``false``, the client will not be able to reuse the device +information it already has - the server has destroyed the session. + User-Interactive Authentication API ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ From bd26a41114719c59c43befbc944301d49bfa4603 Mon Sep 17 00:00:00 2001 From: Travis Ralston Date: Fri, 15 May 2020 13:42:08 -0600 Subject: [PATCH 060/114] off by one error --- .../client_server/newsfragments/{2545.feature => 2546.feature} | 0 1 file changed, 0 insertions(+), 0 deletions(-) rename changelogs/client_server/newsfragments/{2545.feature => 2546.feature} (100%) diff --git a/changelogs/client_server/newsfragments/2545.feature b/changelogs/client_server/newsfragments/2546.feature similarity index 100% rename from changelogs/client_server/newsfragments/2545.feature rename to changelogs/client_server/newsfragments/2546.feature From b4beb05dde0d255d0738a11718fc0bc042d53f21 Mon Sep 17 00:00:00 2001 From: Travis Ralston Date: Fri, 15 May 2020 13:44:21 -0600 Subject: [PATCH 061/114] Remove excess whitespace --- specification/client_server_api.rst | 2 -- 1 file changed, 2 deletions(-) diff --git a/specification/client_server_api.rst b/specification/client_server_api.rst index 3c99d305..4082fceb 100644 --- a/specification/client_server_api.rst +++ b/specification/client_server_api.rst @@ -411,8 +411,6 @@ each device at any one time. Soft logout ~~~~~~~~~~~ - - When a requests fail due to a 401 status code per above, the server can include an extra response parameter, ``soft_logout``, to indicate if the device information has been retained by the server. This defaults to ``false``, From 54102b8d36d31ade99ad1b8023849d1a54a5dcc3 Mon Sep 17 00:00:00 2001 From: Travis Ralston Date: Fri, 15 May 2020 13:50:17 -0600 Subject: [PATCH 062/114] Clarify wording per review suggestions --- specification/proposals_intro.rst | 17 ++++++++--------- 1 file changed, 8 insertions(+), 9 deletions(-) diff --git a/specification/proposals_intro.rst b/specification/proposals_intro.rst index e6cebcf4..7c8e4efd 100644 --- a/specification/proposals_intro.rst +++ b/specification/proposals_intro.rst @@ -382,17 +382,16 @@ in some cases the MSC itself will be small enough to be considered proven. Where unclear if a MSC will require an implementation proof, ask in `#matrix-spec:matrix.org `_. -Early adoption of a MSC/idea -~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +Early release of a MSC/idea +~~~~~~~~~~~~~~~~~~~~~~~~~~~ To help facilitate early releases of software dependent on a spec release, implementations are required to use the following process to ensure that the official Matrix namespace is not cluttered with development or testing data. .. Note:: - Proof of concept implementations that are solely left on a branch/unreleased and for - demonstrating that a particular MSC works do not have to follow this. Note that the - ``develop`` branch adopted by some projects is still subject to this process. + Proof of concept implementations that are solely left on a branch/unreleased for + demonstrating that a particular MSC works do not have to follow this. The process for an implementation willing to be ahead of a spec release is: @@ -404,9 +403,9 @@ The process for an implementation willing to be ahead of a spec release is: `/_matrix/client/unstable/com.example/login`. Vendor prefixes throughout Matrix always use the Java package naming convention. The MSC for the feature should identify which preferred vendor prefix is to be used by early adopters. - * Unstable endpoints **do not** inherit from stable (e.g. `/r0`) APIs. Implementations - cannot assume that a particular endpoint will exist in the unstable namespace - even if the server advertises support for the feature. + * Note that unstable namespaces do not automatically inherit endpoints from stable + namespaces: for example, the fact that ``/_matrix/client/r0/sync`` exists does + not imply that ``/_matrix/client/unstable/com.example/sync`` exists. * If the client needs to be sure the server supports the feature, an unstable feature flag that MUST be vendor prefixed is to be used. This kind of flag shows up in the ``unstable_features`` section of ``/versions`` as, for example, @@ -431,7 +430,7 @@ The process for an implementation willing to be ahead of a spec release is: working as proposed. A typical example of this is an implementation of the MSC, though the implementation does not need to be shipped anywhere and can therefore avoid the forwards/backwards compatibility concerns mentioned here. -5. FCP is gone through, and assuming nothing is flagged the MSC lands. +5. The FCP process is completed, and assuming nothing is flagged the MSC lands. 6. A spec PR is written to incorporate the changes into Matrix. 7. A spec release happens. 8. Implementations switch to using stable prefixes (e.g.: ``/r0``) if the server From a2a1694c35e830cc7271cdd2e6cb242e62f037a4 Mon Sep 17 00:00:00 2001 From: Travis Ralston Date: Fri, 15 May 2020 14:10:06 -0600 Subject: [PATCH 063/114] Spec v2 send_join and send_leave endpoints MSC: https://github.com/matrix-org/matrix-doc/pull/1802 Fixes https://github.com/matrix-org/matrix-doc/issues/2541 This also adds the v2 invite endpoint to the ACL protected list as that appears to be an omission. --- .../{joins.yaml => joins-v1.yaml} | 7 +- api/server-server/joins-v2.yaml | 176 ++++++++++++++++++ .../{leaving.yaml => leaving-v1.yaml} | 7 +- api/server-server/leaving-v2.yaml | 140 ++++++++++++++ .../server_server/newsfragments/2547.new | 1 + specification/server_server_api.rst | 11 +- 6 files changed, 338 insertions(+), 4 deletions(-) rename api/server-server/{joins.yaml => joins-v1.yaml} (98%) create mode 100644 api/server-server/joins-v2.yaml rename api/server-server/{leaving.yaml => leaving-v1.yaml} (97%) create mode 100644 api/server-server/leaving-v2.yaml create mode 100644 changelogs/server_server/newsfragments/2547.new diff --git a/api/server-server/joins.yaml b/api/server-server/joins-v1.yaml similarity index 98% rename from api/server-server/joins.yaml rename to api/server-server/joins-v1.yaml index 1b5f4632..8defda20 100644 --- a/api/server-server/joins.yaml +++ b/api/server-server/joins-v1.yaml @@ -1,4 +1,5 @@ # Copyright 2018 New Vector Ltd +# Copyright 2020 The Matrix.org Foundation C.I.C. # # Licensed under the Apache License, Version 2.0 (the "License"); # you may not use this file except in compliance with the License. @@ -165,6 +166,10 @@ paths: put: summary: Submit a signed join event to a resident server description: |- + .. Note:: + Servers should instead prefer to use the v2 ``/send_join`` + endpoint. + Submits a signed join event to the resident server for it to accept it into the room's graph. Note that events have a different format depending on the room version - check @@ -172,7 +177,7 @@ paths: **The request and response body here describes the common event fields in more detail and may be missing other required fields for a PDU.** - operationId: sendJoin + operationId: sendJoinV1 security: - signedRequest: [] parameters: diff --git a/api/server-server/joins-v2.yaml b/api/server-server/joins-v2.yaml new file mode 100644 index 00000000..585ab4b8 --- /dev/null +++ b/api/server-server/joins-v2.yaml @@ -0,0 +1,176 @@ +# Copyright 2018 New Vector Ltd +# Copyright 2020 The Matrix.org Foundation C.I.C. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +swagger: '2.0' +info: + title: "Matrix Federation Join Room API" + version: "1.0.0" +host: localhost:8448 +schemes: + - https +basePath: /_matrix/federation/v2 +consumes: + - application/json +produces: + - application/json +securityDefinitions: + $ref: definitions/security.yaml +paths: + # Note: there is no v2 of make_join (yet) + "/send_join/{roomId}/{eventId}": + put: + summary: Submit a signed join event to a resident server + description: |- + .. Note:: + This API is nearly identical to the v1 API with the + exception of the response format being fixed. + + This endpoint is preferred over the v1 API as it provides + a more standarised response format. Senders which receive + a 400, 404, or other status code which indicates this endpoint + is not available should retry using the v1 API instead. + + Submits a signed join event to the resident server for it + to accept it into the room's graph. Note that events have + a different format depending on the room version - check + the `room version specification`_ for precise event formats. + **The request and response body here describes the common + event fields in more detail and may be missing other required + fields for a PDU.** + operationId: sendJoinV2 + security: + - signedRequest: [] + parameters: + - in: path + name: roomId + type: string + description: The room ID that is about to be joined. + required: true + x-example: "!abc123:matrix.org" + - in: path + name: eventId + type: string + description: The event ID for the join event. + required: true + x-example: "$abc123:example.org" + - in: body + name: body + type: object + required: true + schema: + type: object + properties: + sender: + type: string + description: The user ID of the joining member. + example: "@someone:example.org" + origin: + type: string + description: The name of the joining homeserver. + example: "matrix.org" + origin_server_ts: + type: integer + format: int64 + description: A timestamp added by the joining homeserver. + example: 1234567890 + type: + type: string + description: The value ``m.room.member``. + example: "m.room.member" + state_key: + type: string + description: The user ID of the joining member. + example: "@someone:example.org" + content: + type: object + title: Membership Event Content + description: The content of the event. + example: {"membership": "join"} + properties: + membership: + type: string + description: The value ``join``. + example: "join" + required: ['membership'] + required: + - state_key + - sender + - origin + - origin_server_ts + - type + - content + example: { + "$ref": "examples/minimal_pdu.json", + "type": "m.room.member", + "state_key": "@someone:example.org", + "origin": "example.org", + "origin_server_ts": 1549041175876, + "sender": "@someone:example.org", + "content": { + "membership": "join" + } + } + responses: + 200: + description: |- + The full state for the room, having accepted the join event. + schema: + type: object + title: Room State + description: The state for the room. + properties: + origin: + type: string + description: The resident server's DNS name. + auth_chain: + type: array + description: |- + The auth chain. Note that events have a different format depending on + the room version - check the `room version specification`_ for precise + event formats. + items: + type: object + title: PDU + description: |- + The `PDUs <#pdus>`_ that make up the auth chain. The event format varies depending + on the room version - check the `room version specification`_ for precise event formats. + schema: + type: object + properties: [] + example: + $ref: "examples/minimal_pdu.json" + state: + type: array + description: |- + The room state. The event format varies depending on the room version - + check the `room version specification`_ for precise event formats. + items: + type: object + title: PDU + description: |- + The `PDUs <#pdus>`_ for the fully resolved state of the room. The event format varies depending + on the room version - check the `room version specification`_ for precise event formats. + schema: + type: object + properties: [] + example: + $ref: "examples/minimal_pdu.json" + required: ["auth_chain", "state", "origin"] + examples: + application/json: { + "origin": "matrix.org", + "auth_chain": [{"$ref": "examples/minimal_pdu.json"}], + "state": [{"$ref": "examples/minimal_pdu.json"}] + } diff --git a/api/server-server/leaving.yaml b/api/server-server/leaving-v1.yaml similarity index 97% rename from api/server-server/leaving.yaml rename to api/server-server/leaving-v1.yaml index e0882fe8..cd07406e 100644 --- a/api/server-server/leaving.yaml +++ b/api/server-server/leaving-v1.yaml @@ -1,4 +1,5 @@ # Copyright 2018 New Vector Ltd +# Copyright 2020 The Matrix.org Foundation C.I.C. # # Licensed under the Apache License, Version 2.0 (the "License"); # you may not use this file except in compliance with the License. @@ -142,6 +143,10 @@ paths: put: summary: Submit a signed leave event to a resident server description: |- + .. Note:: + Servers should instead prefer to use the v2 ``/send_leave`` + endpoint. + Submits a signed leave event to the resident server for it to accept it into the room's graph. Note that events have a different format depending on the room version - check @@ -149,7 +154,7 @@ paths: **The request and response body here describes the common event fields in more detail and may be missing other required fields for a PDU.** - operationId: sendLeave + operationId: sendLeaveV1 security: - signedRequest: [] parameters: diff --git a/api/server-server/leaving-v2.yaml b/api/server-server/leaving-v2.yaml new file mode 100644 index 00000000..ed16773e --- /dev/null +++ b/api/server-server/leaving-v2.yaml @@ -0,0 +1,140 @@ +# Copyright 2018 New Vector Ltd +# Copyright 2020 The Matrix.org Foundation C.I.C. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +swagger: '2.0' +info: + title: "Matrix Federation Leave Room API" + version: "1.0.0" +host: localhost:8448 +schemes: + - https +basePath: /_matrix/federation/v2 +consumes: + - application/json +produces: + - application/json +securityDefinitions: + $ref: definitions/security.yaml +paths: + # Note: there is no v2 of make_leave (yet) + "/send_leave/{roomId}/{eventId}": + put: + summary: Submit a signed leave event to a resident server + description: |- + .. Note:: + This API is nearly identical to the v1 API with the + exception of the response format being fixed. + + This endpoint is preferred over the v1 API as it provides + a more standarised response format. Senders which receive + a 400, 404, or other status code which indicates this endpoint + is not available should retry using the v1 API instead. + + Submits a signed leave event to the resident server for it + to accept it into the room's graph. Note that events have + a different format depending on the room version - check + the `room version specification`_ for precise event formats. + **The request and response body here describes the common + event fields in more detail and may be missing other required + fields for a PDU.** + operationId: sendLeaveV2 + security: + - signedRequest: [] + parameters: + - in: path + name: roomId + type: string + description: The room ID that is about to be left. + required: true + x-example: "!abc123:matrix.org" + - in: path + name: eventId + type: string + description: The event ID for the leave event. + required: true + x-example: "$abc123:example.org" + - in: body + name: body + type: object + required: true + schema: + type: object + properties: + sender: + type: string + description: The user ID of the leaving member. + example: "@someone:example.org" + origin: + type: string + description: The name of the leaving homeserver. + example: "matrix.org" + origin_server_ts: + type: integer + format: int64 + description: A timestamp added by the leaving homeserver. + example: 1234567890 + type: + type: string + description: The value ``m.room.member``. + example: "m.room.member" + state_key: + type: string + description: The user ID of the leaving member. + example: "@someone:example.org" + content: + type: object + title: Membership Event Content + description: The content of the event. + example: {"membership": "leave"} + properties: + membership: + type: string + description: The value ``leave``. + example: "leave" + required: ['membership'] + depth: + type: integer + description: This field must be present but is ignored; it may be 0. + example: 12 + required: + - state_key + - sender + - origin + - origin_server_ts + - type + - depth + - content + example: { + "$ref": "examples/minimal_pdu.json", + "type": "m.room.member", + "state_key": "@someone:example.org", + "origin": "example.org", + "origin_server_ts": 1549041175876, + "sender": "@someone:example.org", + "content": { + "membership": "leave" + } + } + responses: + 200: + description: |- + An empty response to indicate the event was accepted into the graph by + the receiving homeserver. + schema: + type: object + title: Empty Object + description: An empty object. + examples: + application/json: {} diff --git a/changelogs/server_server/newsfragments/2547.new b/changelogs/server_server/newsfragments/2547.new new file mode 100644 index 00000000..c558fb5b --- /dev/null +++ b/changelogs/server_server/newsfragments/2547.new @@ -0,0 +1 @@ +Add new v2 ``/send_join`` and ``/send_leave`` endpoints per `MSC1802 `_. diff --git a/specification/server_server_api.rst b/specification/server_server_api.rst index 655a8cfc..b0a2bc1a 100644 --- a/specification/server_server_api.rst +++ b/specification/server_server_api.rst @@ -780,7 +780,9 @@ and responds to the joining server with the full set of state for the newly-joined room. The resident server must also send the event to other servers participating in the room. -{{joins_ss_http_api}} +{{joins_v1_ss_http_api}} + +{{joins_v2_ss_http_api}} .. TODO-spec - (paul) I don't really understand why the full auth_chain events are given @@ -817,7 +819,9 @@ signs the event and replaces the ``event_id`` with it's own. This is then sent t the resident server via ``/send_leave``. The resident server will then send the event to other servers in the room. -{{leaving_ss_http_api}} +{{leaving_v1_ss_http_api}} + +{{leaving_v2_ss_http_api}} Third-party invites ------------------- @@ -1071,8 +1075,11 @@ The following endpoint prefixes MUST be protected: * ``/_matrix/federation/v1/make_join`` * ``/_matrix/federation/v1/make_leave`` * ``/_matrix/federation/v1/send_join`` +* ``/_matrix/federation/v2/send_join`` * ``/_matrix/federation/v1/send_leave`` +* ``/_matrix/federation/v2/send_leave`` * ``/_matrix/federation/v1/invite`` +* ``/_matrix/federation/v2/invite`` * ``/_matrix/federation/v1/state`` * ``/_matrix/federation/v1/state_ids`` * ``/_matrix/federation/v1/backfill`` From 19cac033058898eaf1d90b2b16651ad70ff94f3c Mon Sep 17 00:00:00 2001 From: Travis Ralston Date: Fri, 15 May 2020 22:45:18 -0600 Subject: [PATCH 064/114] Update specification/client_server_api.rst Co-authored-by: Hubert Chathi --- specification/client_server_api.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/specification/client_server_api.rst b/specification/client_server_api.rst index 4082fceb..62550d0a 100644 --- a/specification/client_server_api.rst +++ b/specification/client_server_api.rst @@ -411,7 +411,7 @@ each device at any one time. Soft logout ~~~~~~~~~~~ -When a requests fail due to a 401 status code per above, the server can +When a request fails due to a 401 status code per above, the server can include an extra response parameter, ``soft_logout``, to indicate if the device information has been retained by the server. This defaults to ``false``, implying the server has deleted the device alongside the access token. From 28d460fb62c61f963181c890dc933b8289f4a1cc Mon Sep 17 00:00:00 2001 From: Dominic Fischer Date: Sat, 16 May 2020 21:35:26 +0100 Subject: [PATCH 065/114] Fix Typo --- api/client-server/admin.yaml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/api/client-server/admin.yaml b/api/client-server/admin.yaml index 09942a10..8794a65c 100644 --- a/api/client-server/admin.yaml +++ b/api/client-server/admin.yaml @@ -82,7 +82,7 @@ paths: devices: type: object description: |- - Each key is an identitfier for one of the user's devices. + Each key is an identifier for one of the user's devices. additionalProperties: type: object title: DeviceInfo From c3582e5fc6206f25ab06e4590d2fa54835f3d661 Mon Sep 17 00:00:00 2001 From: Dominic Fischer Date: Sat, 16 May 2020 21:41:09 +0100 Subject: [PATCH 066/114] Add changelog --- changelogs/client_server/newsfragments/2553.clarification | 1 + 1 file changed, 1 insertion(+) create mode 100644 changelogs/client_server/newsfragments/2553.clarification diff --git a/changelogs/client_server/newsfragments/2553.clarification b/changelogs/client_server/newsfragments/2553.clarification new file mode 100644 index 00000000..3ccb2333 --- /dev/null +++ b/changelogs/client_server/newsfragments/2553.clarification @@ -0,0 +1 @@ +Fix various typos throughout the specification. From a7e7496cd6d709bc30d858d48e4f86773214df8e Mon Sep 17 00:00:00 2001 From: Nicolas Werner Date: Sun, 17 May 2020 22:18:00 +0200 Subject: [PATCH 067/114] Fix some numbers in the spec text. Odds were not written as an exponent. Special tag names count was not updated. Fixes #2550 Signed-off-by: Nicolas Werner --- changelogs/client_server/newsfragments/2554.clarification | 1 + specification/modules/end_to_end_encryption.rst | 2 +- specification/modules/tags.rst | 2 +- 3 files changed, 3 insertions(+), 2 deletions(-) create mode 100644 changelogs/client_server/newsfragments/2554.clarification diff --git a/changelogs/client_server/newsfragments/2554.clarification b/changelogs/client_server/newsfragments/2554.clarification new file mode 100644 index 00000000..826fafd8 --- /dev/null +++ b/changelogs/client_server/newsfragments/2554.clarification @@ -0,0 +1 @@ +Fix some numbers in the specification to match their explanation text. diff --git a/specification/modules/end_to_end_encryption.rst b/specification/modules/end_to_end_encryption.rst index 0223d9f5..7411abe4 100644 --- a/specification/modules/end_to_end_encryption.rst +++ b/specification/modules/end_to_end_encryption.rst @@ -513,7 +513,7 @@ received the other party's part. Thus an attacker essentially only has one attem attack the Diffie-Hellman exchange, and hence we can verify fewer bits while still achieving a high degree of security: if we verify n bits, then an attacker has a 1 in 2\ :sup:`n` chance of success. For example, if we verify 40 bits, then an attacker has -a 1 in 1,099,511,627,776 chance (or less than 1 in 1012 chance) of success. A failed +a 1 in 1,099,511,627,776 chance (or less than 1 in 10\ :sup:`12` chance) of success. A failed attack would result in a mismatched Short Authentication String, alerting users to the attack. diff --git a/specification/modules/tags.rst b/specification/modules/tags.rst index 9d0cd554..33948509 100644 --- a/specification/modules/tags.rst +++ b/specification/modules/tags.rst @@ -55,7 +55,7 @@ The tag namespace is defined as follows: display name directly). These non-namespaced tags are supported for historical reasons. New tags should use one of the defined namespaces above. -Two special names are listed in the specification: +Three special names are listed in the specification: The following tags are defined in the ``m.*`` namespace: * ``m.favourite``: The user's favourite rooms. These should be shown with higher precedence than other rooms. From ddeb695b9740a01140e215c90d1cb318d48f6795 Mon Sep 17 00:00:00 2001 From: Travis Ralston Date: Mon, 18 May 2020 00:19:30 -0600 Subject: [PATCH 068/114] Use 'several' in place of counting --- specification/modules/tags.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/specification/modules/tags.rst b/specification/modules/tags.rst index 33948509..a5e23770 100644 --- a/specification/modules/tags.rst +++ b/specification/modules/tags.rst @@ -55,7 +55,7 @@ The tag namespace is defined as follows: display name directly). These non-namespaced tags are supported for historical reasons. New tags should use one of the defined namespaces above. -Three special names are listed in the specification: +Several special names are listed in the specification: The following tags are defined in the ``m.*`` namespace: * ``m.favourite``: The user's favourite rooms. These should be shown with higher precedence than other rooms. From c2d5516517559ceaa4e6bfa56024392adfa7472d Mon Sep 17 00:00:00 2001 From: Travis Ralston Date: Mon, 18 May 2020 00:24:05 -0600 Subject: [PATCH 069/114] Clobber changelogs --- changelogs/client_server/newsfragments/2553.clarification | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/changelogs/client_server/newsfragments/2553.clarification b/changelogs/client_server/newsfragments/2553.clarification index 3ccb2333..902a9c3f 100644 --- a/changelogs/client_server/newsfragments/2553.clarification +++ b/changelogs/client_server/newsfragments/2553.clarification @@ -1 +1 @@ -Fix various typos throughout the specification. +Fix various spelling errors throughout the specification. From 77d83b9ce503b9874bad98180eb233789b4bcd6a Mon Sep 17 00:00:00 2001 From: Travis Ralston Date: Tue, 19 May 2020 08:05:47 -0600 Subject: [PATCH 070/114] Clarify that soft logout is about client state --- specification/client_server_api.rst | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/specification/client_server_api.rst b/specification/client_server_api.rst index 62550d0a..11a1e3cd 100644 --- a/specification/client_server_api.rst +++ b/specification/client_server_api.rst @@ -412,17 +412,17 @@ Soft logout ~~~~~~~~~~~ When a request fails due to a 401 status code per above, the server can -include an extra response parameter, ``soft_logout``, to indicate if the -device information has been retained by the server. This defaults to ``false``, -implying the server has deleted the device alongside the access token. +include an extra response parameter, ``soft_logout``, to indicate if the client's +persisted information can be retained. This defaults to ``false``, indicating +that the server has destroyed the session. Any persisted state held by the client, +such as encryption keys and device information, will not be able to be reused. When ``soft_logout`` is true, the client can acquire a new access token by specifying the device ID it is already using to the login API. In most cases a ``soft_logout: true`` response indicates that the user's session has expired on the server-side and the user simply needs to provide their credentials again. -If ``soft_logout`` is ``false``, the client will not be able to reuse the device -information it already has - the server has destroyed the session. +In either case, the client's previously known access token will no longer function. User-Interactive Authentication API ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ From d419e3b38a223ab2ad56442997a6578a5341f5ae Mon Sep 17 00:00:00 2001 From: Travis Ralston Date: Tue, 19 May 2020 08:08:31 -0600 Subject: [PATCH 071/114] Remove extraneous spaces --- api/server-server/joins-v1.yaml | 4 ++-- api/server-server/joins-v2.yaml | 2 +- api/server-server/leaving-v1.yaml | 4 ++-- api/server-server/leaving-v2.yaml | 2 +- api/server-server/transactions.yaml | 2 +- 5 files changed, 7 insertions(+), 7 deletions(-) diff --git a/api/server-server/joins-v1.yaml b/api/server-server/joins-v1.yaml index 8defda20..86dd1722 100644 --- a/api/server-server/joins-v1.yaml +++ b/api/server-server/joins-v1.yaml @@ -63,7 +63,7 @@ paths: 200: description: |- A template to be used for the rest of the `Joining Rooms`_ handshake. Note that - events have a different format depending on the room version - check the + events have a different format depending on the room version - check the `room version specification`_ for precise event formats. **The response body here describes the common event fields in more detail and may be missing other required fields for a PDU.** @@ -172,7 +172,7 @@ paths: Submits a signed join event to the resident server for it to accept it into the room's graph. Note that events have - a different format depending on the room version - check + a different format depending on the room version - check the `room version specification`_ for precise event formats. **The request and response body here describes the common event fields in more detail and may be missing other required diff --git a/api/server-server/joins-v2.yaml b/api/server-server/joins-v2.yaml index 585ab4b8..d9569684 100644 --- a/api/server-server/joins-v2.yaml +++ b/api/server-server/joins-v2.yaml @@ -44,7 +44,7 @@ paths: Submits a signed join event to the resident server for it to accept it into the room's graph. Note that events have - a different format depending on the room version - check + a different format depending on the room version - check the `room version specification`_ for precise event formats. **The request and response body here describes the common event fields in more detail and may be missing other required diff --git a/api/server-server/leaving-v1.yaml b/api/server-server/leaving-v1.yaml index cd07406e..2d7785f6 100644 --- a/api/server-server/leaving-v1.yaml +++ b/api/server-server/leaving-v1.yaml @@ -54,7 +54,7 @@ paths: 200: description: |- A template to be used to call ``/send_leave``. Note that - events have a different format depending on the room version - check the + events have a different format depending on the room version - check the `room version specification`_ for precise event formats. **The response body here describes the common event fields in more detail and may be missing other required fields for a PDU.** @@ -149,7 +149,7 @@ paths: Submits a signed leave event to the resident server for it to accept it into the room's graph. Note that events have - a different format depending on the room version - check + a different format depending on the room version - check the `room version specification`_ for precise event formats. **The request and response body here describes the common event fields in more detail and may be missing other required diff --git a/api/server-server/leaving-v2.yaml b/api/server-server/leaving-v2.yaml index ed16773e..ac3111ed 100644 --- a/api/server-server/leaving-v2.yaml +++ b/api/server-server/leaving-v2.yaml @@ -44,7 +44,7 @@ paths: Submits a signed leave event to the resident server for it to accept it into the room's graph. Note that events have - a different format depending on the room version - check + a different format depending on the room version - check the `room version specification`_ for precise event formats. **The request and response body here describes the common event fields in more detail and may be missing other required diff --git a/api/server-server/transactions.yaml b/api/server-server/transactions.yaml index 9cc8be75..b34c1902 100644 --- a/api/server-server/transactions.yaml +++ b/api/server-server/transactions.yaml @@ -38,7 +38,7 @@ paths: The sending server must wait and retry for a 200 OK response before sending a transaction with a different ``txnId`` to the receiving server. - Note that events have a different format depending on the room version - check + Note that events have a different format depending on the room version - check the `room version specification`_ for precise event formats. operationId: sendTransaction security: From 7c0046ffc75ba1e6e82e6331039e0cd8c45a037f Mon Sep 17 00:00:00 2001 From: Travis Ralston Date: Tue, 19 May 2020 08:09:19 -0600 Subject: [PATCH 072/114] describes -> describe --- api/server-server/leaving-v2.yaml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/api/server-server/leaving-v2.yaml b/api/server-server/leaving-v2.yaml index ac3111ed..3e82c414 100644 --- a/api/server-server/leaving-v2.yaml +++ b/api/server-server/leaving-v2.yaml @@ -46,7 +46,7 @@ paths: to accept it into the room's graph. Note that events have a different format depending on the room version - check the `room version specification`_ for precise event formats. - **The request and response body here describes the common + **The request and response body here describe the common event fields in more detail and may be missing other required fields for a PDU.** operationId: sendLeaveV2 From 05e83f2e7d7c09cd1518b4dc773d9baf4eeb3080 Mon Sep 17 00:00:00 2001 From: Travis Ralston Date: Tue, 19 May 2020 09:54:07 -0600 Subject: [PATCH 073/114] Fix more describes -> describe --- api/server-server/joins-v1.yaml | 2 +- api/server-server/joins-v2.yaml | 2 +- api/server-server/leaving-v1.yaml | 2 +- 3 files changed, 3 insertions(+), 3 deletions(-) diff --git a/api/server-server/joins-v1.yaml b/api/server-server/joins-v1.yaml index 86dd1722..39287264 100644 --- a/api/server-server/joins-v1.yaml +++ b/api/server-server/joins-v1.yaml @@ -174,7 +174,7 @@ paths: to accept it into the room's graph. Note that events have a different format depending on the room version - check the `room version specification`_ for precise event formats. - **The request and response body here describes the common + **The request and response body here describe the common event fields in more detail and may be missing other required fields for a PDU.** operationId: sendJoinV1 diff --git a/api/server-server/joins-v2.yaml b/api/server-server/joins-v2.yaml index d9569684..a98d20fd 100644 --- a/api/server-server/joins-v2.yaml +++ b/api/server-server/joins-v2.yaml @@ -46,7 +46,7 @@ paths: to accept it into the room's graph. Note that events have a different format depending on the room version - check the `room version specification`_ for precise event formats. - **The request and response body here describes the common + **The request and response body here describe the common event fields in more detail and may be missing other required fields for a PDU.** operationId: sendJoinV2 diff --git a/api/server-server/leaving-v1.yaml b/api/server-server/leaving-v1.yaml index 2d7785f6..c92b7920 100644 --- a/api/server-server/leaving-v1.yaml +++ b/api/server-server/leaving-v1.yaml @@ -151,7 +151,7 @@ paths: to accept it into the room's graph. Note that events have a different format depending on the room version - check the `room version specification`_ for precise event formats. - **The request and response body here describes the common + **The request and response body here describe the common event fields in more detail and may be missing other required fields for a PDU.** operationId: sendLeaveV1 From 10e4ad72ca7bd6ed36b1611e0723fcf494c34306 Mon Sep 17 00:00:00 2001 From: Travis Ralston Date: Tue, 19 May 2020 11:58:28 -0600 Subject: [PATCH 074/114] Clarify retention of information for soft logout Co-authored-by: Hubert Chathi --- specification/client_server_api.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/specification/client_server_api.rst b/specification/client_server_api.rst index 11a1e3cd..f4cc7478 100644 --- a/specification/client_server_api.rst +++ b/specification/client_server_api.rst @@ -415,7 +415,7 @@ When a request fails due to a 401 status code per above, the server can include an extra response parameter, ``soft_logout``, to indicate if the client's persisted information can be retained. This defaults to ``false``, indicating that the server has destroyed the session. Any persisted state held by the client, -such as encryption keys and device information, will not be able to be reused. +such as encryption keys and device information, must not be reused and must be discarded. When ``soft_logout`` is true, the client can acquire a new access token by specifying the device ID it is already using to the login API. In most cases From f5ebe33a9ce858ad3bde26a35bd15437e4eb3ee5 Mon Sep 17 00:00:00 2001 From: Patrick Cloke Date: Tue, 19 May 2020 14:35:11 -0400 Subject: [PATCH 075/114] Reword how bad values are handled. --- proposals/2540-stricter-event-validation.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/proposals/2540-stricter-event-validation.md b/proposals/2540-stricter-event-validation.md index 3d58411d..ac28efc4 100644 --- a/proposals/2540-stricter-event-validation.md +++ b/proposals/2540-stricter-event-validation.md @@ -22,7 +22,7 @@ even support these by default. One common additional feature is handling In a future room version, homeserver implementations are to strictly enforce the JSON compliance of the Canonical JSON specification for events. Events that -do not abide by these rules should be rejected with the error code `M_NOT_JSON`. +do not abide by these rules should be treated like any other bad JSON value. The rationale for doing this in a future room version is to avoid a split brain room -- where some federated servers believe an event is valid and others reject From 9765116c47cf1045f0499bd3e190405fee8e05e0 Mon Sep 17 00:00:00 2001 From: Kegan Dougal Date: Wed, 20 May 2020 10:11:30 +0100 Subject: [PATCH 076/114] Make backfill wording clearer --- api/server-server/backfill.yaml | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/api/server-server/backfill.yaml b/api/server-server/backfill.yaml index d53de6c0..e98c9e90 100644 --- a/api/server-server/backfill.yaml +++ b/api/server-server/backfill.yaml @@ -32,8 +32,8 @@ paths: summary: Retrieves the events which precede the given event description: |- Retrieves a sliding-window history of previous PDUs that occurred in the given room. - Starting from the PDU ID(s) given in the ``v`` argument, the PDUs that preceded it - are retrieved, up to the total number given by the ``limit``. + Starting from the PDU ID(s) given in the ``v`` argument, the PDUs given in ``v`` and + the PDUs that preceded it are retrieved, up to the total number given by the ``limit``. operationId: backfillRoom security: - signedRequest: [] From fc91946e5a4b5ea888912420ed3ab074dfc104f9 Mon Sep 17 00:00:00 2001 From: Kegsay Date: Wed, 20 May 2020 11:07:35 +0100 Subject: [PATCH 077/114] Update api/server-server/backfill.yaml Co-authored-by: Richard van der Hoff <1389908+richvdh@users.noreply.github.com> --- api/server-server/backfill.yaml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/api/server-server/backfill.yaml b/api/server-server/backfill.yaml index e98c9e90..4bf62663 100644 --- a/api/server-server/backfill.yaml +++ b/api/server-server/backfill.yaml @@ -33,7 +33,7 @@ paths: description: |- Retrieves a sliding-window history of previous PDUs that occurred in the given room. Starting from the PDU ID(s) given in the ``v`` argument, the PDUs given in ``v`` and - the PDUs that preceded it are retrieved, up to the total number given by the ``limit``. + the PDUs that preceded them are retrieved, up to the total number given by the ``limit``. operationId: backfillRoom security: - signedRequest: [] From 07716711f14378a08f0d45064c49ecfde8c9e66b Mon Sep 17 00:00:00 2001 From: Patrick Cloke Date: Wed, 20 May 2020 08:58:32 -0400 Subject: [PATCH 078/114] Give more guidance on how invalid events should be handled. Co-authored-by: Richard van der Hoff <1389908+richvdh@users.noreply.github.com> --- proposals/2540-stricter-event-validation.md | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/proposals/2540-stricter-event-validation.md b/proposals/2540-stricter-event-validation.md index ac28efc4..31c4c39b 100644 --- a/proposals/2540-stricter-event-validation.md +++ b/proposals/2540-stricter-event-validation.md @@ -21,8 +21,10 @@ even support these by default. One common additional feature is handling ## Proposal In a future room version, homeserver implementations are to strictly enforce -the JSON compliance of the Canonical JSON specification for events. Events that -do not abide by these rules should be treated like any other bad JSON value. +the JSON compliance of the Canonical JSON specification for events. +Non-compliant events should be treated like any other malformed event, +for example by rejecting the request with an HTTP 400 error with `M_BAD_JSON`, +or by discarding the event. The rationale for doing this in a future room version is to avoid a split brain room -- where some federated servers believe an event is valid and others reject From 3875d50de80463d32b777975a94a16a99f4bffd7 Mon Sep 17 00:00:00 2001 From: Travis Ralston Date: Wed, 20 May 2020 07:50:39 -0600 Subject: [PATCH 079/114] Add missing changelog --- changelogs/server_server/newsfragments/2559.clarification | 1 + 1 file changed, 1 insertion(+) create mode 100644 changelogs/server_server/newsfragments/2559.clarification diff --git a/changelogs/server_server/newsfragments/2559.clarification b/changelogs/server_server/newsfragments/2559.clarification new file mode 100644 index 00000000..cdb9cfe0 --- /dev/null +++ b/changelogs/server_server/newsfragments/2559.clarification @@ -0,0 +1 @@ +Clarify which events are targeted by backfill. From 74b627ac0b824a5fa40cf1a943a684afbf7a5516 Mon Sep 17 00:00:00 2001 From: Travis Ralston Date: Wed, 20 May 2020 17:42:10 -0600 Subject: [PATCH 080/114] Fix the response format of the `/send` endpoint Fixes https://github.com/matrix-org/matrix-doc/issues/2236 --- api/server-server/transactions.yaml | 64 ++++++++----------- .../newsfragments/2560.clarification | 1 + 2 files changed, 28 insertions(+), 37 deletions(-) create mode 100644 changelogs/server_server/newsfragments/2560.clarification diff --git a/api/server-server/transactions.yaml b/api/server-server/transactions.yaml index b34c1902..a6348e13 100644 --- a/api/server-server/transactions.yaml +++ b/api/server-server/transactions.yaml @@ -77,43 +77,33 @@ paths: The result of processing the transaction. The server is to use this response even in the event of one or more PDUs failing to be processed. schema: - type: array - minItems: 2 - maxItems: 2 - items: - - type: integer - description: The value ``200``. - example: 200 - - type: object - title: PDU Processing Results - description: The results for the processing of each PDU in the transaction. - properties: - pdus: - type: object - description: |- - The PDUs from the original transaction. The string key represents the ID of the - PDU (event) that was processed. - additionalProperties: - type: object - title: PDU Processing Result - description: Information about how the PDU was handled. - properties: - error: - type: string - description: |- - A human readable description about what went wrong in processing this PDU. - If no error is present, the PDU can be considered successfully handled. - example: "You are not allowed to send a message to this room." - required: ['pdus'] + type: object + title: PDU Processing Results + description: The results for the processing of each PDU in the transaction. + properties: + pdus: + type: object + description: |- + The PDUs from the original transaction. The string key represents the ID of the + PDU (event) that was processed. + additionalProperties: + type: object + title: PDU Processing Result + description: Information about how the PDU was handled. + properties: + error: + type: string + description: |- + A human readable description about what went wrong in processing this PDU. + If no error is present, the PDU can be considered successfully handled. + example: "You are not allowed to send a message to this room." + required: ['pdus'] examples: - application/json: [ - 200, - { - "pdus": { - "$successful_event:example.org": {}, - "$failed_event:example.org": { - "error": "You are not allowed to send a message to this room." - } + application/json: { + "pdus": { + "$successful_event:example.org": {}, + "$failed_event:example.org": { + "error": "You are not allowed to send a message to this room." } } - ] + } diff --git a/changelogs/server_server/newsfragments/2560.clarification b/changelogs/server_server/newsfragments/2560.clarification new file mode 100644 index 00000000..c60bc6f3 --- /dev/null +++ b/changelogs/server_server/newsfragments/2560.clarification @@ -0,0 +1 @@ +Fix the response format of the ``/send`` endpoint. From 04e8244dbc3ffb7d805bd6c97bf7271c9a472162 Mon Sep 17 00:00:00 2001 From: Travis Ralston Date: Wed, 20 May 2020 17:44:40 -0600 Subject: [PATCH 081/114] Clarify various bits of wording Co-authored-by: Richard van der Hoff <1389908+richvdh@users.noreply.github.com> --- specification/proposals_intro.rst | 8 +++----- 1 file changed, 3 insertions(+), 5 deletions(-) diff --git a/specification/proposals_intro.rst b/specification/proposals_intro.rst index 7c8e4efd..2e6f4389 100644 --- a/specification/proposals_intro.rst +++ b/specification/proposals_intro.rst @@ -390,17 +390,15 @@ are required to use the following process to ensure that the official Matrix nam is not cluttered with development or testing data. .. Note:: - Proof of concept implementations that are solely left on a branch/unreleased for - demonstrating that a particular MSC works do not have to follow this. - -The process for an implementation willing to be ahead of a spec release is: + Unreleased implementations (including proofs-of-concept demonstrating that a + particular MSC works) do not have to follow this process. 1. Have an idea for a feature. 2. Implement the feature using unstable endpoints, vendor prefixes, and unstable feature flags as appropriate. * When using unstable endpoints, they MUST include a vendor prefix. For example: - `/_matrix/client/unstable/com.example/login`. Vendor prefixes throughout Matrix + ``/_matrix/client/unstable/com.example/login``. Vendor prefixes throughout Matrix always use the Java package naming convention. The MSC for the feature should identify which preferred vendor prefix is to be used by early adopters. * Note that unstable namespaces do not automatically inherit endpoints from stable From 8220de58bddeb7653dd917cf2831f543698dc50a Mon Sep 17 00:00:00 2001 From: Travis Ralston Date: Wed, 20 May 2020 17:48:13 -0600 Subject: [PATCH 082/114] Drop loudly and instead encourage --- specification/proposals_intro.rst | 5 +---- 1 file changed, 1 insertion(+), 4 deletions(-) diff --git a/specification/proposals_intro.rst b/specification/proposals_intro.rst index 2e6f4389..6d8dc8a9 100644 --- a/specification/proposals_intro.rst +++ b/specification/proposals_intro.rst @@ -436,7 +436,7 @@ is not cluttered with development or testing data. specification version, but does have the feature flag, unstable prefixes should still be used. 9. A transition period of about 2 months starts immediately after the spec release, - before implementations start to loudly encourage other implementations to switch + before implementations start to encourage other implementations to switch to stable endpoints. For example, a server implementation should start asking client implementations to support the stable endpoints 2 months after the spec release, if they haven't already. The same applies in the reverse: if clients @@ -444,9 +444,6 @@ is not cluttered with development or testing data. supporting the new spec release, some noise should be raised in the general direction of the implementation. - * Please be considerate when being noisy to implementations. A gentle reminder - in their issue tracker is generally good enough. - .. Note:: MSCs MUST still describe what the stable endpoints/feature looks like with a note towards the bottom for what the unstable feature flag/prefixes are. For example, From 1388742a52876f9b13d5f05631baf5ae3959825e Mon Sep 17 00:00:00 2001 From: Travis Ralston Date: Wed, 20 May 2020 18:00:27 -0600 Subject: [PATCH 083/114] Mention that we use OpenAPI 3's response code format See comments of https://github.com/matrix-org/matrix-doc/pull/2238 --- api/openapi_extensions.md | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/api/openapi_extensions.md b/api/openapi_extensions.md index 339452ba..fced21fc 100644 --- a/api/openapi_extensions.md +++ b/api/openapi_extensions.md @@ -43,3 +43,11 @@ doesn't work, as in the following example: properties: ... ``` + +## OpenAPI 3's "2xx" format for response codes + + + +In some cases, the schema will have HTTP response code definitions like +`2xx`, `3xx`, and `4xx`. These indicate that a response code within those +ranges (`2xx` = `200` to `299`) is valid for the schema. From b11cf560b54ddc7a73c03443a05cde41f7a98302 Mon Sep 17 00:00:00 2001 From: Travis Ralston Date: Fri, 15 May 2020 17:24:27 -0600 Subject: [PATCH 084/114] Add spec for new alias handling (client-server) MSC: https://github.com/matrix-org/matrix-doc/pull/2432 This commit does not deal with areas which will be covered by the room version specifications (namely the redaction algorithm). It feels a bit overly cruel to completely obliterate all mentions of `m.room.aliases` from the spec as client/server developers may encounter the event type in the wild. To ensure that CTRL+F still works, a brief mention that they do nothing has been put in place, leaving no other references (except the redaction algorithm - see previous paragraph). --- api/client-server/directory.yaml | 82 ++++++++++++++++++- api/client-server/room_state.yaml | 24 ++++++ event-schemas/examples/m.room.aliases | 8 -- event-schemas/examples/m.room.canonical_alias | 6 +- event-schemas/schema/m.room.aliases | 24 ------ event-schemas/schema/m.room.canonical_alias | 22 +++-- specification/client_server_api.rst | 31 ++++--- specification/modules/instant_messaging.rst | 10 +-- 8 files changed, 144 insertions(+), 63 deletions(-) delete mode 100644 event-schemas/examples/m.room.aliases delete mode 100644 event-schemas/schema/m.room.aliases diff --git a/api/client-server/directory.yaml b/api/client-server/directory.yaml index 2c7c8386..67bdf236 100644 --- a/api/client-server/directory.yaml +++ b/api/client-server/directory.yaml @@ -19,7 +19,7 @@ host: localhost:8008 schemes: - https - http -basePath: /_matrix/client/%CLIENT_MAJOR_VERSION%/directory +basePath: /_matrix/client/%CLIENT_MAJOR_VERSION% consumes: - application/json produces: @@ -27,7 +27,7 @@ produces: securityDefinitions: $ref: definitions/security.yaml paths: - "/room/{roomAlias}": + "/directory/room/{roomAlias}": put: summary: Create a new mapping from room alias to room ID. operationId: setRoomAlias @@ -129,7 +129,16 @@ paths: description: |- Remove a mapping of room alias to room ID. - Servers may choose to implement additional access control checks here, for instance that room aliases can only be deleted by their creator or a server administrator. + Servers may choose to implement additional access control checks here, for instance that + room aliases can only be deleted by their creator or a server administrator. + + .. Note:: + Servers may choose to update the ``alt_aliases`` for the ``m.room.canonical_alias`` + state event in the room when an alias is removed. Servers which choose to update the + canonical alias event are recommended to, in addition to their other relevant permission + checks, delete the alias and return a successful response even if the user does not + have permission to update the ``m.room.canonical_alias`` event. + operationId: deleteRoomAlias security: - accessToken: [] @@ -159,3 +168,70 @@ paths: "$ref": "definitions/errors/error.yaml" tags: - Room directory + "/rooms/{roomId}/aliases": + get: + summary: Get a list of local aliases on a given room. + description: |- + Get a list of aliases maintained by the local server for the + given room. + + This endpoint can be called by users who are in the room (external + users receive an ``M_FORBIDDEN`` error response). If the room's + ``m.room.history_visibility`` maps to ``world_readable``, any + user can call this endpoint. + + Servers may choose to implement additional access control checks here, + such as allowing server administrators to view aliases regardless of + membership. + + .. Note:: + Clients are not recommended to display this list of aliases prominently + as they are not curated, unlike those listed in the ``m.room.canonical_alias`` + state event. + + operationId: getLocalAliases + security: + - accessToken: [] + parameters: + - in: path + type: string + name: roomId + description: The room ID to find local aliases of. + required: true + x-example: "!abc123:example.org" + responses: + 200: + description: |- + The list of local aliases for the room. + examples: + application/json: { + "aliases": [ + "#somewhere:example.com", + "#another:example.com", + "#hat_trick:example.com" + ] + } + schema: + type: object + properties: + aliases: + type: array + description: The server's local aliases on the room. Can be empty. + items: + type: string + required: ['aliases'] + 403: + description: The user is not permitted to retrieve the list of local aliases for the room. + examples: + application/json: { + "errcode": "M_FORBIDDEN", + "error": "You are not a member of the room." + } + schema: + "$ref": "definitions/errors/error.yaml" + 429: + description: This request was rate-limited. + schema: + "$ref": "definitions/errors/rate_limited.yaml" + tags: + - Room directory diff --git a/api/client-server/room_state.yaml b/api/client-server/room_state.yaml index 640e2009..20b9c1fd 100644 --- a/api/client-server/room_state.yaml +++ b/api/client-server/room_state.yaml @@ -45,6 +45,12 @@ paths: The body of the request should be the content object of the event; the fields in this object will vary depending on the type of event. See `Room Events`_ for the ``m.`` event specification. + + If the event type being sent is ``m.room.canonical_alias`` servers + SHOULD ensure that any new aliases being listed in the event are valid + per their grammar/syntax and that they point to the room ID where the + state event is to be sent. Servers do not validate aliases which are + being removed or are already present in the state event. operationId: setRoomStateWithKey security: - accessToken: [] @@ -104,5 +110,23 @@ paths: "errcode": "M_FORBIDDEN", "error": "You do not have permission to send the event." } + 400: + description: |- + The sender's request is malformed. + + Some example error codes include: + + * ``M_INVALID_PARAMETER``: One or more aliases within the ``m.room.canonical_alias`` + event have invalid syntax. + + * ``M_BAD_ALIAS``: One or more aliases within the ``m.room.canonical_alias`` event + do not point to the room ID for which the state event is to be sent to. + schema: + $ref: "definitions/errors/error.yaml" + examples: + application/json: { + "errcode": "M_BAD_ALIAS", + "error": "The alias '#hello:example.org' does not point to this room." + } tags: - Room participation diff --git a/event-schemas/examples/m.room.aliases b/event-schemas/examples/m.room.aliases deleted file mode 100644 index 3e20332f..00000000 --- a/event-schemas/examples/m.room.aliases +++ /dev/null @@ -1,8 +0,0 @@ -{ - "$ref": "core/state_event.json", - "state_key": "example.org", - "type": "m.room.aliases", - "content": { - "aliases": ["#somewhere:example.org", "#another:example.org"] - } -} diff --git a/event-schemas/examples/m.room.canonical_alias b/event-schemas/examples/m.room.canonical_alias index 06c3226c..698c11b8 100644 --- a/event-schemas/examples/m.room.canonical_alias +++ b/event-schemas/examples/m.room.canonical_alias @@ -3,6 +3,10 @@ "type": "m.room.canonical_alias", "state_key": "", "content": { - "alias": "#somewhere:localhost" + "alias": "#somewhere:localhost", + "alt_aliases": [ + "#somewhere:example.org", + "#myroom:example.com" + ] } } diff --git a/event-schemas/schema/m.room.aliases b/event-schemas/schema/m.room.aliases deleted file mode 100644 index 348d490d..00000000 --- a/event-schemas/schema/m.room.aliases +++ /dev/null @@ -1,24 +0,0 @@ ---- -allOf: - - $ref: core-event-schema/state_event.yaml -description: 'This event is sent by a homeserver directly to inform of changes to the list of aliases it knows about for that room. The ``state_key`` for this event is set to the homeserver which owns the room alias. The entire set of known aliases for the room is the union of all the ``m.room.aliases`` events, one for each homeserver. Clients **should** check the validity of any room alias given in this list before presenting it to the user as trusted fact. The lists given by this event should be considered simply as advice on which aliases might exist, for which the client can perform the lookup to confirm whether it receives the correct room ID.' -properties: - content: - properties: - aliases: - description: A list of room aliases. - items: - type: string - type: array - required: - - aliases - type: object - state_key: - description: The homeserver domain which owns these room aliases. - type: string - type: - enum: - - m.room.aliases - type: string -title: Informs the room about what room aliases it has been given. -type: object diff --git a/event-schemas/schema/m.room.canonical_alias b/event-schemas/schema/m.room.canonical_alias index 5e8e0524..f227871a 100644 --- a/event-schemas/schema/m.room.canonical_alias +++ b/event-schemas/schema/m.room.canonical_alias @@ -3,21 +3,25 @@ allOf: - $ref: core-event-schema/state_event.yaml description: |- This event is used to inform the room about which alias should be - considered the canonical one. This could be for display purposes or as - suggestion to users which alias to use to advertise the room. - - A room with an ``m.room.canonical_alias`` event with an absent, null, or - empty ``alias`` field should be treated the same as a room with no - ``m.room.canonical_alias`` event. + considered the canonical one, and which other aliases point to the room. + This could be for display purposes or as suggestion to users which alias + to use to advertise and access the room. properties: content: properties: alias: - description: The canonical alias. + description: | + The canonical alias for the room. If not present, null, or empty the + room should be considered to have no canonical alias. type: string + alt_aliases: + description: | + Alternative aliases the room advertises. This list can have aliases + despite the ``alias`` field being null, empty, or otherwise not present. + type: array + items: + type: string type: object - required: - - alias state_key: description: A zero-length string. pattern: '^$' diff --git a/specification/client_server_api.rst b/specification/client_server_api.rst index fce879a2..fda2c106 100644 --- a/specification/client_server_api.rst +++ b/specification/client_server_api.rst @@ -1594,8 +1594,6 @@ Room Events This specification outlines several standard event types, all of which are prefixed with ``m.`` -{{m_room_aliases_event}} - {{m_room_canonical_alias_event}} {{m_room_create_event}} @@ -1608,6 +1606,15 @@ prefixed with ``m.`` {{m_room_redaction_event}} +Historical events ++++++++++++++++++ + +Some events within the ``m.`` namespace might appear in rooms, however they +serve no significant meaning in this version of the specification. They are: + +* ``m.room.aliases`` + +Previous versions of the specification have more information on these events. Syncing ~~~~~~~ @@ -1813,6 +1820,8 @@ one of the following event types: - ``m.room.aliases`` allows key ``aliases``. - ``m.room.history_visibility`` allows key ``history_visibility``. +.. TODO: Room v6 changes how redaction works in MSC2432 (no special meaning for aliases) + The server should add the event causing the redaction to the ``unsigned`` property of the redacted event, under the ``redacted_because`` key. When a client receives a redaction event it should change the redacted event in the @@ -1871,15 +1880,15 @@ send update requests to other servers. However, homeservers MUST handle ``GET`` requests to resolve aliases on other servers; they should do this using the federation API if necessary. -Rooms store a *partial* list of room aliases via the ``m.room.aliases`` state -event. This alias list is partial because it cannot guarantee that the alias -list is in any way accurate or up-to-date, as room aliases can point to -different room IDs over time. Crucially, the aliases in this event are -**purely informational** and SHOULD NOT be treated as accurate. They SHOULD -be checked before they are used or shared with another user. If a room -appears to have a room alias of ``#alias:example.com``, this SHOULD be checked -to make sure that the room's ID matches the ``room_id`` returned from the -request. +Rooms do not store a list of all aliases present on a room, though members +of the room with relevant permissions may publish preferred aliases through +the ``m.room.canonical_alias`` state event. The aliases in the state event +should point to the room ID they are published within, however room aliases +can and do drift to other room IDs over time. Clients SHOULD NOT treat the +aliases as accurate. They SHOULD be checked before they are used or shared +with another user. If a room appears to have a room alias of ``#alias:example.com``, +this SHOULD be checked to make sure that the room's ID matches the ``room_id`` +returned from the request. {{directory_cs_http_api}} diff --git a/specification/modules/instant_messaging.rst b/specification/modules/instant_messaging.rst index 679eabdc..705fd2d9 100644 --- a/specification/modules/instant_messaging.rst +++ b/specification/modules/instant_messaging.rst @@ -275,13 +275,9 @@ choose a name: 1. If the room has an `m.room.name`_ state event with a non-empty ``name`` field, use the name given by that field. -#. If the room has an `m.room.canonical_alias`_ state event with a non-empty - ``alias`` field, use the alias given by that field as the name. - -#. If neither of the above conditions are met, the client can optionally guess - an alias from the ``m.room.alias`` events in the room. This is a temporary - measure while clients do not promote canonical aliases as prominently. This - step may be removed in a future version of the specification. +#. If the room has an `m.room.canonical_alias`_ state event with a valid + ``alias`` field, use the alias given by that field as the name. Note that + clients should avoid using ``alt_aliases`` when calculating the room name. #. If none of the above conditions are met, a name should be composed based on the members of the room. Clients should consider `m.room.member`_ events From 2b6086d87c269f4d246b2a9dc22fd7de1f5e792e Mon Sep 17 00:00:00 2001 From: Travis Ralston Date: Wed, 20 May 2020 20:15:09 -0600 Subject: [PATCH 085/114] Add changelog entries It's a bit awkward that this shows up as a backwards compatible change, but it's also not breaking. --- changelogs/client_server/newsfragments/2562.feature | 1 + changelogs/client_server/newsfragments/2562.new | 1 + 2 files changed, 2 insertions(+) create mode 100644 changelogs/client_server/newsfragments/2562.feature create mode 100644 changelogs/client_server/newsfragments/2562.new diff --git a/changelogs/client_server/newsfragments/2562.feature b/changelogs/client_server/newsfragments/2562.feature new file mode 100644 index 00000000..556018da --- /dev/null +++ b/changelogs/client_server/newsfragments/2562.feature @@ -0,0 +1 @@ +Replaced legacy room alias handling with a more sustainable solution per `MSC2432 `_. diff --git a/changelogs/client_server/newsfragments/2562.new b/changelogs/client_server/newsfragments/2562.new new file mode 100644 index 00000000..c7b3e4ca --- /dev/null +++ b/changelogs/client_server/newsfragments/2562.new @@ -0,0 +1 @@ +Added ``/rooms/{roomId}/aliases`` for retrieving local aliases for a room. From 0300a3cab4a77a5757c9e6acd065261f4eb03954 Mon Sep 17 00:00:00 2001 From: Travis Ralston Date: Wed, 20 May 2020 20:45:19 -0600 Subject: [PATCH 086/114] Move redaction algorithm into room version specification We stick it in a client section of v1 as the earliest version to define the algorithm is v1, and the client-server spec tells clients to use this algorithm. --- specification/client_server_api.rst | 34 +---------------- specification/rooms/v1.rst | 59 +++++++++++++++++++++++++---- 2 files changed, 53 insertions(+), 40 deletions(-) diff --git a/specification/client_server_api.rst b/specification/client_server_api.rst index fce879a2..8a94b711 100644 --- a/specification/client_server_api.rst +++ b/specification/client_server_api.rst @@ -1779,39 +1779,7 @@ redacted include a ``redacted_because`` key whose value is the event that caused it to be redacted, which may include a reason. -Upon receipt of a redaction event, the server should strip off any keys not in -the following list: - -- ``event_id`` -- ``type`` -- ``room_id`` -- ``sender`` -- ``state_key`` -- ``content`` -- ``hashes`` -- ``signatures`` -- ``depth`` -- ``prev_events`` -- ``prev_state`` -- ``auth_events`` -- ``origin`` -- ``origin_server_ts`` -- ``membership`` - -.. Note: - Some of the keys, such as ``hashes``, will appear on the federation-formatted - event and therefore the client may not be aware of them. - -The content object should also be stripped of all keys, unless it is one of -one of the following event types: - -- ``m.room.member`` allows key ``membership``. -- ``m.room.create`` allows key ``creator``. -- ``m.room.join_rules`` allows key ``join_rule``. -- ``m.room.power_levels`` allows keys ``ban``, ``events``, ``events_default``, - ``kick``, ``redact``, ``state_default``, ``users``, ``users_default``. -- ``m.room.aliases`` allows key ``aliases``. -- ``m.room.history_visibility`` allows key ``history_visibility``. +The exact algorithm to apply against an event is defined in the `room version specification`_. The server should add the event causing the redaction to the ``unsigned`` property of the redacted event, under the ``redacted_because`` key. When a diff --git a/specification/rooms/v1.rst b/specification/rooms/v1.rst index 10eee54d..23d8a833 100644 --- a/specification/rooms/v1.rst +++ b/specification/rooms/v1.rst @@ -1,4 +1,5 @@ .. Copyright 2017,2019 New Vector Ltd +.. Copyright 2020 The Matrix.org Foundation C.I.C. .. .. Licensed under the Apache License, Version 2.0 (the "License"); .. you may not use this file except in compliance with the License. @@ -21,13 +22,57 @@ blocks for other room versions. .. contents:: Table of Contents .. sectnum:: +Client considerations +--------------------- + +Clients may need to consider some algorithms performed by the server for their own +implementation. + +Redactions +~~~~~~~~~~ + +Upon receipt of a redaction event, the server should strip off any keys not in +the following list: + +- ``event_id`` +- ``type`` +- ``room_id`` +- ``sender`` +- ``state_key`` +- ``content`` +- ``hashes`` +- ``signatures`` +- ``depth`` +- ``prev_events`` +- ``prev_state`` +- ``auth_events`` +- ``origin`` +- ``origin_server_ts`` +- ``membership`` + +.. Note: + Some of the keys, such as ``hashes``, will appear on the federation-formatted + event and therefore the client may not be aware of them. + +The content object should also be stripped of all keys, unless it is one of +one of the following event types: + +- ``m.room.member`` allows key ``membership``. +- ``m.room.create`` allows key ``creator``. +- ``m.room.join_rules`` allows key ``join_rule``. +- ``m.room.power_levels`` allows keys ``ban``, ``events``, ``events_default``, + ``kick``, ``redact``, ``state_default``, ``users``, ``users_default``. +- ``m.room.aliases`` allows key ``aliases``. +- ``m.room.history_visibility`` allows key ``history_visibility``. + Server implementation components -------------------------------- .. WARNING:: The information contained in this section is strictly for server implementors. Applications which use the Client-Server API are generally unaffected by the - details contained here, and can safely ignore their presence. + intricacies contained here. The section above regarding client considerations + is the resource that Client-Server API use cases should reference. The algorithms defined here should only apply to version 1 rooms. Other algorithms @@ -112,7 +157,7 @@ The types of state events that affect authorization are: .. NOTE:: Power levels are inferred from defaults when not explicitly supplied. - For example, mentions of the ``sender``'s power level can also refer + For example, mentions of the ``sender``'s power level can also refer to the default power level for users in the room. The rules are as follows: @@ -250,7 +295,7 @@ The rules are as follows: #. If there is no previous ``m.room.power_levels`` event in the room, allow. #. For the keys ``users_default``, ``events_default``, - ``state_default``, ``ban``, ``redact``, ``kick``, ``invite`` check if they + ``state_default``, ``ban``, ``redact``, ``kick``, ``invite`` check if they were added, changed or removed. For each found alteration: i. If the current value is higher than the ``sender``'s current power level, @@ -258,13 +303,13 @@ The rules are as follows: #. If the new value is higher than the ``sender``'s current power level, reject. - - #. For each entry being added, changed or removed in both the ``events`` and + + #. For each entry being added, changed or removed in both the ``events`` and ``users`` keys: - + i. If the current value is higher than the ``sender``'s current power level, reject. - + #. If the new value is higher than the ``sender``'s current power level, reject. From be353115594b3572fc07c8f8b5d40e57b6c161fb Mon Sep 17 00:00:00 2001 From: Travis Ralston Date: Wed, 20 May 2020 20:28:27 -0600 Subject: [PATCH 087/114] s/should/must for redaction algorithm This feels like it was a mistake some time ago considering the redaction algorithm is used in very strict algorithms like event signing. --- specification/rooms/v1.rst | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/specification/rooms/v1.rst b/specification/rooms/v1.rst index 23d8a833..9282c1f3 100644 --- a/specification/rooms/v1.rst +++ b/specification/rooms/v1.rst @@ -31,7 +31,7 @@ implementation. Redactions ~~~~~~~~~~ -Upon receipt of a redaction event, the server should strip off any keys not in +Upon receipt of a redaction event, the server must strip off any keys not in the following list: - ``event_id`` @@ -54,7 +54,7 @@ the following list: Some of the keys, such as ``hashes``, will appear on the federation-formatted event and therefore the client may not be aware of them. -The content object should also be stripped of all keys, unless it is one of +The content object must also be stripped of all keys, unless it is one of one of the following event types: - ``m.room.member`` allows key ``membership``. From a1324aa9dca7e9ca01acde43cc5db3c7f95ad718 Mon Sep 17 00:00:00 2001 From: Travis Ralston Date: Wed, 20 May 2020 20:44:08 -0600 Subject: [PATCH 088/114] Move MSC2432 (alias handling) to v6 --- specification/index.rst | 1 + specification/rooms/v6.rst | 59 ++++++++++++++++++++++++++++++++++++++ specification/targets.yaml | 4 +++ 3 files changed, 64 insertions(+) create mode 100644 specification/rooms/v6.rst diff --git a/specification/index.rst b/specification/index.rst index 728383ab..aa8a4641 100644 --- a/specification/index.rst +++ b/specification/index.rst @@ -550,6 +550,7 @@ The available room versions are: * `Version 3 `_ - **Stable**. Introduces events whose IDs are the event's hash. * `Version 4 `_ - **Stable**. Builds on v3 by using URL-safe base64 for event IDs. * `Version 5 `_ - **Stable**. Introduces enforcement of signing key validity periods. +* `Version 6 `_ - **Stable**. Alters several authorization rules for events. Specification Versions ---------------------- diff --git a/specification/rooms/v6.rst b/specification/rooms/v6.rst new file mode 100644 index 00000000..de0ab889 --- /dev/null +++ b/specification/rooms/v6.rst @@ -0,0 +1,59 @@ +.. Copyright 2020 The Matrix.org Foundation C.I.C. +.. +.. Licensed under the Apache License, Version 2.0 (the "License"); +.. you may not use this file except in compliance with the License. +.. You may obtain a copy of the License at +.. +.. http://www.apache.org/licenses/LICENSE-2.0 +.. +.. Unless required by applicable law or agreed to in writing, software +.. distributed under the License is distributed on an "AS IS" BASIS, +.. WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +.. See the License for the specific language governing permissions and +.. limitations under the License. + +Room Version 6 +============== + +This room version builds on `version 5 `_ while changing various +authorization rules performed on events. + +.. contents:: Table of Contents +.. sectnum:: + + +Client considerations +--------------------- + +The redaction algorithm has changed from `room version 1 `_ to remove +all rules against events of type ``m.room.aliases``. Room versions 2, 3, 4, and +5 all use v1's redaction algorithm. The algorithm is otherwise unchanged. + + +Server implementation components +-------------------------------- + +.. WARNING:: + The information contained in this section is strictly for server implementors. + Applications which use the Client-Server API are generally unaffected by the + intricacies contained here. The section above regarding client considerations + is the resource that Client-Server API use cases should reference. + + +Room version 6 makes the following alterations to algorithms described in `room version 5 `_. + +Redactions +~~~~~~~~~~ + +As mentioned in the client considerations portion of this specification, all +special meaning has been removed for events of type ``m.room.aliases``. The +algorithm is otherwise unchanged. + +Authorization rules for events +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Like redactions, all rules relating specifically to events of type ``m.room.aliases`` +are removed. They must still pass authorization checks relating to state events. + +The remaining rules are the same as in `room version 3 `_ +(the last inherited room version to specify the authorization rules). diff --git a/specification/targets.yaml b/specification/targets.yaml index 4e0b068d..5f0e32cc 100644 --- a/specification/targets.yaml +++ b/specification/targets.yaml @@ -45,6 +45,10 @@ targets: files: - rooms/v5.rst version_label: v5 + rooms@v6: # this is translated to be rooms/v6.html + files: + - rooms/v6.rst + version_label: v6 appendices: files: - appendices.rst From 74c51b05a434ddcfb3b8dc63ebfeda56c29456ba Mon Sep 17 00:00:00 2001 From: Travis Ralston Date: Wed, 20 May 2020 22:13:55 -0600 Subject: [PATCH 089/114] Incorporate MSC2209 (`notifications` auth rules) MSC: https://github.com/matrix-org/matrix-doc/pull/2209 The changes are slightly difficult to word without dumping the text in and playing a game of spot the difference, so we now use our pre-existing pygments support to render a representation of the difference. The difference is shown in markdown-like format instead of RST for ease of understanding. It's also not rendered HTML for largely complexity reasons. --- scripts/css/pygments.css | 83 ++++++++++++++++++++++++++++++++++++++ scripts/gendoc.py | 1 + specification/rooms/v6.rst | 32 +++++++++++++++ 3 files changed, 116 insertions(+) create mode 100644 scripts/css/pygments.css diff --git a/scripts/css/pygments.css b/scripts/css/pygments.css new file mode 100644 index 00000000..a212151a --- /dev/null +++ b/scripts/css/pygments.css @@ -0,0 +1,83 @@ +/* +Original styles generated from: + pygmentize -f html -S colorful -a pre.code > ./scripts/css/pygments.css + +Rules for which we don't want the syntax highlighter to kick in are commented +out at the bottom. + +Windows users: if you regenerate this file, you'll need to re-save it as utf-8 +to make docutils happy. +*/ + +/* DIFFS */ +pre.code .gd { color: #A00000 } /* Generic.Deleted */ +pre.code .gi { color: #00A000 } /* Generic.Inserted */ + +/* UNUSED */ +/*pre.code .hll { background-color: #ffffcc }*/ +/*pre.code { background: #ffffff; }*/ +/*pre.code .c { color: #888888 } !* Comment *!*/ +/*pre.code .err { color: #FF0000; background-color: #FFAAAA } !* Error *!*/ +/*pre.code .k { color: #008800; font-weight: bold } !* Keyword *!*/ +/*pre.code .o { color: #333333 } !* Operator *!*/ +/*pre.code .ch { color: #888888 } !* Comment.Hashbang *!*/ +/*pre.code .cm { color: #888888 } !* Comment.Multiline *!*/ +/*pre.code .cp { color: #557799 } !* Comment.Preproc *!*/ +/*pre.code .cpf { color: #888888 } !* Comment.PreprocFile *!*/ +/*pre.code .c1 { color: #888888 } !* Comment.Single *!*/ +/*pre.code .cs { color: #cc0000; font-weight: bold } !* Comment.Special *!*/ +/*pre.code .ge { font-style: italic } !* Generic.Emph *!*/ +/*pre.code .gr { color: #FF0000 } !* Generic.Error *!*/ +/*pre.code .gh { color: #000080; font-weight: bold } !* Generic.Heading *!*/ +/*pre.code .go { color: #888888 } !* Generic.Output *!*/ +/*pre.code .gp { color: #c65d09; font-weight: bold } !* Generic.Prompt *!*/ +/*pre.code .gs { font-weight: bold } !* Generic.Strong *!*/ +/*pre.code .gu { color: #800080; font-weight: bold } !* Generic.Subheading *!*/ +/*pre.code .gt { color: #0044DD } !* Generic.Traceback *!*/ +/*pre.code .kc { color: #008800; font-weight: bold } !* Keyword.Constant *!*/ +/*pre.code .kd { color: #008800; font-weight: bold } !* Keyword.Declaration *!*/ +/*pre.code .kn { color: #008800; font-weight: bold } !* Keyword.Namespace *!*/ +/*pre.code .kp { color: #003388; font-weight: bold } !* Keyword.Pseudo *!*/ +/*pre.code .kr { color: #008800; font-weight: bold } !* Keyword.Reserved *!*/ +/*pre.code .kt { color: #333399; font-weight: bold } !* Keyword.Type *!*/ +/*pre.code .m { color: #6600EE; font-weight: bold } !* Literal.Number *!*/ +/*pre.code .s { background-color: #fff0f0 } !* Literal.String *!*/ +/*pre.code .na { color: #0000CC } !* Name.Attribute *!*/ +/*pre.code .nb { color: #007020 } !* Name.Builtin *!*/ +/*pre.code .nc { color: #BB0066; font-weight: bold } !* Name.Class *!*/ +/*pre.code .no { color: #003366; font-weight: bold } !* Name.Constant *!*/ +/*pre.code .nd { color: #555555; font-weight: bold } !* Name.Decorator *!*/ +/*pre.code .ni { color: #880000; font-weight: bold } !* Name.Entity *!*/ +/*pre.code .ne { color: #FF0000; font-weight: bold } !* Name.Exception *!*/ +/*pre.code .nf { color: #0066BB; font-weight: bold } !* Name.Function *!*/ +/*pre.code .nl { color: #997700; font-weight: bold } !* Name.Label *!*/ +/*pre.code .nn { color: #0e84b5; font-weight: bold } !* Name.Namespace *!*/ +/*pre.code .nt { color: #007700 } !* Name.Tag *!*/ +/*pre.code .nv { color: #996633 } !* Name.Variable *!*/ +/*pre.code .ow { color: #000000; font-weight: bold } !* Operator.Word *!*/ +/*pre.code .w { color: #bbbbbb } !* Text.Whitespace *!*/ +/*pre.code .mb { color: #6600EE; font-weight: bold } !* Literal.Number.Bin *!*/ +/*pre.code .mf { color: #6600EE; font-weight: bold } !* Literal.Number.Float *!*/ +/*pre.code .mh { color: #005588; font-weight: bold } !* Literal.Number.Hex *!*/ +/*pre.code .mi { color: #0000DD; font-weight: bold } !* Literal.Number.Integer *!*/ +/*pre.code .mo { color: #4400EE; font-weight: bold } !* Literal.Number.Oct *!*/ +/*pre.code .sa { background-color: #fff0f0 } !* Literal.String.Affix *!*/ +/*pre.code .sb { background-color: #fff0f0 } !* Literal.String.Backtick *!*/ +/*pre.code .sc { color: #0044DD } !* Literal.String.Char *!*/ +/*pre.code .dl { background-color: #fff0f0 } !* Literal.String.Delimiter *!*/ +/*pre.code .sd { color: #DD4422 } !* Literal.String.Doc *!*/ +/*pre.code .s2 { background-color: #fff0f0 } !* Literal.String.Double *!*/ +/*pre.code .se { color: #666666; font-weight: bold; background-color: #fff0f0 } !* Literal.String.Escape *!*/ +/*pre.code .sh { background-color: #fff0f0 } !* Literal.String.Heredoc *!*/ +/*pre.code .si { background-color: #eeeeee } !* Literal.String.Interpol *!*/ +/*pre.code .sx { color: #DD2200; background-color: #fff0f0 } !* Literal.String.Other *!*/ +/*pre.code .sr { color: #000000; background-color: #fff0ff } !* Literal.String.Regex *!*/ +/*pre.code .s1 { background-color: #fff0f0 } !* Literal.String.Single *!*/ +/*pre.code .ss { color: #AA6600 } !* Literal.String.Symbol *!*/ +/*pre.code .bp { color: #007020 } !* Name.Builtin.Pseudo *!*/ +/*pre.code .fm { color: #0066BB; font-weight: bold } !* Name.Function.Magic *!*/ +/*pre.code .vc { color: #336699 } !* Name.Variable.Class *!*/ +/*pre.code .vg { color: #dd7700; font-weight: bold } !* Name.Variable.Global *!*/ +/*pre.code .vi { color: #3333BB } !* Name.Variable.Instance *!*/ +/*pre.code .vm { color: #996633 } !* Name.Variable.Magic *!*/ +/*pre.code .il { color: #0000DD; font-weight: bold } !* Literal.Number.Integer.Long *!*/ diff --git a/scripts/gendoc.py b/scripts/gendoc.py index 72e3047c..7e68ccd7 100755 --- a/scripts/gendoc.py +++ b/scripts/gendoc.py @@ -273,6 +273,7 @@ def rst2html(i, o, stylesheets): writer_name="html", settings_overrides={ "stylesheet_path": stylesheets, + "syntax_highlight": "short", }, ) diff --git a/specification/rooms/v6.rst b/specification/rooms/v6.rst index de0ab889..e165f771 100644 --- a/specification/rooms/v6.rst +++ b/specification/rooms/v6.rst @@ -55,5 +55,37 @@ Authorization rules for events Like redactions, all rules relating specifically to events of type ``m.room.aliases`` are removed. They must still pass authorization checks relating to state events. +Additionally, the authorization rules for events of type ``m.room.power_levels`` +now include the content key ``notifications``. This new rule takes the place of the +rule which checks the ``events`` and ``users`` keys. + +For completeness, the changes to the auth rules can be represented as follows: + +.. code:: diff + + ... + + -If type is `m.room.aliases`: + - + - a. If event has no `state_key`, reject. + - b. If sender's domain doesn't matches `state_key`, reject. + - c. Otherwise, allow. + + ... + + If type is `m.room.power_levels`: + + ... + + - * For each entry being added, changed or removed in both the `events` and `users` keys: + + * For each entry being added, changed or removed in the `events`, `users`, and `notifications` keys: + + i. If the current value is higher than the `sender`'s current power level, reject. + + ii. If the new value is higher than the `sender`'s current power level, reject. + + ... + + The remaining rules are the same as in `room version 3 `_ (the last inherited room version to specify the authorization rules). From 66ab480967c7dc1b2ad4f3750b94f837d1eda9ef Mon Sep 17 00:00:00 2001 From: Travis Ralston Date: Wed, 20 May 2020 21:58:58 -0600 Subject: [PATCH 090/114] Incorporate MSC2540 (Canonical JSON validation) MSC: https://github.com/matrix-org/matrix-doc/pull/2540 --- specification/appendices/signing_json.rst | 11 +++++++++++ specification/rooms/v1.rst | 6 ++++++ specification/rooms/v6.rst | 9 +++++++++ 3 files changed, 26 insertions(+) diff --git a/specification/appendices/signing_json.rst b/specification/appendices/signing_json.rst index 8036950e..fbeb0010 100644 --- a/specification/appendices/signing_json.rst +++ b/specification/appendices/signing_json.rst @@ -39,6 +39,17 @@ range where they can be accurately represented using IEEE double precision floating point numbers since a number of JSON libraries represent all numbers using this representation. +.. WARNING:: + Events in room versions 1, 2, 3, 4, and 5 might not be fully compliant with + these restrictions. Servers SHOULD be capable of handling JSON which is considered + invalid by these restrictions where possible. + + The most notable consideration is that integers might not be in the range + specified above. + +.. Note:: + Float values are not permitted by this encoding. + .. code:: python import json diff --git a/specification/rooms/v1.rst b/specification/rooms/v1.rst index 9282c1f3..a71bdfb4 100644 --- a/specification/rooms/v1.rst +++ b/specification/rooms/v1.rst @@ -352,6 +352,12 @@ Events in version 1 rooms have the following structure: {{definition_ss_pdu}} +Canonical JSON +~~~~~~~~~~~~~~ + +Servers MUST NOT strictly enforce the JSON format specified in the +`appendices <../appendices.html#canonical-json>`_ for the reasons described there. + .. _`auth events selection`: ../server_server/%SERVER_RELEASE_LABEL%.html#auth-events-selection .. _`Signing Events`: ../server_server/%SERVER_RELEASE_LABEL%.html#signing-events diff --git a/specification/rooms/v6.rst b/specification/rooms/v6.rst index e165f771..b1acccd2 100644 --- a/specification/rooms/v6.rst +++ b/specification/rooms/v6.rst @@ -89,3 +89,12 @@ For completeness, the changes to the auth rules can be represented as follows: The remaining rules are the same as in `room version 3 `_ (the last inherited room version to specify the authorization rules). + +Canonical JSON +~~~~~~~~~~~~~~ + +Servers MUST strictly enforce the JSON format specified in the +`appendices <../appendices.html#canonical-json>`_. This translates to a 400 ``M_BAD_JSON`` error +on most endpoints, or discarding of events over federation. For example, the federation API's +``/send`` endpoint would discard the event whereas the Client Server API's ``/send/{eventType}`` +endpoint would return a ``M_BAD_JSON`` error. From 98416bf94898395ae5f1b81bfcf8f659a1e76d8b Mon Sep 17 00:00:00 2001 From: Travis Ralston Date: Wed, 20 May 2020 22:16:16 -0600 Subject: [PATCH 091/114] Add changelog for client-server API --- changelogs/client_server/newsfragments/2563.clarification | 1 + 1 file changed, 1 insertion(+) create mode 100644 changelogs/client_server/newsfragments/2563.clarification diff --git a/changelogs/client_server/newsfragments/2563.clarification b/changelogs/client_server/newsfragments/2563.clarification new file mode 100644 index 00000000..a8805f4a --- /dev/null +++ b/changelogs/client_server/newsfragments/2563.clarification @@ -0,0 +1 @@ +Move redaction algorithm into the room version specifications. From 5b38c04e1921261dfe5eb7169068c1ef86a9cd33 Mon Sep 17 00:00:00 2001 From: Hubert Chathi Date: Fri, 22 May 2020 15:50:32 -0400 Subject: [PATCH 092/114] make definition of signatures fields more consistent --- api/client-server/definitions/device_keys.yaml | 5 +---- api/server-server/definitions/keys.yaml | 18 ++++++------------ api/server-server/third_party_invite.yaml | 12 ++++++------ api/server-server/user_keys.yaml | 8 +++++--- 4 files changed, 18 insertions(+), 25 deletions(-) diff --git a/api/client-server/definitions/device_keys.yaml b/api/client-server/definitions/device_keys.yaml index 41504aba..ce770c09 100644 --- a/api/client-server/definitions/device_keys.yaml +++ b/api/client-server/definitions/device_keys.yaml @@ -47,16 +47,13 @@ properties: "ed25519:JLAFKJWSCS": "lEuiRJBit0IG6nUf5pUzWTUEsRVVe/HJkoKuEww9ULI" signatures: type: object + title: Signatures description: |- Signatures for the device key object. A map from user ID, to a map from ``:`` to the signature. The signature is calculated using the process described at `Signing JSON`_. - additionalProperties: - type: object - additionalProperties: - type: string example: "@alice:example.com": "ed25519:JLAFKJWSCS": "dSO80A01XiigH3uBiDVx/EjzaoycHcjq9lfQX0uWsqxl2giMIiSPR8a4d291W1ihKJL/a+myXS367WT6NAIcBA" diff --git a/api/server-server/definitions/keys.yaml b/api/server-server/definitions/keys.yaml index 1e025a52..7e300402 100644 --- a/api/server-server/definitions/keys.yaml +++ b/api/server-server/definitions/keys.yaml @@ -75,18 +75,12 @@ properties: required: ["expired_ts", "key"] signatures: type: object - description: Digital signatures for this object signed using the ``verify_keys``. - additionalProperties: - type: object - title: Signed Server - example: { - "example.org": { - "ad25519:abc123": "VGhpcyBzaG91bGQgYWN0dWFsbHkgYmUgYSBzaWduYXR1cmU" - } - } - additionalProperties: - type: string - name: Encoded Signature Verification Key + description: |- + Digital signatures for this object signed using the ``verify_keys``. + + The signature is calculated using the process described at `Signing + JSON`_. + title: Signatures valid_until_ts: type: integer format: int64 diff --git a/api/server-server/third_party_invite.yaml b/api/server-server/third_party_invite.yaml index 0b7aac5b..8897bdcf 100644 --- a/api/server-server/third_party_invite.yaml +++ b/api/server-server/third_party_invite.yaml @@ -102,12 +102,12 @@ paths: properties: signatures: type: object - description: The server signatures for this event. - additionalProperties: - type: object - title: Server Signatures - additionalProperties: - type: string + title: Signatures + description: |- + The server signatures for this event. + + The signature is calculated using the process + described at `Signing JSON`_. example: { "magic.forest": { "ed25519:3": "fQpGIW1Snz+pwLZu6sTy2aHy/DYWWTspTJRPyNp0PKkymfIsNffysMl6ObMMFdIJhk6g6pwlIqZ54rxo8SLmAg" diff --git a/api/server-server/user_keys.yaml b/api/server-server/user_keys.yaml index 93237d80..9224d508 100644 --- a/api/server-server/user_keys.yaml +++ b/api/server-server/user_keys.yaml @@ -88,10 +88,12 @@ paths: description: The key, encoded using unpadded base64. signatures: type: object + title: Signatures description: |- - Signature for the device. Mapped from user ID to signature object. - additionalProperties: - type: string + Signature of the key object. + + The signature is calculated using the process described at `Signing + JSON`_. required: ['key', 'signatures'] example: { "@alice:example.com": { From 6a8ba9b1d6372f9a71d6b8554947548556e84c8e Mon Sep 17 00:00:00 2001 From: Hubert Chathi Date: Fri, 22 May 2020 16:10:16 -0400 Subject: [PATCH 093/114] improve description of key object in CS and include a link in Fed --- api/server-server/user_keys.yaml | 2 +- .../modules/end_to_end_encryption.rst | 20 ++++++++++++++++--- specification/server_server_api.rst | 1 + 3 files changed, 19 insertions(+), 4 deletions(-) diff --git a/api/server-server/user_keys.yaml b/api/server-server/user_keys.yaml index 9224d508..166f1ff0 100644 --- a/api/server-server/user_keys.yaml +++ b/api/server-server/user_keys.yaml @@ -73,7 +73,7 @@ paths: One-time keys for the queried devices. A map from user ID, to a map from devices to a map from ``:`` to the key object. - See the Client-Server Key Algorithms section for more information on + See the `Client-Server Key Algorithms`_ section for more information on the Key Object format. additionalProperties: type: object diff --git a/specification/modules/end_to_end_encryption.rst b/specification/modules/end_to_end_encryption.rst index 7411abe4..8bc46de8 100644 --- a/specification/modules/end_to_end_encryption.rst +++ b/specification/modules/end_to_end_encryption.rst @@ -83,9 +83,23 @@ Base64`_. Example: "JGLn/yafz74HB2AbPLYJWIVGnKAtqECOBf11yyXac2Y" The name ``signed_curve25519`` also corresponds to the Curve25519 algorithm, -but keys using this algorithm are objects with the properties ``key`` (giving -the Base64-encoded 32-byte Curve25519 public key), and ``signatures`` (giving a -signature for the key object, as described in `Signing JSON`_). Example: +but a key using this algorithm is represented by an object with a the following +properties: + +``KeyObject`` + +========== ================ ===================================================== +Parameter Type Description +========== ================ ===================================================== +key string **Required.** The unpadded Base64-encoded 32-byte + Curve25519 public key. +signatures Signatures **Required.** Signatures of the key object. + + The signature is calculated using the process described + at `Signing JSON`_. +========== ================ ===================================================== + +Example: .. code:: json diff --git a/specification/server_server_api.rst b/specification/server_server_api.rst index b0a2bc1a..00ddeffc 100644 --- a/specification/server_server_api.rst +++ b/specification/server_server_api.rst @@ -1257,3 +1257,4 @@ issue. .. _`Device Management module`: ../client_server/%CLIENT_RELEASE_LABEL%.html#device-management .. _`End-to-End Encryption module`: ../client_server/%CLIENT_RELEASE_LABEL%.html#end-to-end-encryption .. _`room version specification`: ../index.html#room-versions +.. _`Client-Server Key Algorithms`: ../client_server/%CLIENT_RELEASE_LABEL%.html#key-algorithms From 5270ed0c0ae6d1d35375c4aa2832cfa2391c2544 Mon Sep 17 00:00:00 2001 From: Tulir Asokan Date: Sun, 24 May 2020 14:24:49 +0300 Subject: [PATCH 094/114] Fix typo in read markers module Signed-off-by: Tulir Asokan --- specification/modules/read_markers.rst | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/specification/modules/read_markers.rst b/specification/modules/read_markers.rst index a77f2013..be06d037 100644 --- a/specification/modules/read_markers.rst +++ b/specification/modules/read_markers.rst @@ -47,7 +47,7 @@ The read markers API can additionally update the user's read receipt (``m.read`` location in the same operation as setting the fully read marker location. This is because read receipts and read markers are commonly updated at the same time, and therefore the client might wish to save an extra HTTP call. Providing an -``m.read`` location performs the same task as a request to ``/receipts/m.read/$event:example.org``. +``m.read`` location performs the same task as a request to ``/receipt/m.read/$event:example.org``. {{read_markers_cs_http_api}} @@ -56,7 +56,7 @@ Server behaviour The server MUST prevent clients from setting ``m.fully_read`` directly in room account data. The server must additionally ensure that it treats the presence of ``m.read`` in the ``/read_markers`` request the same as how it -would for a request to ``/receipts/m.read/$event:example.org``. +would for a request to ``/receipt/m.read/$event:example.org``. Upon updating the ``m.fully_read`` event due to a request to ``/read_markers``, the server MUST send the updated account data event through to the client via From bdd8f03fef638cb07cf0d38a21ad211ecf0d7ce1 Mon Sep 17 00:00:00 2001 From: Travis Ralston Date: Sun, 24 May 2020 11:13:58 -0600 Subject: [PATCH 095/114] Add changelog --- changelogs/client_server/newsfragments/2569.clarification | 1 + 1 file changed, 1 insertion(+) create mode 100644 changelogs/client_server/newsfragments/2569.clarification diff --git a/changelogs/client_server/newsfragments/2569.clarification b/changelogs/client_server/newsfragments/2569.clarification new file mode 100644 index 00000000..902a9c3f --- /dev/null +++ b/changelogs/client_server/newsfragments/2569.clarification @@ -0,0 +1 @@ +Fix various spelling errors throughout the specification. From 93f68866c5635078e84eaf8bcf55c8eb274c57f8 Mon Sep 17 00:00:00 2001 From: Travis Ralston Date: Mon, 25 May 2020 08:06:57 -0600 Subject: [PATCH 096/114] Fix spec generation not using titles on pseudo objects --- scripts/templating/matrix_templates/units.py | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/scripts/templating/matrix_templates/units.py b/scripts/templating/matrix_templates/units.py index 9d2c69f2..5a7ff94e 100644 --- a/scripts/templating/matrix_templates/units.py +++ b/scripts/templating/matrix_templates/units.py @@ -213,6 +213,7 @@ def get_json_schema_object_fields(obj, enforce_title=False): res = process_data_type(additionalProps) tables = res["tables"] val_title = res["title"] + gen_title = "{%s: %s}" % (key_type, val_title) if res.get("enum_desc") and val_title != "enum": # A map to enum needs another table with enum description tables.append(TypeTable( @@ -220,7 +221,7 @@ def get_json_schema_object_fields(obj, enforce_title=False): rows=[TypeTableRow(key="(mapped value)", title="enum", desc=res["desc"])] )) return { - "title": "{%s: %s}" % (key_type, val_title), + "title": obj_title if obj_title else gen_title, "tables": tables, } From 36ed6e94a2bd2c5e591fdd87cb15acdae53ca03a Mon Sep 17 00:00:00 2001 From: Travis Ralston Date: Mon, 25 May 2020 08:08:10 -0600 Subject: [PATCH 097/114] Re-introduce signature schemas --- api/client-server/definitions/device_keys.yaml | 4 ++++ api/client-server/definitions/third_party_signed.yaml | 4 ++++ api/server-server/definitions/keys.yaml | 4 ++++ api/server-server/third_party_invite.yaml | 4 ++++ api/server-server/user_keys.yaml | 4 ++++ event-schemas/schema/m.room.member | 4 ++++ 6 files changed, 24 insertions(+) diff --git a/api/client-server/definitions/device_keys.yaml b/api/client-server/definitions/device_keys.yaml index ce770c09..4f7cffe7 100644 --- a/api/client-server/definitions/device_keys.yaml +++ b/api/client-server/definitions/device_keys.yaml @@ -54,6 +54,10 @@ properties: The signature is calculated using the process described at `Signing JSON`_. + additionalProperties: + type: object + additionalProperties: + type: string example: "@alice:example.com": "ed25519:JLAFKJWSCS": "dSO80A01XiigH3uBiDVx/EjzaoycHcjq9lfQX0uWsqxl2giMIiSPR8a4d291W1ihKJL/a+myXS367WT6NAIcBA" diff --git a/api/client-server/definitions/third_party_signed.yaml b/api/client-server/definitions/third_party_signed.yaml index c9c761a1..7ce1a1d1 100644 --- a/api/client-server/definitions/third_party_signed.yaml +++ b/api/client-server/definitions/third_party_signed.yaml @@ -33,6 +33,10 @@ properties: type: object description: A signatures object containing a signature of the entire signed object. title: Signatures + additionalProperties: + type: object + additionalProperties: + type: string example: { "example.org": { "ed25519:0": "some9signature" diff --git a/api/server-server/definitions/keys.yaml b/api/server-server/definitions/keys.yaml index 7e300402..306d4d00 100644 --- a/api/server-server/definitions/keys.yaml +++ b/api/server-server/definitions/keys.yaml @@ -81,6 +81,10 @@ properties: The signature is calculated using the process described at `Signing JSON`_. title: Signatures + additionalProperties: + type: object + additionalProperties: + type: string valid_until_ts: type: integer format: int64 diff --git a/api/server-server/third_party_invite.yaml b/api/server-server/third_party_invite.yaml index 8897bdcf..fbd2a9d9 100644 --- a/api/server-server/third_party_invite.yaml +++ b/api/server-server/third_party_invite.yaml @@ -103,6 +103,10 @@ paths: signatures: type: object title: Signatures + additionalProperties: + type: object + additionalProperties: + type: string description: |- The server signatures for this event. diff --git a/api/server-server/user_keys.yaml b/api/server-server/user_keys.yaml index 166f1ff0..6ed51af0 100644 --- a/api/server-server/user_keys.yaml +++ b/api/server-server/user_keys.yaml @@ -89,6 +89,10 @@ paths: signatures: type: object title: Signatures + additionalProperties: + type: object + additionalProperties: + type: string description: |- Signature of the key object. diff --git a/event-schemas/schema/m.room.member b/event-schemas/schema/m.room.member index 1033c098..bff2002e 100644 --- a/event-schemas/schema/m.room.member +++ b/event-schemas/schema/m.room.member @@ -82,6 +82,10 @@ properties: description: 'A single signature from the verifying server, in the format specified by the Signing Events section of the server-server API.' title: Signatures type: object + additionalProperties: + type: object + additionalProperties: + type: string token: description: The token property of the containing third_party_invite object. type: string From 730d164df7fd23fb908f27bda4511f76b769cd36 Mon Sep 17 00:00:00 2001 From: Travis Ralston Date: Mon, 25 May 2020 08:15:56 -0600 Subject: [PATCH 098/114] Add changelogs --- changelogs/client_server/newsfragments/2566.clarification | 1 + changelogs/server_server/newsfragments/2566.clarification | 1 + 2 files changed, 2 insertions(+) create mode 100644 changelogs/client_server/newsfragments/2566.clarification create mode 100644 changelogs/server_server/newsfragments/2566.clarification diff --git a/changelogs/client_server/newsfragments/2566.clarification b/changelogs/client_server/newsfragments/2566.clarification new file mode 100644 index 00000000..15e29b2f --- /dev/null +++ b/changelogs/client_server/newsfragments/2566.clarification @@ -0,0 +1 @@ +Clarify signature object structures for encryption. diff --git a/changelogs/server_server/newsfragments/2566.clarification b/changelogs/server_server/newsfragments/2566.clarification new file mode 100644 index 00000000..15e29b2f --- /dev/null +++ b/changelogs/server_server/newsfragments/2566.clarification @@ -0,0 +1 @@ +Clarify signature object structures for encryption. From b876ac5893dddf4a6b6b4b8152ce0ea5c2d2f482 Mon Sep 17 00:00:00 2001 From: Travis Ralston Date: Mon, 25 May 2020 08:27:55 -0600 Subject: [PATCH 099/114] Clarify which server names to sign requests with Fixes https://github.com/matrix-org/matrix-doc/issues/2512 --- changelogs/server_server/newsfragments/2570.clarification | 1 + specification/server_server_api.rst | 5 +++++ 2 files changed, 6 insertions(+) create mode 100644 changelogs/server_server/newsfragments/2570.clarification diff --git a/changelogs/server_server/newsfragments/2570.clarification b/changelogs/server_server/newsfragments/2570.clarification new file mode 100644 index 00000000..3fac40bf --- /dev/null +++ b/changelogs/server_server/newsfragments/2570.clarification @@ -0,0 +1 @@ +Clarify the server names to use when signing requests. diff --git a/specification/server_server_api.rst b/specification/server_server_api.rst index b0a2bc1a..ca147665 100644 --- a/specification/server_server_api.rst +++ b/specification/server_server_api.rst @@ -291,6 +291,11 @@ Step 1 sign JSON: } } +The server names in the JSON above are the server names involved. Delegation from +the `server name resolution section <#resolving-server-names>`_ above do not affect +these - the server names from before delegation would take place are used. This +same condition applies throughout the request signing process. + Step 2 add Authorization header: .. code:: From 200212ffd741be53beec8fea1de1100826d44c66 Mon Sep 17 00:00:00 2001 From: Travis Ralston Date: Mon, 25 May 2020 08:33:08 -0600 Subject: [PATCH 100/114] Clarify that a creation event and member event are needed for createRoom Fixes https://github.com/matrix-org/matrix-doc/issues/2494 --- api/client-server/create_room.yaml | 16 +++++++++++----- .../newsfragments/2571.clarification | 1 + 2 files changed, 12 insertions(+), 5 deletions(-) create mode 100644 changelogs/client_server/newsfragments/2571.clarification diff --git a/api/client-server/create_room.yaml b/api/client-server/create_room.yaml index 2487707e..446d8aef 100644 --- a/api/client-server/create_room.yaml +++ b/api/client-server/create_room.yaml @@ -38,20 +38,26 @@ paths: the new room, including checking power levels for each event. It MUST apply the events implied by the request in the following order: - 0. A default ``m.room.power_levels`` event, giving the room creator + 1. The ``m.room.create`` event itself. Must be the first event in the + room. + + 2. An ``m.room.member`` event for the creator to join the room. This is + needed so the remaining events can be sent. + + 3. A default ``m.room.power_levels`` event, giving the room creator (and not other members) permission to send state events. Overridden by the ``power_level_content_override`` parameter. - 1. Events set by the ``preset``. Currently these are the ``m.room.join_rules``, + 4. Events set by the ``preset``. Currently these are the ``m.room.join_rules``, ``m.room.history_visibility``, and ``m.room.guest_access`` state events. - 2. Events listed in ``initial_state``, in the order that they are + 5. Events listed in ``initial_state``, in the order that they are listed. - 3. Events implied by ``name`` and ``topic`` (``m.room.name`` and ``m.room.topic`` + 6. Events implied by ``name`` and ``topic`` (``m.room.name`` and ``m.room.topic`` state events). - 4. Invite events implied by ``invite`` and ``invite_3pid`` (``m.room.member`` with + 7. Invite events implied by ``invite`` and ``invite_3pid`` (``m.room.member`` with ``membership: invite`` and ``m.room.third_party_invite``). The available presets do the following with respect to room state: diff --git a/changelogs/client_server/newsfragments/2571.clarification b/changelogs/client_server/newsfragments/2571.clarification new file mode 100644 index 00000000..e9437f3c --- /dev/null +++ b/changelogs/client_server/newsfragments/2571.clarification @@ -0,0 +1 @@ +Clarify which events are created as part of ``/createRoom``. From ac187904bbf334ea2a6b31abf4059f5eb3d27637 Mon Sep 17 00:00:00 2001 From: Travis Ralston Date: Mon, 25 May 2020 08:35:26 -0600 Subject: [PATCH 101/114] Update MSC1779 to forward link to matrix.org/foundation --- proposals/1779-open-governance.md | 11 +++++++++-- 1 file changed, 9 insertions(+), 2 deletions(-) diff --git a/proposals/1779-open-governance.md b/proposals/1779-open-governance.md index bfe86eb8..d9a53c51 100644 --- a/proposals/1779-open-governance.md +++ b/proposals/1779-open-governance.md @@ -1,4 +1,11 @@ -# Proposal for Open Governance of Matrix.org +# Proposal for Open Governance of Matrix.org + +---- + +**For updated information on the Foundation, please see https://matrix.org/foundation/**. +This MSC exists for historical reference. + +---- This whole document is the proposed constitution proposal for Matrix.org, and will form the basis of the first full Articles of Association (AoA) for [The @@ -446,7 +453,7 @@ a high-level mission lock aligned with the above: The foundation was then converted into a Community Interest Company, formalising its non-profit status under the approval of the independent [Community Interest -Companies Regulator](https://www.gov.uk/government/organisations/office-of-the-regulator-of-community-interest-companies), +Companies Regulator](https://www.gov.uk/government/organisations/office-of-the-regulator-of-community-interest-companies), which took effect Jan 2019. We are currently planning to release r0 of the Matrix Spec at the end of Jan 2019, and From e8548f3c3b92e92990657571af61dea1efc95b25 Mon Sep 17 00:00:00 2001 From: Travis Ralston Date: Mon, 25 May 2020 09:42:54 -0600 Subject: [PATCH 102/114] Remove claim that profile info is exclusively managed by the server Fixes https://github.com/matrix-org/matrix-doc/issues/2496 --- changelogs/client_server/newsfragments/2574.clarification | 1 + event-schemas/schema/m.room.member | 4 ++-- 2 files changed, 3 insertions(+), 2 deletions(-) create mode 100644 changelogs/client_server/newsfragments/2574.clarification diff --git a/changelogs/client_server/newsfragments/2574.clarification b/changelogs/client_server/newsfragments/2574.clarification new file mode 100644 index 00000000..d771d374 --- /dev/null +++ b/changelogs/client_server/newsfragments/2574.clarification @@ -0,0 +1 @@ +Remove claims that the homeserver is exclusively responsible for profile information in membership events. diff --git a/event-schemas/schema/m.room.member b/event-schemas/schema/m.room.member index 1033c098..dabd1b08 100644 --- a/event-schemas/schema/m.room.member +++ b/event-schemas/schema/m.room.member @@ -48,10 +48,10 @@ properties: content: properties: avatar_url: - description: 'The avatar URL for this user, if any. This is added by the homeserver.' + description: 'The avatar URL for this user, if any.' type: string displayname: - description: 'The display name for this user, if any. This is added by the homeserver.' + description: 'The display name for this user, if any.' type: - "string" - "null" From 06b0c1762a879f4db86b09261c69cfc1066c4231 Mon Sep 17 00:00:00 2001 From: Travis Ralston Date: Mon, 25 May 2020 09:55:32 -0600 Subject: [PATCH 103/114] Move the send_join response to its own definition Both APIs need clarification, so dedupe the changes. We should also probably deprecate some of this stuff so we can eventually remove it. --- .../definitions/send_join_response.yaml | 55 +++++++++++++++++++ api/server-server/joins-v1.yaml | 42 +------------- api/server-server/joins-v2.yaml | 42 +------------- 3 files changed, 57 insertions(+), 82 deletions(-) create mode 100644 api/server-server/definitions/send_join_response.yaml diff --git a/api/server-server/definitions/send_join_response.yaml b/api/server-server/definitions/send_join_response.yaml new file mode 100644 index 00000000..96ceaefc --- /dev/null +++ b/api/server-server/definitions/send_join_response.yaml @@ -0,0 +1,55 @@ +# Copyright 2018 New Vector Ltd +# Copyright 2020 The Matrix.org Foundation C.I.C. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +type: object +title: Room State +description: The state for the room. +properties: + origin: + type: string + description: The resident server's DNS name. + auth_chain: + type: array + description: |- + The auth chain. Note that events have a different format depending on + the room version - check the `room version specification`_ for precise + event formats. + items: + type: object + title: PDU + description: |- + The `PDUs <#pdus>`_ that make up the auth chain. The event format varies depending + on the room version - check the `room version specification`_ for precise event formats. + schema: + type: object + properties: [] + example: + $ref: "../examples/minimal_pdu.json" + state: + type: array + description: |- + The room state. The event format varies depending on the room version - + check the `room version specification`_ for precise event formats. + items: + type: object + title: PDU + description: |- + The `PDUs <#pdus>`_ for the fully resolved state of the room. The event format varies depending + on the room version - check the `room version specification`_ for precise event formats. + schema: + type: object + properties: [] + example: + $ref: "../examples/minimal_pdu.json" +required: ["auth_chain", "state", "origin"] diff --git a/api/server-server/joins-v1.yaml b/api/server-server/joins-v1.yaml index 39287264..18c88db0 100644 --- a/api/server-server/joins-v1.yaml +++ b/api/server-server/joins-v1.yaml @@ -262,47 +262,7 @@ paths: - type: integer description: The value ``200``. example: 200 - - type: object - title: Room State - description: The state for the room. - properties: - origin: - type: string - description: The resident server's DNS name. - auth_chain: - type: array - description: |- - The auth chain. Note that events have a different format depending on - the room version - check the `room version specification`_ for precise - event formats. - items: - type: object - title: PDU - description: |- - The `PDUs <#pdus>`_ that make up the auth chain. The event format varies depending - on the room version - check the `room version specification`_ for precise event formats. - schema: - type: object - properties: [] - example: - $ref: "examples/minimal_pdu.json" - state: - type: array - description: |- - The room state. The event format varies depending on the room version - - check the `room version specification`_ for precise event formats. - items: - type: object - title: PDU - description: |- - The `PDUs <#pdus>`_ for the fully resolved state of the room. The event format varies depending - on the room version - check the `room version specification`_ for precise event formats. - schema: - type: object - properties: [] - example: - $ref: "examples/minimal_pdu.json" - required: ["auth_chain", "state", "origin"] + - $ref: "./definitions/send_join_response.yaml" examples: application/json: [ 200, diff --git a/api/server-server/joins-v2.yaml b/api/server-server/joins-v2.yaml index a98d20fd..b5a1544c 100644 --- a/api/server-server/joins-v2.yaml +++ b/api/server-server/joins-v2.yaml @@ -127,47 +127,7 @@ paths: description: |- The full state for the room, having accepted the join event. schema: - type: object - title: Room State - description: The state for the room. - properties: - origin: - type: string - description: The resident server's DNS name. - auth_chain: - type: array - description: |- - The auth chain. Note that events have a different format depending on - the room version - check the `room version specification`_ for precise - event formats. - items: - type: object - title: PDU - description: |- - The `PDUs <#pdus>`_ that make up the auth chain. The event format varies depending - on the room version - check the `room version specification`_ for precise event formats. - schema: - type: object - properties: [] - example: - $ref: "examples/minimal_pdu.json" - state: - type: array - description: |- - The room state. The event format varies depending on the room version - - check the `room version specification`_ for precise event formats. - items: - type: object - title: PDU - description: |- - The `PDUs <#pdus>`_ for the fully resolved state of the room. The event format varies depending - on the room version - check the `room version specification`_ for precise event formats. - schema: - type: object - properties: [] - example: - $ref: "examples/minimal_pdu.json" - required: ["auth_chain", "state", "origin"] + $ref: "./definitions/send_join_response.yaml" examples: application/json: { "origin": "matrix.org", From 67d46fa881169ee6cea9d20028189d1869c5c83b Mon Sep 17 00:00:00 2001 From: Travis Ralston Date: Mon, 25 May 2020 10:59:41 -0600 Subject: [PATCH 104/114] Clarify the auth_chain and state for send_join Fixes https://github.com/matrix-org/matrix-doc/issues/2467 Fixes https://github.com/matrix-org/matrix-doc/issues/2464 This is based on looking at Synapse, not actually testing it. Within the send_join handling functions, an "event context" is acquired [here](https://github.com/matrix-org/synapse/blob/d14c4d6b6d02e8c55a6647ce0f0b3b44356d6173/synapse/handlers/federation.py#L1472) which then later uses the previous state (populated [here](https://github.com/matrix-org/synapse/blob/d14c4d6b6d02e8c55a6647ce0f0b3b44356d6173/synapse/state/__init__.py#L286-L289) in the event context) [here](https://github.com/matrix-org/synapse/blob/d14c4d6b6d02e8c55a6647ce0f0b3b44356d6173/synapse/handlers/federation.py#L1494-L1501) to return a response. Within the event context, the previous state IDs are populated [here](https://github.com/matrix-org/synapse/blob/d14c4d6b6d02e8c55a6647ce0f0b3b44356d6173/synapse/state/__init__.py#L351). --- .../definitions/send_join_response.yaml | 13 ++++++++----- .../server_server/newsfragments/2575.clarification | 1 + 2 files changed, 9 insertions(+), 5 deletions(-) create mode 100644 changelogs/server_server/newsfragments/2575.clarification diff --git a/api/server-server/definitions/send_join_response.yaml b/api/server-server/definitions/send_join_response.yaml index 96ceaefc..2837556b 100644 --- a/api/server-server/definitions/send_join_response.yaml +++ b/api/server-server/definitions/send_join_response.yaml @@ -22,9 +22,10 @@ properties: auth_chain: type: array description: |- - The auth chain. Note that events have a different format depending on - the room version - check the `room version specification`_ for precise - event formats. + The auth chain for the entire current room state prior to the join event. + + Note that events have a different format depending on the room version - check the + `room version specification`_ for precise event formats. items: type: object title: PDU @@ -39,8 +40,10 @@ properties: state: type: array description: |- - The room state. The event format varies depending on the room version - - check the `room version specification`_ for precise event formats. + The resolved current room state prior to the join event. + + The event format varies depending on the room version - check the `room version specification`_ + for precise event formats. items: type: object title: PDU diff --git a/changelogs/server_server/newsfragments/2575.clarification b/changelogs/server_server/newsfragments/2575.clarification new file mode 100644 index 00000000..5eb18394 --- /dev/null +++ b/changelogs/server_server/newsfragments/2575.clarification @@ -0,0 +1 @@ +Clarify the state/auth chain requirements for ``/send_join``. From e9998c53350442cc145dc7d05b7d77b18cd6abe7 Mon Sep 17 00:00:00 2001 From: Travis Ralston Date: Mon, 25 May 2020 12:21:27 -0600 Subject: [PATCH 105/114] Clarify that server names are at least 1 character Fixes https://github.com/matrix-org/matrix-doc/issues/2460 --- specification/appendices/identifier_grammar.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/specification/appendices/identifier_grammar.rst b/specification/appendices/identifier_grammar.rst index 4cce1f90..95c22fb1 100644 --- a/specification/appendices/identifier_grammar.rst +++ b/specification/appendices/identifier_grammar.rst @@ -45,7 +45,7 @@ following grammar:: IPv6char = DIGIT / %x41-46 / %x61-66 / ":" / "." ; 0-9, A-F, a-f, :, . - dns-name = *255dns-char + dns-name = 1*255dns-char dns-char = DIGIT / ALPHA / "-" / "." From 669377888ee6cf831f9ce0de2d1fa1b1457a1f90 Mon Sep 17 00:00:00 2001 From: Travis Ralston Date: Mon, 25 May 2020 12:24:29 -0600 Subject: [PATCH 106/114] Fix more typos throughout the specification Fixes https://github.com/matrix-org/matrix-doc/issues/2193 --- api/server-server/keys_server.yaml | 2 +- changelogs/server_server/newsfragments/2577.clarification | 1 + 2 files changed, 2 insertions(+), 1 deletion(-) create mode 100644 changelogs/server_server/newsfragments/2577.clarification diff --git a/api/server-server/keys_server.yaml b/api/server-server/keys_server.yaml index 465bb294..81b98b1e 100644 --- a/api/server-server/keys_server.yaml +++ b/api/server-server/keys_server.yaml @@ -34,7 +34,7 @@ paths: Intermediate notary servers should cache a response for half of its lifetime to avoid serving a stale response. Originating servers should avoid returning responses that expire in less than an hour to avoid - repeated reqests for a certificate that is about to expire. Requesting + repeated requests for a certificate that is about to expire. Requesting servers should limit how frequently they query for certificates to avoid flooding a server with requests. diff --git a/changelogs/server_server/newsfragments/2577.clarification b/changelogs/server_server/newsfragments/2577.clarification new file mode 100644 index 00000000..902a9c3f --- /dev/null +++ b/changelogs/server_server/newsfragments/2577.clarification @@ -0,0 +1 @@ +Fix various spelling errors throughout the specification. From d4c19a0e808e39dbcc92a7af1193cbb1daae803f Mon Sep 17 00:00:00 2001 From: Travis Ralston Date: Tue, 26 May 2020 12:36:18 -0600 Subject: [PATCH 107/114] Federation Co-authored-by: Andrew Morgan <1342360+anoadragon453@users.noreply.github.com> --- specification/rooms/v6.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/specification/rooms/v6.rst b/specification/rooms/v6.rst index b1acccd2..e5378d0e 100644 --- a/specification/rooms/v6.rst +++ b/specification/rooms/v6.rst @@ -95,6 +95,6 @@ Canonical JSON Servers MUST strictly enforce the JSON format specified in the `appendices <../appendices.html#canonical-json>`_. This translates to a 400 ``M_BAD_JSON`` error -on most endpoints, or discarding of events over federation. For example, the federation API's +on most endpoints, or discarding of events over federation. For example, the Federation API's ``/send`` endpoint would discard the event whereas the Client Server API's ``/send/{eventType}`` endpoint would return a ``M_BAD_JSON`` error. From bea6b321a59fb06d255bb2d02862613ca54fe732 Mon Sep 17 00:00:00 2001 From: Travis Ralston Date: Tue, 26 May 2020 12:37:52 -0600 Subject: [PATCH 108/114] Change the nots order Co-authored-by: Andrew Morgan <1342360+anoadragon453@users.noreply.github.com> --- api/client-server/directory.yaml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/api/client-server/directory.yaml b/api/client-server/directory.yaml index 67bdf236..7a08faf2 100644 --- a/api/client-server/directory.yaml +++ b/api/client-server/directory.yaml @@ -185,7 +185,7 @@ paths: membership. .. Note:: - Clients are not recommended to display this list of aliases prominently + Clients are recommended not to display this list of aliases prominently as they are not curated, unlike those listed in the ``m.room.canonical_alias`` state event. From a5cb9c655c083e38989215e6e405599def15f9f7 Mon Sep 17 00:00:00 2001 From: Travis Ralston Date: Tue, 26 May 2020 12:38:40 -0600 Subject: [PATCH 109/114] It's important to say who Co-authored-by: Andrew Morgan <1342360+anoadragon453@users.noreply.github.com> --- api/client-server/registration.yaml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/api/client-server/registration.yaml b/api/client-server/registration.yaml index a6d5d275..bab9139c 100644 --- a/api/client-server/registration.yaml +++ b/api/client-server/registration.yaml @@ -346,7 +346,7 @@ paths: logout_devices: type: boolean description: |- - Whether the other access tokens, and their associated devices, for the user should be + Whether the user's other access tokens, and their associated devices, should be revoked if the request succeeds. Defaults to true. When ``false``, the server can still take advantage of `the soft logout method <#soft-logout>`_ From 47c531adff5cea38dd4621a943257e6052a4997a Mon Sep 17 00:00:00 2001 From: Travis Ralston Date: Tue, 26 May 2020 12:46:50 -0600 Subject: [PATCH 110/114] Clarify exactly which servers Co-authored-by: Andrew Morgan <1342360+anoadragon453@users.noreply.github.com> --- specification/server_server_api.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/specification/server_server_api.rst b/specification/server_server_api.rst index ca147665..3d591ca8 100644 --- a/specification/server_server_api.rst +++ b/specification/server_server_api.rst @@ -291,7 +291,7 @@ Step 1 sign JSON: } } -The server names in the JSON above are the server names involved. Delegation from +The server names in the JSON above are the server names for each homeserver involved. Delegation from the `server name resolution section <#resolving-server-names>`_ above do not affect these - the server names from before delegation would take place are used. This same condition applies throughout the request signing process. From 4b64d804374eea4d48185772a5599f9c0201dd8f Mon Sep 17 00:00:00 2001 From: Travis Ralston Date: Tue, 26 May 2020 13:04:04 -0600 Subject: [PATCH 111/114] Remove v6 TODO To avoid the conflict, hopefully. --- specification/client_server_api.rst | 2 -- 1 file changed, 2 deletions(-) diff --git a/specification/client_server_api.rst b/specification/client_server_api.rst index fda2c106..6060388b 100644 --- a/specification/client_server_api.rst +++ b/specification/client_server_api.rst @@ -1820,8 +1820,6 @@ one of the following event types: - ``m.room.aliases`` allows key ``aliases``. - ``m.room.history_visibility`` allows key ``history_visibility``. -.. TODO: Room v6 changes how redaction works in MSC2432 (no special meaning for aliases) - The server should add the event causing the redaction to the ``unsigned`` property of the redacted event, under the ``redacted_because`` key. When a client receives a redaction event it should change the redacted event in the From 19a97c62e241347a3507d51c7bb617b6922aff5a Mon Sep 17 00:00:00 2001 From: Travis Ralston Date: Wed, 27 May 2020 07:19:31 -0600 Subject: [PATCH 112/114] Client-Server r0.6.1 --- changelogs/client_server.rst | 37 +++++++++++++++++++ .../newsfragments/2245.clarification | 1 - .../newsfragments/2351.clarification | 1 - .../newsfragments/2369.clarification | 1 - .../newsfragments/2381.clarification | 1 - .../newsfragments/2415.clarification | 1 - .../newsfragments/2430.clarification | 1 - .../client_server/newsfragments/2434.feature | 1 - .../newsfragments/2453.clarification | 1 - .../newsfragments/2492.clarification | 1 - .../newsfragments/2519.clarification | 1 - .../client_server/newsfragments/2523.feature | 1 - .../newsfragments/2524.clarification | 1 - .../newsfragments/2525.clarification | 1 - .../client_server/newsfragments/2532.feature | 1 - .../client_server/newsfragments/2546.feature | 1 - .../newsfragments/2553.clarification | 1 - .../newsfragments/2554.clarification | 1 - .../client_server/newsfragments/2562.feature | 1 - .../client_server/newsfragments/2562.new | 1 - .../newsfragments/2563.clarification | 1 - .../newsfragments/2566.clarification | 1 - .../newsfragments/2569.clarification | 1 - .../newsfragments/2571.clarification | 1 - .../newsfragments/2574.clarification | 1 - specification/client_server_api.rst | 1 + 26 files changed, 38 insertions(+), 24 deletions(-) delete mode 100644 changelogs/client_server/newsfragments/2245.clarification delete mode 100644 changelogs/client_server/newsfragments/2351.clarification delete mode 100644 changelogs/client_server/newsfragments/2369.clarification delete mode 100644 changelogs/client_server/newsfragments/2381.clarification delete mode 100644 changelogs/client_server/newsfragments/2415.clarification delete mode 100644 changelogs/client_server/newsfragments/2430.clarification delete mode 100644 changelogs/client_server/newsfragments/2434.feature delete mode 100644 changelogs/client_server/newsfragments/2453.clarification delete mode 100644 changelogs/client_server/newsfragments/2492.clarification delete mode 100644 changelogs/client_server/newsfragments/2519.clarification delete mode 100644 changelogs/client_server/newsfragments/2523.feature delete mode 100644 changelogs/client_server/newsfragments/2524.clarification delete mode 100644 changelogs/client_server/newsfragments/2525.clarification delete mode 100644 changelogs/client_server/newsfragments/2532.feature delete mode 100644 changelogs/client_server/newsfragments/2546.feature delete mode 100644 changelogs/client_server/newsfragments/2553.clarification delete mode 100644 changelogs/client_server/newsfragments/2554.clarification delete mode 100644 changelogs/client_server/newsfragments/2562.feature delete mode 100644 changelogs/client_server/newsfragments/2562.new delete mode 100644 changelogs/client_server/newsfragments/2563.clarification delete mode 100644 changelogs/client_server/newsfragments/2566.clarification delete mode 100644 changelogs/client_server/newsfragments/2569.clarification delete mode 100644 changelogs/client_server/newsfragments/2571.clarification delete mode 100644 changelogs/client_server/newsfragments/2574.clarification diff --git a/changelogs/client_server.rst b/changelogs/client_server.rst index bc12da5c..6f8aee1c 100644 --- a/changelogs/client_server.rst +++ b/changelogs/client_server.rst @@ -1,3 +1,40 @@ +r0.6.1 +====== + +New Endpoints +------------- + +- Added ``/rooms/{roomId}/aliases`` for retrieving local aliases for a room. (`#2562 `_) + + +Backwards Compatible Changes +---------------------------- + +- Added data structures for defining moderation policies in rooms per `MSC2313 `_. (`#2434 `_) +- Optionally invalidate other access tokens during password modification per `MSC2457 `_. (`#2523 `_) +- Add User-Interactive Authentication for SSO-backed homeserver per `MSC2454 `_. (`#2532 `_) +- Add soft-logout support per `MSC1466 `_. (`#2546 `_) +- Replaced legacy room alias handling with a more sustainable solution per `MSC2432 `_. (`#2562 `_) + + +Spec Clarifications +------------------- + +- List available enum values for the room versions capability. (`#2245 `_) +- Fix various spelling errors throughout the specification. (`#2351 `_, `#2415 `_, `#2453 `_, `#2524 `_, `#2553 `_, `#2569 `_) +- Minor clarifications to token-based User-Interactive Authentication. (`#2369 `_) +- Minor clarification for what the user directory searches. (`#2381 `_) +- Fix key export format example to match the specification. (`#2430 `_) +- Clarify the IV data type for encrypted files. (`#2492 `_) +- Fix the ``.m.rule.contains_user_name`` default push rule to set the highlight tweak. (`#2519 `_) +- Clarify that an ``event_id`` is returned when sending events. (`#2525 `_) +- Fix some numbers in the specification to match their explanation text. (`#2554 `_) +- Move redaction algorithm into the room version specifications. (`#2563 `_) +- Clarify signature object structures for encryption. (`#2566 `_) +- Clarify which events are created as part of ``/createRoom``. (`#2571 `_) +- Remove claims that the homeserver is exclusively responsible for profile information in membership events. (`#2574 `_) + + r0.6.0 ====== diff --git a/changelogs/client_server/newsfragments/2245.clarification b/changelogs/client_server/newsfragments/2245.clarification deleted file mode 100644 index 31e5c2df..00000000 --- a/changelogs/client_server/newsfragments/2245.clarification +++ /dev/null @@ -1 +0,0 @@ -List available enum values for the room versions capability. diff --git a/changelogs/client_server/newsfragments/2351.clarification b/changelogs/client_server/newsfragments/2351.clarification deleted file mode 100644 index 902a9c3f..00000000 --- a/changelogs/client_server/newsfragments/2351.clarification +++ /dev/null @@ -1 +0,0 @@ -Fix various spelling errors throughout the specification. diff --git a/changelogs/client_server/newsfragments/2369.clarification b/changelogs/client_server/newsfragments/2369.clarification deleted file mode 100644 index c7fd0fd9..00000000 --- a/changelogs/client_server/newsfragments/2369.clarification +++ /dev/null @@ -1 +0,0 @@ -Minor clarifications to token-based User-Interactive Authentication. diff --git a/changelogs/client_server/newsfragments/2381.clarification b/changelogs/client_server/newsfragments/2381.clarification deleted file mode 100644 index 70bd6d18..00000000 --- a/changelogs/client_server/newsfragments/2381.clarification +++ /dev/null @@ -1 +0,0 @@ -Minor clarification for what the user directory searches. diff --git a/changelogs/client_server/newsfragments/2415.clarification b/changelogs/client_server/newsfragments/2415.clarification deleted file mode 100644 index 902a9c3f..00000000 --- a/changelogs/client_server/newsfragments/2415.clarification +++ /dev/null @@ -1 +0,0 @@ -Fix various spelling errors throughout the specification. diff --git a/changelogs/client_server/newsfragments/2430.clarification b/changelogs/client_server/newsfragments/2430.clarification deleted file mode 100644 index 56be0c1a..00000000 --- a/changelogs/client_server/newsfragments/2430.clarification +++ /dev/null @@ -1 +0,0 @@ -Fix key export format example to match the specification. diff --git a/changelogs/client_server/newsfragments/2434.feature b/changelogs/client_server/newsfragments/2434.feature deleted file mode 100644 index 2b8f36f5..00000000 --- a/changelogs/client_server/newsfragments/2434.feature +++ /dev/null @@ -1 +0,0 @@ -Added data structures for defining moderation policies in rooms per `MSC2313 `_. diff --git a/changelogs/client_server/newsfragments/2453.clarification b/changelogs/client_server/newsfragments/2453.clarification deleted file mode 100644 index 902a9c3f..00000000 --- a/changelogs/client_server/newsfragments/2453.clarification +++ /dev/null @@ -1 +0,0 @@ -Fix various spelling errors throughout the specification. diff --git a/changelogs/client_server/newsfragments/2492.clarification b/changelogs/client_server/newsfragments/2492.clarification deleted file mode 100644 index 88c478ec..00000000 --- a/changelogs/client_server/newsfragments/2492.clarification +++ /dev/null @@ -1 +0,0 @@ -Clarify the IV data type for encrypted files. diff --git a/changelogs/client_server/newsfragments/2519.clarification b/changelogs/client_server/newsfragments/2519.clarification deleted file mode 100644 index 4b32da1e..00000000 --- a/changelogs/client_server/newsfragments/2519.clarification +++ /dev/null @@ -1 +0,0 @@ -Fix the ``.m.rule.contains_user_name`` default push rule to set the highlight tweak. diff --git a/changelogs/client_server/newsfragments/2523.feature b/changelogs/client_server/newsfragments/2523.feature deleted file mode 100644 index e45d1c2f..00000000 --- a/changelogs/client_server/newsfragments/2523.feature +++ /dev/null @@ -1 +0,0 @@ -Optionally invalidate other access tokens during password modification per `MSC2457 `_. diff --git a/changelogs/client_server/newsfragments/2524.clarification b/changelogs/client_server/newsfragments/2524.clarification deleted file mode 100644 index 902a9c3f..00000000 --- a/changelogs/client_server/newsfragments/2524.clarification +++ /dev/null @@ -1 +0,0 @@ -Fix various spelling errors throughout the specification. diff --git a/changelogs/client_server/newsfragments/2525.clarification b/changelogs/client_server/newsfragments/2525.clarification deleted file mode 100644 index df1fc538..00000000 --- a/changelogs/client_server/newsfragments/2525.clarification +++ /dev/null @@ -1 +0,0 @@ -Clarify that an ``event_id`` is returned when sending events. diff --git a/changelogs/client_server/newsfragments/2532.feature b/changelogs/client_server/newsfragments/2532.feature deleted file mode 100644 index cf74a289..00000000 --- a/changelogs/client_server/newsfragments/2532.feature +++ /dev/null @@ -1 +0,0 @@ -Add User-Interactive Authentication for SSO-backed homeserver per `MSC2454 `_. diff --git a/changelogs/client_server/newsfragments/2546.feature b/changelogs/client_server/newsfragments/2546.feature deleted file mode 100644 index 40c066f4..00000000 --- a/changelogs/client_server/newsfragments/2546.feature +++ /dev/null @@ -1 +0,0 @@ -Add soft-logout support per `MSC1466 `_. diff --git a/changelogs/client_server/newsfragments/2553.clarification b/changelogs/client_server/newsfragments/2553.clarification deleted file mode 100644 index 902a9c3f..00000000 --- a/changelogs/client_server/newsfragments/2553.clarification +++ /dev/null @@ -1 +0,0 @@ -Fix various spelling errors throughout the specification. diff --git a/changelogs/client_server/newsfragments/2554.clarification b/changelogs/client_server/newsfragments/2554.clarification deleted file mode 100644 index 826fafd8..00000000 --- a/changelogs/client_server/newsfragments/2554.clarification +++ /dev/null @@ -1 +0,0 @@ -Fix some numbers in the specification to match their explanation text. diff --git a/changelogs/client_server/newsfragments/2562.feature b/changelogs/client_server/newsfragments/2562.feature deleted file mode 100644 index 556018da..00000000 --- a/changelogs/client_server/newsfragments/2562.feature +++ /dev/null @@ -1 +0,0 @@ -Replaced legacy room alias handling with a more sustainable solution per `MSC2432 `_. diff --git a/changelogs/client_server/newsfragments/2562.new b/changelogs/client_server/newsfragments/2562.new deleted file mode 100644 index c7b3e4ca..00000000 --- a/changelogs/client_server/newsfragments/2562.new +++ /dev/null @@ -1 +0,0 @@ -Added ``/rooms/{roomId}/aliases`` for retrieving local aliases for a room. diff --git a/changelogs/client_server/newsfragments/2563.clarification b/changelogs/client_server/newsfragments/2563.clarification deleted file mode 100644 index a8805f4a..00000000 --- a/changelogs/client_server/newsfragments/2563.clarification +++ /dev/null @@ -1 +0,0 @@ -Move redaction algorithm into the room version specifications. diff --git a/changelogs/client_server/newsfragments/2566.clarification b/changelogs/client_server/newsfragments/2566.clarification deleted file mode 100644 index 15e29b2f..00000000 --- a/changelogs/client_server/newsfragments/2566.clarification +++ /dev/null @@ -1 +0,0 @@ -Clarify signature object structures for encryption. diff --git a/changelogs/client_server/newsfragments/2569.clarification b/changelogs/client_server/newsfragments/2569.clarification deleted file mode 100644 index 902a9c3f..00000000 --- a/changelogs/client_server/newsfragments/2569.clarification +++ /dev/null @@ -1 +0,0 @@ -Fix various spelling errors throughout the specification. diff --git a/changelogs/client_server/newsfragments/2571.clarification b/changelogs/client_server/newsfragments/2571.clarification deleted file mode 100644 index e9437f3c..00000000 --- a/changelogs/client_server/newsfragments/2571.clarification +++ /dev/null @@ -1 +0,0 @@ -Clarify which events are created as part of ``/createRoom``. diff --git a/changelogs/client_server/newsfragments/2574.clarification b/changelogs/client_server/newsfragments/2574.clarification deleted file mode 100644 index d771d374..00000000 --- a/changelogs/client_server/newsfragments/2574.clarification +++ /dev/null @@ -1 +0,0 @@ -Remove claims that the homeserver is exclusively responsible for profile information in membership events. diff --git a/specification/client_server_api.rst b/specification/client_server_api.rst index ad8539cb..574999be 100644 --- a/specification/client_server_api.rst +++ b/specification/client_server_api.rst @@ -45,6 +45,7 @@ Other versions of this specification The following other versions are also available, in reverse chronological order: - `HEAD `_: Includes all changes since the latest versioned release. +- `r0.6.1 `_ - `r0.6.0 `_ - `r0.5.0 `_ - `r0.4.0 `_ From 09986af8fc2ccda4a4bbbfec3c0651663da2f84d Mon Sep 17 00:00:00 2001 From: Travis Ralston Date: Wed, 27 May 2020 07:21:59 -0600 Subject: [PATCH 113/114] Server-Server r0.1.4 Includes a link to r0.1.3 because we forgot --- changelogs/server_server.rst | 31 +++++++++++++++++++ .../server_server/newsfragments/2035.new | 1 - .../newsfragments/2392.clarification | 1 - .../server_server/newsfragments/2470.removal | 1 - .../newsfragments/2510.clarification | 1 - .../newsfragments/2527.clarification | 1 - .../newsfragments/2538.clarification | 1 - .../server_server/newsfragments/2547.new | 1 - .../newsfragments/2559.clarification | 1 - .../newsfragments/2560.clarification | 1 - .../newsfragments/2566.clarification | 1 - .../newsfragments/2570.clarification | 1 - .../newsfragments/2575.clarification | 1 - .../newsfragments/2577.clarification | 1 - specification/server_server_api.rst | 2 ++ 15 files changed, 33 insertions(+), 13 deletions(-) delete mode 100644 changelogs/server_server/newsfragments/2035.new delete mode 100644 changelogs/server_server/newsfragments/2392.clarification delete mode 100644 changelogs/server_server/newsfragments/2470.removal delete mode 100644 changelogs/server_server/newsfragments/2510.clarification delete mode 100644 changelogs/server_server/newsfragments/2527.clarification delete mode 100644 changelogs/server_server/newsfragments/2538.clarification delete mode 100644 changelogs/server_server/newsfragments/2547.new delete mode 100644 changelogs/server_server/newsfragments/2559.clarification delete mode 100644 changelogs/server_server/newsfragments/2560.clarification delete mode 100644 changelogs/server_server/newsfragments/2566.clarification delete mode 100644 changelogs/server_server/newsfragments/2570.clarification delete mode 100644 changelogs/server_server/newsfragments/2575.clarification delete mode 100644 changelogs/server_server/newsfragments/2577.clarification diff --git a/changelogs/server_server.rst b/changelogs/server_server.rst index 22da543c..ee4a3c1c 100644 --- a/changelogs/server_server.rst +++ b/changelogs/server_server.rst @@ -1,3 +1,34 @@ +r0.1.4 +====== + +New Endpoints +------------- + +- Add new ``POST /publicRooms`` endpoint for filtering the room directory. (`#2035 `_) +- Add new v2 ``/send_join`` and ``/send_leave`` endpoints per `MSC1802 `_. (`#2547 `_) + + +Removed Endpoints +----------------- + +- Remove the unused ``query_auth`` API per `MSC2451 `_. (`#2470 `_) + + +Spec Clarifications +------------------- + +- Move auth event selection to a more obvious location. (`#2392 `_) +- Fix typo in Request Authentication python example. (`#2510 `_) +- Clarify which fields are required on the key server endpoints. (`#2527 `_) +- Clarify the limits of ``prev_events`` and ``auth_events`` for PDUs. (`#2538 `_) +- Clarify which events are targeted by backfill. (`#2559 `_) +- Fix the response format of the ``/send`` endpoint. (`#2560 `_) +- Clarify signature object structures for encryption. (`#2566 `_) +- Clarify the server names to use when signing requests. (`#2570 `_) +- Clarify the state/auth chain requirements for ``/send_join``. (`#2575 `_) +- Fix various spelling errors throughout the specification. (`#2577 `_) + + r0.1.3 ====== diff --git a/changelogs/server_server/newsfragments/2035.new b/changelogs/server_server/newsfragments/2035.new deleted file mode 100644 index 6794c4ea..00000000 --- a/changelogs/server_server/newsfragments/2035.new +++ /dev/null @@ -1 +0,0 @@ -Add new ``POST /publicRooms`` endpoint for filtering the room directory. diff --git a/changelogs/server_server/newsfragments/2392.clarification b/changelogs/server_server/newsfragments/2392.clarification deleted file mode 100644 index b09420e4..00000000 --- a/changelogs/server_server/newsfragments/2392.clarification +++ /dev/null @@ -1 +0,0 @@ -Move auth event selection to a more obvious location. diff --git a/changelogs/server_server/newsfragments/2470.removal b/changelogs/server_server/newsfragments/2470.removal deleted file mode 100644 index 51cdadd4..00000000 --- a/changelogs/server_server/newsfragments/2470.removal +++ /dev/null @@ -1 +0,0 @@ -Remove the unused ``query_auth`` API per `MSC2451 `_. diff --git a/changelogs/server_server/newsfragments/2510.clarification b/changelogs/server_server/newsfragments/2510.clarification deleted file mode 100644 index 9c96c5a5..00000000 --- a/changelogs/server_server/newsfragments/2510.clarification +++ /dev/null @@ -1 +0,0 @@ -Fix typo in Request Authentication python example. diff --git a/changelogs/server_server/newsfragments/2527.clarification b/changelogs/server_server/newsfragments/2527.clarification deleted file mode 100644 index 329d5da2..00000000 --- a/changelogs/server_server/newsfragments/2527.clarification +++ /dev/null @@ -1 +0,0 @@ -Clarify which fields are required on the key server endpoints. diff --git a/changelogs/server_server/newsfragments/2538.clarification b/changelogs/server_server/newsfragments/2538.clarification deleted file mode 100644 index 4b709a8d..00000000 --- a/changelogs/server_server/newsfragments/2538.clarification +++ /dev/null @@ -1 +0,0 @@ -Clarify the limits of ``prev_events`` and ``auth_events`` for PDUs. diff --git a/changelogs/server_server/newsfragments/2547.new b/changelogs/server_server/newsfragments/2547.new deleted file mode 100644 index c558fb5b..00000000 --- a/changelogs/server_server/newsfragments/2547.new +++ /dev/null @@ -1 +0,0 @@ -Add new v2 ``/send_join`` and ``/send_leave`` endpoints per `MSC1802 `_. diff --git a/changelogs/server_server/newsfragments/2559.clarification b/changelogs/server_server/newsfragments/2559.clarification deleted file mode 100644 index cdb9cfe0..00000000 --- a/changelogs/server_server/newsfragments/2559.clarification +++ /dev/null @@ -1 +0,0 @@ -Clarify which events are targeted by backfill. diff --git a/changelogs/server_server/newsfragments/2560.clarification b/changelogs/server_server/newsfragments/2560.clarification deleted file mode 100644 index c60bc6f3..00000000 --- a/changelogs/server_server/newsfragments/2560.clarification +++ /dev/null @@ -1 +0,0 @@ -Fix the response format of the ``/send`` endpoint. diff --git a/changelogs/server_server/newsfragments/2566.clarification b/changelogs/server_server/newsfragments/2566.clarification deleted file mode 100644 index 15e29b2f..00000000 --- a/changelogs/server_server/newsfragments/2566.clarification +++ /dev/null @@ -1 +0,0 @@ -Clarify signature object structures for encryption. diff --git a/changelogs/server_server/newsfragments/2570.clarification b/changelogs/server_server/newsfragments/2570.clarification deleted file mode 100644 index 3fac40bf..00000000 --- a/changelogs/server_server/newsfragments/2570.clarification +++ /dev/null @@ -1 +0,0 @@ -Clarify the server names to use when signing requests. diff --git a/changelogs/server_server/newsfragments/2575.clarification b/changelogs/server_server/newsfragments/2575.clarification deleted file mode 100644 index 5eb18394..00000000 --- a/changelogs/server_server/newsfragments/2575.clarification +++ /dev/null @@ -1 +0,0 @@ -Clarify the state/auth chain requirements for ``/send_join``. diff --git a/changelogs/server_server/newsfragments/2577.clarification b/changelogs/server_server/newsfragments/2577.clarification deleted file mode 100644 index 902a9c3f..00000000 --- a/changelogs/server_server/newsfragments/2577.clarification +++ /dev/null @@ -1 +0,0 @@ -Fix various spelling errors throughout the specification. diff --git a/specification/server_server_api.rst b/specification/server_server_api.rst index 61d4375d..9deb267c 100644 --- a/specification/server_server_api.rst +++ b/specification/server_server_api.rst @@ -80,6 +80,8 @@ Other versions of this specification The following other versions are also available, in reverse chronological order: - `HEAD `_: Includes all changes since the latest versioned release. +- `r0.1.4 `_ +- `r0.1.3 `_ - `r0.1.2 `_ - `r0.1.1 `_ - `r0.1.0 `_ From 3fc5540755b21ff36499e74137cd31f1b9413e63 Mon Sep 17 00:00:00 2001 From: Travis Ralston Date: Fri, 29 May 2020 07:00:38 -0600 Subject: [PATCH 114/114] Fix r0.1.4 changelog reference --- changelogs/server_server.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/changelogs/server_server.rst b/changelogs/server_server.rst index ee4a3c1c..367fb092 100644 --- a/changelogs/server_server.rst +++ b/changelogs/server_server.rst @@ -4,7 +4,7 @@ r0.1.4 New Endpoints ------------- -- Add new ``POST /publicRooms`` endpoint for filtering the room directory. (`#2035 `_) +- Add new ``POST /publicRooms`` endpoint for filtering the room directory. (`#2305 `_) - Add new v2 ``/send_join`` and ``/send_leave`` endpoints per `MSC1802 `_. (`#2547 `_)