From 92f3989f2b00627201cb16a5fdfd2ee1fa095180 Mon Sep 17 00:00:00 2001 From: Will Hunt Date: Thu, 5 Jul 2018 11:37:06 +0100 Subject: [PATCH 01/16] Dirty replace of DNS name -> hostname --- specification/server_server_api.rst | 20 ++++++++++---------- 1 file changed, 10 insertions(+), 10 deletions(-) diff --git a/specification/server_server_api.rst b/specification/server_server_api.rst index 3738ea05..70d495d3 100644 --- a/specification/server_server_api.rst +++ b/specification/server_server_api.rst @@ -18,7 +18,7 @@ Federation API Matrix homeservers use the Federation APIs (also known as server-server APIs) to communicate with each other. Homeservers use these APIs to push messages to -each other in real-time, to +each other in real-time, to historic messages from each other, and to query profile and presence information about users on each other's servers. @@ -73,22 +73,22 @@ Server Discovery Resolving Server Names ~~~~~~~~~~~~~~~~~~~~~~ -Each matrix homeserver is identified by a server name consisting of a DNS name +Each matrix homeserver is identified by a server name consisting of a hostname and an optional TLS port. .. code:: - server_name = dns_name [ ":" tls_port] + server_name = hostname [ ":" tls_port] dns_name = tls_port = *DIGIT .. ** If the port is present then the server is discovered by looking up an AAAA or -A record for the DNS name and connecting to the specified TLS port. If the port +A record for the hostname and connecting to the specified TLS port. If the port is absent then the server is discovered by looking up a ``_matrix._tcp`` SRV -record for the DNS name. If this record does not exist then the server is -discovered by looking up an AAAA or A record on the DNS name and taking the +record for the hostname. If this record does not exist then the server is +discovered by looking up an AAAA or A record on the hostname and taking the default fallback port number of 8448. Homeservers may use SRV records to load balance requests between multiple TLS endpoints or to failover to another endpoint if an endpoint fails. @@ -165,7 +165,7 @@ events sent by that server can still be checked. ==================== =================== ====================================== Key Type Description ==================== =================== ====================================== -``server_name`` String DNS name of the homeserver. +``server_name`` String hostname of the homeserver. ``verify_keys`` Object Public keys of the homeserver for verifying digital signatures. ``old_verify_keys`` Object The public keys that the server used @@ -273,7 +273,7 @@ at ``/_matrix/key/v1``. ==================== =================== ====================================== Key Type Description ==================== =================== ====================================== -``server_name`` String DNS name of the homeserver. +``server_name`` String hostname of the homeserver. ``verify_keys`` Object Public keys of the homeserver for verifying digital signatures. ``signatures`` Object Digital signatures for this object @@ -897,10 +897,10 @@ the resident homeserver. The required fields are: ======================== ============ ========================================= ``type`` String The value ``m.room.member``. ``auth_events`` List An event-reference list containing the - authorization events that would allow + authorization events that would allow this member to join. ``content`` Object The event content. -``depth`` Integer (this field must be present but is +``depth`` Integer (this field must be present but is ignored; it may be 0) ``origin`` String The name of the resident homeserver. ``origin_server_ts`` Integer A timestamp added by the resident From 26505533dd497859ae69b18ecd008fa56a14d563 Mon Sep 17 00:00:00 2001 From: Will Hunt Date: Tue, 24 Jul 2018 16:06:59 +0100 Subject: [PATCH 02/16] drop dns_name = host in favour of host --- specification/appendices/identifier_grammar.rst | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/specification/appendices/identifier_grammar.rst b/specification/appendices/identifier_grammar.rst index e85bf410..3abcfae2 100644 --- a/specification/appendices/identifier_grammar.rst +++ b/specification/appendices/identifier_grammar.rst @@ -25,8 +25,7 @@ number of identifiers, as described below. The server name represents the address at which the homeserver in question can be reached by other homeservers. The complete grammar is:: - server_name = dns_name [ ":" port] - dns_name = host + server_name = host [ ":" port] port = *DIGIT where ``host`` is as defined by `RFC3986, section 3.2.2 From 0ddf578b6159e0c8effa5cbbb019efc9290fdff9 Mon Sep 17 00:00:00 2001 From: Travis Ralston Date: Fri, 20 Jul 2018 11:26:16 -0600 Subject: [PATCH 03/16] Combine all queries into their own section This removes the Directory and Profile sections, instead opting to document them as Queries. The behaviour of profile queries is based on Synapse's behaviour. A few issues have been opened to improve the behaviour: * https://github.com/matrix-org/matrix-doc/issues/1434 * https://github.com/matrix-org/matrix-doc/issues/1435 * https://github.com/matrix-org/matrix-doc/issues/1436 * https://github.com/matrix-org/matrix-doc/issues/1437 This fixes https://github.com/matrix-org/matrix-doc/issues/1404 --- api/server-server/directory.yaml | 69 ---------- api/server-server/query.yaml | 181 +++++++++++++++++++++++++++ api/server-server/query_general.yaml | 44 ------- specification/server_server_api.rst | 38 ++---- 4 files changed, 190 insertions(+), 142 deletions(-) delete mode 100644 api/server-server/directory.yaml create mode 100644 api/server-server/query.yaml delete mode 100644 api/server-server/query_general.yaml diff --git a/api/server-server/directory.yaml b/api/server-server/directory.yaml deleted file mode 100644 index 94f115b9..00000000 --- a/api/server-server/directory.yaml +++ /dev/null @@ -1,69 +0,0 @@ -# Copyright 2017 Kamax.io -# Copyright 2018 New Vector Ltd -# -# Licensed under the Apache License, Version 2.0 (the "License"); -# you may not use this file except in compliance with the License. -# You may obtain a copy of the License at -# -# http://www.apache.org/licenses/LICENSE-2.0 -# -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. - -swagger: '2.0' -info: - title: "Matrix Federation Query API" - version: "1.0.0" -host: localhost:8448 -schemes: - - https -basePath: /_matrix/federation/v1 -produces: - - application/json -paths: - "/query/directory": - get: - summary: Retrieve the room ID and list of resident homeservers for a room - alias. - description: Retrieve the room ID and list of resident homeservers for a room - alias. - parameters: - - in: query - name: room_alias - type: string - description: Room alias. - required: true - x-example: "#room_alias:example.org" - responses: - 200: - description: The corresponding room ID and list of known resident - homeservers for the room. - schema: - type: object - properties: - room_id: - type: string - description: The room ID mapped to the queried room alias. - x-example: "!roomid1234:example.org" - servers: - type: array - description: An array of server names that are likely to hold - then given room. This list may or may not include the server - answering the query. - items: - type: string - required: - - "room_id" - - "servers" - examples: - application/json: { - "room_id": "!roomid1234:example.org", - "servers": [ - "example.org", - "example.com", - "another.example.com:8449", - ] - } diff --git a/api/server-server/query.yaml b/api/server-server/query.yaml new file mode 100644 index 00000000..23b9a575 --- /dev/null +++ b/api/server-server/query.yaml @@ -0,0 +1,181 @@ +# Copyright 2017 Kamax.io +# Copyright 2018 New Vector Ltd +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +swagger: '2.0' +info: + title: "Matrix Federation Query API" + version: "1.0.0" +host: localhost:8448 +schemes: + - https +basePath: /_matrix/federation/v1 +produces: + - application/json +paths: + "/query/{queryType}": + get: + summary: Query for information + description: |- + Performs a single query request on the receiving homeserver. The query string + arguments are dependent on which type of query is being made. Known query types + are specified as their own endpoints as an extension to this definition. + operationId: queryInfo + parameters: + - in: path + name: queryType + type: string + description: The type of query to make + required: true + x-example: profile + responses: + 200: + description: |- + The query response. The schema varies depending on the query being made. + "/query/directory": + get: + summary: Query for the room ID and resident homeservers for a room alias + description: |- + Performs a query to get the mapped room ID and list of resident homeservers in + the room for a given room alias. Homeservers should only query room aliases + that belong to the target server (idenfified by the DNS Name in the alias). + The target server may not appear in the resident servers list. + + Servers may wish to cache the response to this query to prevent requesting the + information too often. + operationId: queryRoomDirectory + parameters: + - in: query + name: room_alias + type: string + description: The room alias to query. + required: true + x-example: "#room_alias:example.org" + responses: + 200: + description: |- + The corresponding room ID and list of known resident homeservers for the room. + schema: + type: object + properties: + room_id: + type: string + description: The room ID mapped to the queried room alias. + x-example: "!roomid1234:example.org" + servers: + type: array + description: |- + An array of server names that are likely to hold then given room. This + list may or may not include the server answering the query. + items: + type: string + required: + - "room_id" + - "servers" + examples: + application/json: { + "room_id": "!roomid1234:example.org", + "servers": [ + "example.org", + "example.com", + "another.example.com:8449", + ] + } + 400: + description: |- + The room alias is not hosted on the server. This can happen if the directory + server is named "example.org" and the room alias ends with "matrix.org". + schema: + $ref: "../client-server/definitions/errors/error.yaml" + examples: + application/json: { + "errcode": "M_UNKNOWN", + "error": "Room alias not hosted on this homeserver." + } + 404: + description: The room alias was not found. + schema: + $ref: "../client-server/definitions/errors/error.yaml" + examples: + application/json: { + "errcode": "M_NOT_FOUND", + "error": "Room alias not found." + } + "/query/profile": + get: + summary: Query for profile information about a given user + description: |- + Performs a query to get profile information, such as a display name or avatar, + for a given user. Homeservers should only query profiles for users that belong + to the target server (identified by the DNS Name in the user ID). + + Servers may wish to cache the response to this query to prevent requesting the + information too often. + parameters: + - in: query + name: user_id + type: string + description: The user ID to query. + required: true + x-example: "@someone:example.org" + - in: query + name: field + type: string + description: |- + The field to query. If specified, the server will only return the given field + in the response. If not specified, the server will return the full profile for + the user. + responses: + 200: + description: |- + The profile for the user. If a ``field`` is specified in the request, only the + matching field should be included in the response. If no ``field`` was specified, + the response should include the fields of the user's profile that can be made + public, such as the display name and avatar. + + If the ``field`` is for a field that the server does not recognize, an empty object + should be returned. If the ``field`` is recognized, but the user does not have it + set on their profile, ``null`` should be returned for the value of that field. + + If the user does not exist, an empty object should be returned regardless of the + ``field`` being queried. + schema: + type: object + properties: + displayname: + type: string + description: |- + The display name of the user. If the user does not have a display name set, + this should be the value ``null``. + x-example: "John Doe" + avatar_url: + type: string + description: |- + The avatar URL for the user's avatar. If the user does not have an avatar + set, this should be the value ``null``. + x-example: "mxc://matrix.org/MyC00lAvatar" + examples: + application/json: { + "displayname": "John Doe", + "avatar_url": "mxc://matrix.org/MyC00lAvatar" + } + 400: + description: The user does not belong to the server. + schema: + $ref: "../client-server/definitions/errors/error.yaml" + examples: + application/json: { + "errcode": "M_UNKNOWN", + "error": "User is not hosted on this homeserver." + } diff --git a/api/server-server/query_general.yaml b/api/server-server/query_general.yaml deleted file mode 100644 index 97e3406e..00000000 --- a/api/server-server/query_general.yaml +++ /dev/null @@ -1,44 +0,0 @@ -# Copyright 2018 New Vector Ltd -# -# Licensed under the Apache License, Version 2.0 (the "License"); -# you may not use this file except in compliance with the License. -# You may obtain a copy of the License at -# -# http://www.apache.org/licenses/LICENSE-2.0 -# -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. - -swagger: '2.0' -info: - title: "Matrix Federation Query API" - version: "1.0.0" -host: localhost:8448 -schemes: - - https -basePath: /_matrix/federation/v1 -produces: - - application/json -paths: - "/query/{queryType}": - get: - summary: Query for information - description: |- - Performs a single query request on the receiving homeserver. The query string - arguments are dependent on which type of query is being made. Known query types - are specified as their own endpoints as an extension to this definition. - operationId: queryInfo - parameters: - - in: path - name: queryType - type: string - description: The type of query to make - required: true - x-example: profile - responses: - 200: - description: |- - The query response. The schema varies depending on the query being made. diff --git a/specification/server_server_api.rst b/specification/server_server_api.rst index f1825f27..b7d5a82f 100644 --- a/specification/server_server_api.rst +++ b/specification/server_server_api.rst @@ -686,8 +686,6 @@ All these URLs are name-spaced within a prefix of:: {{events_ss_http_api}} -{{query_general_ss_http_api}} - {{joins_ss_http_api}} @@ -1107,36 +1105,18 @@ Rejecting a presence invite:: - Explain the zero-byte presence inference logic See also: docs/client-server/model/presence -Profiles --------- +Querying for information +------------------------ -The server API for profiles is based entirely on the following Federation -Queries. There are no additional EDU or PDU types involved, other than the -implicit ``m.presence`` and ``m.room.member`` events (see section below). +Queries are a way to retrieve information from a homeserver abotu a resource, +such as a user or room. The endpoints here are often called in conjunction with +a request from a client on the client-server API in order to complete the call. -Querying profile information:: +There are several types of queries that can be made. The generic endpoint to +represent all queries is described first, followed by the more specific queries +that can be made. - Query type: profile - - Arguments: - user_id: the ID of the user whose profile to return - field: (optional) string giving a field name - - Returns: JSON object containing the following keys: - displayname: string of free-form text - avatar_url: string containing an HTTP-scheme URL - -If the query contains the optional ``field`` key, it should give the name of a -result field. If such is present, then the result should contain only a field -of that name, with no others present. If not, the result should contain as much -of the user's profile as the homeserver has available and can make public. - -Directory ---------- - -The server API for directory queries is also based on Federation Queries. - -{{directory_ss_http_api}} +{{query_ss_http_api}} Send-to-device messaging ------------------------ From d914c402e24e69536902703a305dbf4391c57303 Mon Sep 17 00:00:00 2001 From: Travis Ralston Date: Tue, 24 Jul 2018 16:03:35 -0600 Subject: [PATCH 04/16] Improve the profile query to have more sane responses This is a mix of Synapse and Dendrite behaviour, mostly Dendrite. Synapse returns `null` for field values that aren't set, however Dendrite just doesn't return them and instead opts for an empty object. Further, synapse is lacking in error codes in this area. Dendrite does some additional validation on this API which introduces more errors for bad requests, instead of defaulting to empty objects/200 OK responses. Likewise, Dendrite returns a 404 when the user is not found while Synapse returns 200 OK/empty object. --- api/server-server/query.yaml | 38 ++++++++++++++++++++++++------------ 1 file changed, 25 insertions(+), 13 deletions(-) diff --git a/api/server-server/query.yaml b/api/server-server/query.yaml index 23b9a575..4599d130 100644 --- a/api/server-server/query.yaml +++ b/api/server-server/query.yaml @@ -131,7 +131,8 @@ paths: x-example: "@someone:example.org" - in: query name: field - type: string + type: enum + enum: ['displayname', 'avatar_url'] description: |- The field to query. If specified, the server will only return the given field in the response. If not specified, the server will return the full profile for @@ -144,26 +145,22 @@ paths: the response should include the fields of the user's profile that can be made public, such as the display name and avatar. - If the ``field`` is for a field that the server does not recognize, an empty object - should be returned. If the ``field`` is recognized, but the user does not have it - set on their profile, ``null`` should be returned for the value of that field. - - If the user does not exist, an empty object should be returned regardless of the - ``field`` being queried. + If the user does not have a particular field set on their profile, the server + should exclude it from the response body or give it the value ``null``. schema: type: object properties: displayname: type: string description: |- - The display name of the user. If the user does not have a display name set, - this should be the value ``null``. + The display name of the user. May be omitted if the user does not have a + display name set. x-example: "John Doe" avatar_url: type: string description: |- - The avatar URL for the user's avatar. If the user does not have an avatar - set, this should be the value ``null``. + The avatar URL for the user's avatar. May be omitted if the user does not + have an avatar set. x-example: "mxc://matrix.org/MyC00lAvatar" examples: application/json: { @@ -171,11 +168,26 @@ paths: "avatar_url": "mxc://matrix.org/MyC00lAvatar" } 400: - description: The user does not belong to the server. + description: |- + The request was missing parameters or had invalid values for the parameters. This + can happen for: + + * The user not being hosted on the homeserver, + * Lack of a ``user_id`` in the request, or + * The ``field`` requested not being an allowed value. schema: $ref: "../client-server/definitions/errors/error.yaml" examples: application/json: { - "errcode": "M_UNKNOWN", + "errcode": "M_INVALID_ARGUMENT_VALUE", "error": "User is not hosted on this homeserver." } + 404: + description: The user does not exist or does not have a profile. + schema: + $ref: "../client-server/definitions/errors/error.yaml" + examples: + application/json: { + "errcode": "M_NOT_FOUND", + "error": "User does not exist." + } From 05bb7e105015fc0a07ba7e0b2506f49502fd6ec4 Mon Sep 17 00:00:00 2001 From: Travis Ralston Date: Thu, 2 Aug 2018 18:45:59 -0600 Subject: [PATCH 05/16] Spelling and word choice --- api/server-server/query.yaml | 6 +++--- specification/server_server_api.rst | 2 +- 2 files changed, 4 insertions(+), 4 deletions(-) diff --git a/api/server-server/query.yaml b/api/server-server/query.yaml index 4599d130..2f4b8d1e 100644 --- a/api/server-server/query.yaml +++ b/api/server-server/query.yaml @@ -52,7 +52,7 @@ paths: that belong to the target server (idenfified by the DNS Name in the alias). The target server may not appear in the resident servers list. - Servers may wish to cache the response to this query to prevent requesting the + Servers may wish to cache the response to this query to avoid requesting the information too often. operationId: queryRoomDirectory parameters: @@ -76,7 +76,7 @@ paths: servers: type: array description: |- - An array of server names that are likely to hold then given room. This + An array of server names that are likely to hold the given room. This list may or may not include the server answering the query. items: type: string @@ -120,7 +120,7 @@ paths: for a given user. Homeservers should only query profiles for users that belong to the target server (identified by the DNS Name in the user ID). - Servers may wish to cache the response to this query to prevent requesting the + Servers may wish to cache the response to this query to avoid requesting the information too often. parameters: - in: query diff --git a/specification/server_server_api.rst b/specification/server_server_api.rst index bfc98fb8..36591356 100644 --- a/specification/server_server_api.rst +++ b/specification/server_server_api.rst @@ -968,7 +968,7 @@ Rejecting a presence invite:: Querying for information ------------------------ -Queries are a way to retrieve information from a homeserver abotu a resource, +Queries are a way to retrieve information from a homeserver about a resource, such as a user or room. The endpoints here are often called in conjunction with a request from a client on the client-server API in order to complete the call. From 0b313dbdd9d1a2f098db4240a98429b9ec01c686 Mon Sep 17 00:00:00 2001 From: Travis Ralston Date: Thu, 2 Aug 2018 18:47:17 -0600 Subject: [PATCH 06/16] Don't explain what a 400 Bad Request is --- api/server-server/query.yaml | 29 +---------------------------- 1 file changed, 1 insertion(+), 28 deletions(-) diff --git a/api/server-server/query.yaml b/api/server-server/query.yaml index 2f4b8d1e..f569549e 100644 --- a/api/server-server/query.yaml +++ b/api/server-server/query.yaml @@ -49,8 +49,7 @@ paths: description: |- Performs a query to get the mapped room ID and list of resident homeservers in the room for a given room alias. Homeservers should only query room aliases - that belong to the target server (idenfified by the DNS Name in the alias). - The target server may not appear in the resident servers list. + that belong to the target server (identified by the DNS Name in the alias). Servers may wish to cache the response to this query to avoid requesting the information too often. @@ -92,17 +91,6 @@ paths: "another.example.com:8449", ] } - 400: - description: |- - The room alias is not hosted on the server. This can happen if the directory - server is named "example.org" and the room alias ends with "matrix.org". - schema: - $ref: "../client-server/definitions/errors/error.yaml" - examples: - application/json: { - "errcode": "M_UNKNOWN", - "error": "Room alias not hosted on this homeserver." - } 404: description: The room alias was not found. schema: @@ -167,21 +155,6 @@ paths: "displayname": "John Doe", "avatar_url": "mxc://matrix.org/MyC00lAvatar" } - 400: - description: |- - The request was missing parameters or had invalid values for the parameters. This - can happen for: - - * The user not being hosted on the homeserver, - * Lack of a ``user_id`` in the request, or - * The ``field`` requested not being an allowed value. - schema: - $ref: "../client-server/definitions/errors/error.yaml" - examples: - application/json: { - "errcode": "M_INVALID_ARGUMENT_VALUE", - "error": "User is not hosted on this homeserver." - } 404: description: The user does not exist or does not have a profile. schema: From 73958ecbffbf1ea996961e92d78e354ceb914200 Mon Sep 17 00:00:00 2001 From: Travis Ralston Date: Thu, 2 Aug 2018 16:41:24 -0600 Subject: [PATCH 07/16] Document /event_auth and /query_auth /event_auth is a fairly easy endpoint to determine the use case of. /query_auth is a little harder to investigate and has a fairly interesting purpose: it appears to be used for the sending server to admit defeat and shop around for the right auth chain, correcting it's own perspective as it goes. /query_auth is based off the following research points in synapse: * https://github.com/matrix-org/synapse/blob/43ecfe0b1028fea5e4dda197f5631aed67182ee6/synapse/handlers/federation.py#L1947-L1990 * https://github.com/matrix-org/synapse/blob/43ecfe0b1028fea5e4dda197f5631aed67182ee6/synapse/handlers/federation.py#L2049-L2187 * https://github.com/matrix-org/synapse/blob/43ecfe0b1028fea5e4dda197f5631aed67182ee6/synapse/handlers/federation.py#L1716-L1761 * https://github.com/matrix-org/synapse/blob/43ecfe0b1028fea5e4dda197f5631aed67182ee6/synapse/federation/federation_server.py#L393-L446 * https://github.com/matrix-org/synapse/blob/master/synapse/federation/transport/server.py#L482-L487 --- api/server-server/event_auth.yaml | 172 ++++++++++++++++++++++++++++ specification/server_server_api.rst | 9 ++ 2 files changed, 181 insertions(+) create mode 100644 api/server-server/event_auth.yaml diff --git a/api/server-server/event_auth.yaml b/api/server-server/event_auth.yaml new file mode 100644 index 00000000..77d6fb1e --- /dev/null +++ b/api/server-server/event_auth.yaml @@ -0,0 +1,172 @@ +# Copyright 2018 New Vector Ltd +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +swagger: '2.0' +info: + title: "Matrix Federation Event Authorization API" + version: "1.0.0" +host: localhost:8448 +schemes: + - https +basePath: /_matrix/federation/v1 +produces: + - application/json +paths: + "/event_auth/{roomId}/{eventId}": + get: + summary: Get the auth chain for a given event + description: |- + Retrieves the complete auth chain for a given event. + operationId: getEventAuth + parameters: + - in: path + name: roomId + type: string + description: The room ID to get the auth chain for. + required: true + x-example: "!abc123:matrix.org" + - in: path + name: eventId + type: string + description: The event ID to get the auth chain of. + required: true + x-example: "$helloworld:domain.com" + responses: + 200: + description: The auth chain for the event. + schema: + type: object + properties: + auth_chain: + type: array + description: |- + The full set of authorization events that make up the state of + the room, and their authorization events, recursively. + items: + $ref: "definitions/pdu.yaml" + example: [{"$ref": "examples/pdu.json"}] + required: ['auth_chain'] + "/query_auth/{roomId}/{eventId}": + post: + summary: Compare auth chains with the receiving server + description: |- + Compares the auth chain provided with what the receiving server has for the + room ID and event ID combination. + + The auth difference can be calculated in two parts, where the "remote auth" + is the auth chain provided by the sending server and the "local auth" is the + auth chain the receiving server has. With those lists, the algorithm works + bottom-up after sorting each chain by depth then by event ID. The differences + are then discovered and returned as the response to this API call. + operationId: compareEventAuth + parameters: + - in: path + name: roomId + type: string + description: The room ID to compare the auth chain in. + required: true + x-example: "!abc123:matrix.org" + - in: path + name: eventId + type: string + description: The event ID to compare the auth chain of. + required: true + x-example: "$helloworld:domain.com" + - in: body + name: body + schema: + type: object + properties: + auth_chain: + type: array + description: The auth chain (the "remote auth"). + items: + $ref: "definitions/pdu.yaml" + example: [{"$ref": "examples/pdu.json"}] + missing: + type: array + description: |- + A list of event IDs that the sender thinks the receiver is missing. + items: + type: string + example: [] + rejects: + type: object + description: |- + The set of events that the sending server has rejected from the provided + auth chain. + + The ``string`` key is the event ID that was rejected. + additionalProperties: + type: object + title: Rejection Reason + properties: + reason: + type: enum + enum: ['auth_error', 'replaced', 'not_ancestor'] + description: |- + The reason for the event being rejected. + required: ['reason'] + example: { + "$some_event:domain.com": { + "reason": "auth_error" + } + } + required: ['auth_chain'] + responses: + 200: + description: The auth chain differences, as determined by the receiver. + schema: + type: object + properties: + auth_chain: + type: array + description: |- + The auth chain the receiver has, and used to determine the auth + chain differences (the "local auth"). + items: + $ref: "definitions/pdu.yaml" + example: [{"$ref": "examples/pdu.json"}] + missing: + type: array + description: |- + The list of event IDs that the receiver believes it is missing, + after comparing the "remote auth" and "local auth" chains. + items: + type: string + example: ["$a_missing_event:domain.com"] + rejects: + type: object + description: |- + The set of events that the receiving server has rejected from the + auth chain, not including events that the sending server is missing + as determined from the difference algorithm. + + The ``string`` key is the event ID that was rejected. + additionalProperties: + type: object + title: Rejection Reason + properties: + reason: + type: enum + enum: ['auth_error', 'replaced', 'not_ancestor'] + description: |- + The reason for the event being rejected. + required: ['reason'] + example: { + "$some_event:domain.com": { + "reason": "auth_error" + } + } + required: ['auth_chain', 'missing', 'rejects'] diff --git a/specification/server_server_api.rst b/specification/server_server_api.rst index 01172c6e..bded4a49 100644 --- a/specification/server_server_api.rst +++ b/specification/server_server_api.rst @@ -440,6 +440,15 @@ The rules are as follows: I think there is some magic about 3pid invites too. +Retrieving event authorization information +++++++++++++++++++++++++++++++++++++++++++ + +The homeserver may be missing event authorization information, or wish to check +with other servers to ensure it is receiving the correct auth chain. These APIs +give the homeserver an avenue for getting the information it needs. + +{{event_auth_ss_http_api}} + EDUs ---- From 2ac80d38d7b38653bb0e97dfbe6f272fb902a03b Mon Sep 17 00:00:00 2001 From: Travis Ralston Date: Fri, 3 Aug 2018 11:23:44 -0600 Subject: [PATCH 08/16] Document the /publicRooms endpoint for federation This intentionally doesn't document the third party network aspect of the endpoint. This is scheduled for a later area for dealing with third party network/IDs and is reported as https://github.com/matrix-org/matrix-doc/issues/1476 The client-server response has been broken out to a shared file: both the client-server and server-server /publicRoom endpoints return the same thing, with slightly different inputs. The inputs (and behaviour) are based upon the docstring here: https://github.com/matrix-org/synapse/blob/43ecfe0b1028fea5e4dda197f5631aed67182ee6/synapse/federation/transport/server.py#L583-L612 --- .../definitions/public_rooms_response.yaml | 105 ++++++++++++++++++ api/client-server/list_public_rooms.yaml | 93 +--------------- api/server-server/public_rooms.yaml | 52 +++++++++ specification/server_server_api.rst | 10 ++ 4 files changed, 168 insertions(+), 92 deletions(-) create mode 100644 api/client-server/definitions/public_rooms_response.yaml create mode 100644 api/server-server/public_rooms.yaml diff --git a/api/client-server/definitions/public_rooms_response.yaml b/api/client-server/definitions/public_rooms_response.yaml new file mode 100644 index 00000000..fc6ccb44 --- /dev/null +++ b/api/client-server/definitions/public_rooms_response.yaml @@ -0,0 +1,105 @@ +# Copyright 2018 New Vector Ltd +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +type: object +description: A list of the rooms on the server. +required: ["chunk"] +properties: + chunk: + title: "PublicRoomsChunks" + type: array + description: |- + A paginated chunk of public rooms. + items: + type: object + title: "PublicRoomsChunk" + required: + - room_id + - num_joined_members + - world_readable + - guest_can_join + properties: + aliases: + type: array + description: |- + Aliases of the room. May be empty. + items: + type: string + canonical_alias: + type: string + description: |- + The canonical alias of the room, if any. + name: + type: string + description: |- + The name of the room, if any. + num_joined_members: + type: number + description: |- + The number of members joined to the room. + room_id: + type: string + description: |- + The ID of the room. + topic: + type: string + description: |- + The topic of the room, if any. + world_readable: + type: boolean + description: |- + Whether the room may be viewed by guest users without joining. + guest_can_join: + type: boolean + description: |- + Whether guest users may join the room and participate in it. + If they can, they will be subject to ordinary power level + rules like any other user. + avatar_url: + type: string + description: The URL for the room's avatar, if one is set. + next_batch: + type: string + description: |- + A pagination token for the response. The absence of this token + means there are no more results to fetch and the client should + stop paginating. + prev_batch: + type: string + description: |- + A pagination token that allows fetching previous results. The + absence of this token means there are no results before this + batch, i.e. this is the first batch. + total_room_count_estimate: + type: number + description: |- + An estimate on the total number of public rooms, if the + server has an estimate. +example: { + "chunk": [ + { + "aliases": ["#murrays:cheese.bar"], + "avatar_url": "mxc://bleeker.street/CHEDDARandBRIE", + "guest_can_join": false, + "name": "CHEESE", + "num_joined_members": 37, + "room_id": "!ol19s:bleecker.street", + "topic": "Tasty tasty cheese", + "world_readable": true + } + ], + "next_batch": "p190q", + "prev_batch": "p1902", + "total_room_count_estimate": 115 +} \ No newline at end of file diff --git a/api/client-server/list_public_rooms.yaml b/api/client-server/list_public_rooms.yaml index 45034488..72a12060 100644 --- a/api/client-server/list_public_rooms.yaml +++ b/api/client-server/list_public_rooms.yaml @@ -144,98 +144,7 @@ paths: 200: description: A list of the rooms on the server. schema: - type: object - description: A list of the rooms on the server. - required: ["chunk"] - properties: - chunk: - title: "PublicRoomsChunks" - type: array - description: |- - A paginated chunk of public rooms. - items: - type: object - title: "PublicRoomsChunk" - required: - - room_id - - num_joined_members - - world_readable - - guest_can_join - properties: - aliases: - type: array - description: |- - Aliases of the room. May be empty. - items: - type: string - canonical_alias: - type: string - description: |- - The canonical alias of the room, if any. - name: - type: string - description: |- - The name of the room, if any. - num_joined_members: - type: number - description: |- - The number of members joined to the room. - room_id: - type: string - description: |- - The ID of the room. - topic: - type: string - description: |- - The topic of the room, if any. - world_readable: - type: boolean - description: |- - Whether the room may be viewed by guest users without joining. - guest_can_join: - type: boolean - description: |- - Whether guest users may join the room and participate in it. - If they can, they will be subject to ordinary power level - rules like any other user. - avatar_url: - type: string - description: The URL for the room's avatar, if one is set. - next_batch: - type: string - description: |- - A pagination token for the response. The absence of this token - means there are no more results to fetch and the client should - stop paginating. - prev_batch: - type: string - description: |- - A pagination token that allows fetching previous results. The - absence of this token means there are no results before this - batch, i.e. this is the first batch. - total_room_count_estimate: - type: number - description: |- - An estimate on the total number of public rooms, if the - server has an estimate. - examples: - application/json: { - "chunk": [ - { - "aliases": ["#murrays:cheese.bar"], - "avatar_url": "mxc://bleeker.street/CHEDDARandBRIE", - "guest_can_join": false, - "name": "CHEESE", - "num_joined_members": 37, - "room_id": "!ol19s:bleecker.street", - "topic": "Tasty tasty cheese", - "world_readable": true - } - ], - "next_batch": "p190q", - "prev_batch": "p1902", - "total_room_count_estimate": 115 - } + $ref: "definitions/public_rooms_response.yaml" tags: - Room discovery post: diff --git a/api/server-server/public_rooms.yaml b/api/server-server/public_rooms.yaml new file mode 100644 index 00000000..6cd07449 --- /dev/null +++ b/api/server-server/public_rooms.yaml @@ -0,0 +1,52 @@ +# Copyright 2018 New Vector Ltd +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +swagger: '2.0' +info: + title: "Matrix Federation Public Rooms API" + version: "1.0.0" +host: localhost:8448 +schemes: + - https +basePath: /_matrix/federation/v1 +produces: + - application/json +paths: + "/publicRooms": + get: + summary: Get all the public rooms for a homeserver + description: |- + Gets all the public rooms for the homeserver. This should not return + rooms that are listed on another homeserver's directory, just those + listed on the receiving homeserver's directory. + operationId: getPublicRooms + parameters: + - in: query + name: limit + type: integer + description: |- + The maximum number of rooms to return. Defaults to 0 (no limit). + x-example: 10 + - in: query + name: since + type: string + description: |- + A pagination token from a previous call to this endpoint to fetch more + rooms. + x-example: "GetMoreRoomsTokenHere" + responses: + 200: + description: The public room list for the homeserver. + schema: + $ref: "../client-server/definitions/public_rooms_response.yaml" diff --git a/specification/server_server_api.rst b/specification/server_server_api.rst index 01172c6e..e058ca68 100644 --- a/specification/server_server_api.rst +++ b/specification/server_server_api.rst @@ -893,6 +893,16 @@ A homeserver may provide a TLS client certificate and the receiving homeserver may check that the client certificate matches the certificate of the origin homeserver. +Public Room Directory +--------------------- + +To compliment the `Client-Server API`_'s room directory, homeservers need a +way to query the public rooms for another server. This can be done by making +a request to the ``/publicRooms`` endpoint for the server the room directory +should be retrieved for. + +{{public_rooms_ss_http_api}} + Presence -------- From 5fbaa1deb564fa39728e1e9628f92a976a7d6188 Mon Sep 17 00:00:00 2001 From: Travis Ralston Date: Fri, 3 Aug 2018 13:10:24 -0600 Subject: [PATCH 09/16] Federation: Document the Host header; Clarify how literal IP addresses are handled Fixes https://github.com/matrix-org/matrix-doc/issues/1161 The IP address clarification is to add an explicit mention of how to handle the case. The prior documentation assumed that all servers would be resolvable with DNS, and does technically have a fallback to use the fallback port, however making it clear feels like a good idea. --- specification/server_server_api.rst | 9 +++++++++ 1 file changed, 9 insertions(+) diff --git a/specification/server_server_api.rst b/specification/server_server_api.rst index 01172c6e..b2ff9c6e 100644 --- a/specification/server_server_api.rst +++ b/specification/server_server_api.rst @@ -98,6 +98,15 @@ default fallback port number of 8448. Homeservers may use SRV records to load balance requests between multiple TLS endpoints or to failover to another endpoint if an endpoint fails. +If the DNS name is a literal IP address, the port specified or the fallback +port should be used. + +When making requests to servers, use the DNS name of the target server in the +``Host`` header, regardless of the host given in the SRV record. For example, +if making a request to ``example.org``, and the SRV record resolves to ``matrix. +example.org``, the ``Host`` header in the request should be ``example.org``. The +port number for target server should not appear in the ``Host`` header. + Server implementation ~~~~~~~~~~~~~~~~~~~~~~ From 13a2b54bf934c30b84b7caf7c1c3d003dbe9a7b2 Mon Sep 17 00:00:00 2001 From: Travis Ralston Date: Fri, 3 Aug 2018 13:57:43 -0600 Subject: [PATCH 10/16] query_auth consumes JSON --- api/server-server/event_auth.yaml | 2 ++ 1 file changed, 2 insertions(+) diff --git a/api/server-server/event_auth.yaml b/api/server-server/event_auth.yaml index 77d6fb1e..f55afddc 100644 --- a/api/server-server/event_auth.yaml +++ b/api/server-server/event_auth.yaml @@ -20,6 +20,8 @@ host: localhost:8448 schemes: - https basePath: /_matrix/federation/v1 +consumes: + - application/json produces: - application/json paths: From 2e6d6e27908a31e54e4bb411dbf58ba01e52495b Mon Sep 17 00:00:00 2001 From: Travis Ralston Date: Fri, 3 Aug 2018 14:01:33 -0600 Subject: [PATCH 11/16] Specify which swagger APIs consume JSON --- api/server-server/joins.yaml | 2 ++ api/server-server/keys_query.yaml | 2 ++ api/server-server/leaving.yaml | 2 ++ api/server-server/third_party_invite.yaml | 2 ++ api/server-server/transactions.yaml | 2 ++ 5 files changed, 10 insertions(+) diff --git a/api/server-server/joins.yaml b/api/server-server/joins.yaml index 759361b7..14142945 100644 --- a/api/server-server/joins.yaml +++ b/api/server-server/joins.yaml @@ -20,6 +20,8 @@ host: localhost:8448 schemes: - https basePath: /_matrix/federation/v1 +consumes: + - application/json produces: - application/json paths: diff --git a/api/server-server/keys_query.yaml b/api/server-server/keys_query.yaml index 8fbe00dc..e616915b 100644 --- a/api/server-server/keys_query.yaml +++ b/api/server-server/keys_query.yaml @@ -20,6 +20,8 @@ host: localhost:8448 schemes: - https basePath: /_matrix/key/v2 +consumes: + - application/json produces: - application/json paths: diff --git a/api/server-server/leaving.yaml b/api/server-server/leaving.yaml index e287bf58..28bcf42c 100644 --- a/api/server-server/leaving.yaml +++ b/api/server-server/leaving.yaml @@ -20,6 +20,8 @@ host: localhost:8448 schemes: - https basePath: /_matrix/federation/v1 +consumes: + - application/json produces: - application/json paths: diff --git a/api/server-server/third_party_invite.yaml b/api/server-server/third_party_invite.yaml index 42bdae23..754a3282 100644 --- a/api/server-server/third_party_invite.yaml +++ b/api/server-server/third_party_invite.yaml @@ -20,6 +20,8 @@ host: localhost:8448 schemes: - https basePath: /_matrix/federation/v1 +consumes: + - application/json produces: - application/json paths: diff --git a/api/server-server/transactions.yaml b/api/server-server/transactions.yaml index 2a9180a1..4f4c6b28 100644 --- a/api/server-server/transactions.yaml +++ b/api/server-server/transactions.yaml @@ -20,6 +20,8 @@ host: localhost:8448 schemes: - https basePath: /_matrix/federation/v1 +consumes: + - application/json produces: - application/json paths: From 7a46bdae02b841dfd3f400b4382064a1c178a5b9 Mon Sep 17 00:00:00 2001 From: Will Hunt Date: Sun, 5 Aug 2018 14:43:05 +0100 Subject: [PATCH 12/16] Spacing --- specification/server_server_api.rst | 1 + 1 file changed, 1 insertion(+) diff --git a/specification/server_server_api.rst b/specification/server_server_api.rst index 653d75ed..38b25328 100644 --- a/specification/server_server_api.rst +++ b/specification/server_server_api.rst @@ -139,6 +139,7 @@ certificate fingerprints to validate any connection made to the homeserver. {{keys_server_ss_http_api}} + Querying Keys Through Another Server ++++++++++++++++++++++++++++++++++++ From c826edf23b4ae77e29189a54e9ab3109fa5f3ab3 Mon Sep 17 00:00:00 2001 From: Will Hunt Date: Sun, 5 Aug 2018 14:46:42 +0100 Subject: [PATCH 13/16] Remove dns_name from S2S Api --- specification/server_server_api.rst | 1 - 1 file changed, 1 deletion(-) diff --git a/specification/server_server_api.rst b/specification/server_server_api.rst index 38b25328..30cf45ab 100644 --- a/specification/server_server_api.rst +++ b/specification/server_server_api.rst @@ -84,7 +84,6 @@ and an optional TLS port. .. code:: server_name = hostname [ ":" tls_port] - dns_name = tls_port = *DIGIT .. ** From dcb39f8890af57e62ed37f9e894af4b5af5d9bc3 Mon Sep 17 00:00:00 2001 From: Travis Ralston Date: Sun, 5 Aug 2018 10:55:55 -0600 Subject: [PATCH 14/16] Fix wording from bad merge --- specification/server_server_api.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/specification/server_server_api.rst b/specification/server_server_api.rst index 30cf45ab..cf3333a1 100644 --- a/specification/server_server_api.rst +++ b/specification/server_server_api.rst @@ -629,7 +629,7 @@ if this optimisation fails. Once the joining server has the room ID and the join candidates, it then needs to obtain enough information about the room to fill in the required fields of -the ``m.room.member`` event. It obtains this by selecting a resident from t +the ``m.room.member`` event. It obtains this by selecting a resident from the candidate list, and using the ``GET /make_join`` endpoint. The resident server will then reply with enough information for the joining server to fill in the event. From 803ef536ba64178b6468084c24aa9cddc7835017 Mon Sep 17 00:00:00 2001 From: David Baker Date: Mon, 6 Aug 2018 15:58:30 +0100 Subject: [PATCH 15/16] Fix grammar in room tag spec Have removed the second clause about how the client interprets them: I was trying to think how to rephrase it but I think in reality it's probably just redundant. --- specification/modules/tags.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/specification/modules/tags.rst b/specification/modules/tags.rst index 8c74c55f..f965c2e8 100644 --- a/specification/modules/tags.rst +++ b/specification/modules/tags.rst @@ -41,7 +41,7 @@ after the rooms with that tag that have an ``order`` key. The name of a tag MUST not exceed 255 bytes. -Tags namespaces are defined in the following way, depending on how the client are expected to interpret them: +The tag namespace is defined as follows: * The namespace ``m.*`` is reserved for tags defined in the Matrix specification. Clients must ignore any tags in this namespace they don't understand. From e7cebf670eb5e8beb45d3f8289687b7de3b1abca Mon Sep 17 00:00:00 2001 From: Florian Jacob Date: Mon, 6 Aug 2018 22:56:28 +0200 Subject: [PATCH 16/16] Fix minor spelling mistake from #1472 Signed-off-by: Florian Jacob --- specification/server_server_api.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/specification/server_server_api.rst b/specification/server_server_api.rst index cf3333a1..bd370149 100644 --- a/specification/server_server_api.rst +++ b/specification/server_server_api.rst @@ -693,7 +693,7 @@ that requested by the requester in the ``v`` parameter). Specify (or remark that it is unspecified) how the server handles divergent history. DFS? BFS? Anything weirder? -Retriving events +Retrieving events ---------------- In some circumstances, a homeserver may be missing a particular event or information