From 7f877606555c005e354e4ebdbc2a08206980b34f Mon Sep 17 00:00:00 2001 From: Daniel Wagner-Hall Date: Wed, 9 Dec 2015 16:45:56 +0000 Subject: [PATCH 01/14] Add links to old releases --- specification/index.rst | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/specification/index.rst b/specification/index.rst index d912eb64..1da013fe 100644 --- a/specification/index.rst +++ b/specification/index.rst @@ -26,6 +26,12 @@ Any developments since the latest release can be found `here`__. .. __: https://matrix.org/speculator/spec/head/ +Old releases of the spec: + +- Client-Server API: `r0.0.0`__ + +.. __: https://matrix.org/docs/spec/r0.0.0 + Before we formally started releasing the specification, the last working copy we had can be found `here`__. From dbba737135b063f3b00afb9d5fcece0698225b36 Mon Sep 17 00:00:00 2001 From: Daniel Wagner-Hall Date: Thu, 10 Dec 2015 15:47:27 +0000 Subject: [PATCH 02/14] Explain scope of uniqueness of txnid --- api/client-server/room_send.yaml | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/api/client-server/room_send.yaml b/api/client-server/room_send.yaml index 70d005c2..b7ccbcfb 100644 --- a/api/client-server/room_send.yaml +++ b/api/client-server/room_send.yaml @@ -48,8 +48,9 @@ paths: name: txnId type: string description: |- - The transaction ID for this event. Clients should generate a - unique ID; it will be used by the server to ensure idempotency of requests. + The transaction ID for this event. Clients should generate an + ID unique across requests with the same access token; it will be + used by the server to ensure idempotency of requests. required: true x-example: "35" - in: body From b9bee6ee733c89ff1401873bb7e331ffe3072e6d Mon Sep 17 00:00:00 2001 From: "Paul \"LeoNerd\" Evans" Date: Mon, 14 Dec 2015 20:00:49 +0000 Subject: [PATCH 03/14] Initial hack at some federation /backfill documentation --- specification/server_server_api.rst | 40 +++++++++++++++++++++++++---- 1 file changed, 35 insertions(+), 5 deletions(-) diff --git a/specification/server_server_api.rst b/specification/server_server_api.rst index 033801f0..97f446a4 100644 --- a/specification/server_server_api.rst +++ b/specification/server_server_api.rst @@ -512,7 +512,6 @@ To backfill events on a given room:: Retrieves a sliding-window history of previous PDUs that occurred on the given room. Starting from the PDU ID(s) given in the "v" argument, the PDUs that preceded it are retrieved, up to a total number given by the "limit" argument. -These are then returned in a new Transaction containing all of the PDUs. To stream events all the events:: @@ -690,11 +689,42 @@ following keys: Backfilling ----------- -.. NOTE:: - This section is a work in progress. -.. TODO-doc - - What it is, when is it used, how is it done +Once a homeserver has joined a room, it receives all the events emitted by +other homeservers in that room, and is thus aware of the entire history of the +room from that moment onwards. Since users in that room are able to request the +history by the ``/messages`` client API endpoint, it's possible that they might +step backwards far enough into history before the homeserver itself was a +member of that room. + +To cover this case, the federation API provides a server-to-server analog of +the ``/messages`` client API, allowing one homeserver to fetch history from +another. This is the ``/backfill`` API. + +To request more history, the requesting homeserver picks another homeserver +that it thinks may have more (most likely this should be a homeserver for some +of the existing users in the room at the earliest point in history it has +currently), and makes a ``/backfill`` request. The parameters of this request +give an event ID that the requesting homeserver wishes to obtain, and a number +specifying how many more events of history before that one to return at most. + +The response to this request is an object with the following keys: + +==================== ======== ============ + Key Type Description +==================== ======== ============ +``pdus`` List A list of events +``origin`` String The name of the resident homeserver +``origin_server_ts`` Integer A timestamp added by the resident homeserver +==================== ======== ============ + +The list of events given in ``pdus`` is returned in reverse chronological +order; having the most recent event first (i.e. the event whose event ID is +that requested by the requestor in the ``v`` parameter). + +.. TODO-spec + Specify (or remark that it is unspecified) how the server handles divergent + history. DFS? BFS? Anything weirder? Authentication From b7c255a887de1166c88ce27f6b73710a9616ad17 Mon Sep 17 00:00:00 2001 From: Richard van der Hoff Date: Thu, 17 Dec 2015 13:40:22 +0000 Subject: [PATCH 04/14] Tweaks/clarifications to search spec Fixes https://matrix.org/jira/browse/SPEC-305 --- api/client-server/search.yaml | 20 ++++++++++++++++---- specification/modules/search.rst | 10 ++++------ 2 files changed, 20 insertions(+), 10 deletions(-) diff --git a/api/client-server/search.yaml b/api/client-server/search.yaml index 0bf7ffd9..0ddbfe0a 100644 --- a/api/client-server/search.yaml +++ b/api/client-server/search.yaml @@ -20,12 +20,19 @@ securityDefinitions: paths: "/search": post: - summary: Search server side for things. + summary: Perform a server-side search. description: |- Performs a full text search across different categories. security: - accessToken: [] parameters: + - in: query + name: next_batch + type: string + description: |- + The point to return events from. If given, this should be a + `next_batch` result from a previous call to this endpoint. + x-example: "YWxsCgpOb25lLDM1ODcwOA" - in: body name: body schema: @@ -260,8 +267,11 @@ paths: type: string title: Next Batch in Group description: |- - Token that can be used to get the next - batch of results in the group, if exists. + Token that can be used to get the next batch + of results in the group, by passing as the + `next_batch` parameter to the next call. If + this field is absent, there are no more + results in this group. order: type: integer title: Group Order @@ -280,7 +290,9 @@ paths: title: Next Batch description: |- Token that can be used to get the next batch of - results, if exists. + results, by passing as the `next_batch` parameter to + the next call. If this field is absent, there are no + more results. examples: application/json: |- { diff --git a/specification/modules/search.rst b/specification/modules/search.rst index f795e18a..def21877 100644 --- a/specification/modules/search.rst +++ b/specification/modules/search.rst @@ -8,8 +8,6 @@ rooms that the user has been in, including those that they have left. Only events that the user is allowed to see will be searched, e.g. it won't include events in rooms that happened after you left. -Client behaviour ----------------- {{search_http_api}} Search Categories @@ -36,9 +34,9 @@ The search will *not* include rooms that are end to end encrypted. The results include a ``rank`` key that can be used to sort the results by relevancy. The higher the ``rank`` the more relevant the result is. -The value of ``count`` may not match the number of results. For example due to -the search query matching 1000s of results and the server truncating the -response. +The value of ``count`` gives an approximation of the total number of +results. Homeservers may give an estimate rather than an exact value for this +field. Ordering -------- @@ -84,7 +82,7 @@ The currently supported locations for the ``next_batch`` token are: - ``search_categories..groups...next_batch`` A server need not support pagination, even if there are more matching results. -In which case they must not return a ``next_batch`` token in the response. +In that case, they must not return a ``next_batch`` token in the response. Security considerations From e4dd346a81c0dd9648637e0df461d882e5c2bf88 Mon Sep 17 00:00:00 2001 From: Richard van der Hoff Date: Thu, 17 Dec 2015 14:07:35 +0000 Subject: [PATCH 05/14] Restore 'client behaviour' section Because it's the standard --- specification/modules/search.rst | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/specification/modules/search.rst b/specification/modules/search.rst index def21877..bb8f0141 100644 --- a/specification/modules/search.rst +++ b/specification/modules/search.rst @@ -8,6 +8,10 @@ rooms that the user has been in, including those that they have left. Only events that the user is allowed to see will be searched, e.g. it won't include events in rooms that happened after you left. +Client behaviour +---------------- +There is a single HTTP API for performing server-side search, documented below. + {{search_http_api}} Search Categories From 9bfe86f474066182f3b3fac2db543bf19af3b9b0 Mon Sep 17 00:00:00 2001 From: Daniel Wagner-Hall Date: Thu, 17 Dec 2015 19:01:20 +0100 Subject: [PATCH 06/14] Add display_name to 3pid invites in m.room.member --- event-schemas/examples/m.room.member#third_party_invite | 1 + event-schemas/schema/m.room.member | 6 +++++- 2 files changed, 6 insertions(+), 1 deletion(-) diff --git a/event-schemas/examples/m.room.member#third_party_invite b/event-schemas/examples/m.room.member#third_party_invite index 1bae2e9f..61718ba1 100644 --- a/event-schemas/examples/m.room.member#third_party_invite +++ b/event-schemas/examples/m.room.member#third_party_invite @@ -5,6 +5,7 @@ "avatar_url": "mxc://localhost/SEsfnsuifSDFSSEF#auto", "displayname": "Alice Margatroid", "third_party_invite": { + "display_name": "alice", "signed": { "mxid": "@alice:localhost", "signatures": { diff --git a/event-schemas/schema/m.room.member b/event-schemas/schema/m.room.member index 28a49c3a..0ffc7fd4 100644 --- a/event-schemas/schema/m.room.member +++ b/event-schemas/schema/m.room.member @@ -27,6 +27,10 @@ "type": "object", "title": "Invite", "properties": { + "display_name": { + "type": "string", + "description": "A name which can be displayed to represent the user instead of their third party identifier" + }, "signed": { "type": "object", "title": "signed", @@ -49,7 +53,7 @@ "required": ["mxid", "signatures", "token"] } }, - "required": ["signed"] + "required": ["display_name", "signed"] } }, "required": ["membership"] From 116253b2f5722c0b42e70d406e4564886afc69eb Mon Sep 17 00:00:00 2001 From: "Paul \"LeoNerd\" Evans" Date: Thu, 17 Dec 2015 20:45:57 +0000 Subject: [PATCH 07/14] Document the /event/:event_id/ federation endpoint --- specification/server_server_api.rst | 11 +++++++++++ 1 file changed, 11 insertions(+) diff --git a/specification/server_server_api.rst b/specification/server_server_api.rst index 97f446a4..2365505e 100644 --- a/specification/server_server_api.rst +++ b/specification/server_server_api.rst @@ -503,6 +503,17 @@ Retrieves a snapshot of the entire current state of the given room. The response will contain a single Transaction, inside which will be a list of PDUs that encode the state. + +To fetch a particular event:: + + GET .../event// + Response: JSON encoding of a partial Transaction containing the event + +Retrieves a single event. The response will contain a partial Transaction, +having just the ``origin``, ``origin_server_ts`` and ``pdus`` fields; the +event will be encoded as the only PDU in the ``pdus`` list. + + To backfill events on a given room:: GET .../backfill// From 1f876778b508030fed025f54a30a9bc9b366f867 Mon Sep 17 00:00:00 2001 From: "Paul \"LeoNerd\" Evans" Date: Mon, 21 Dec 2015 16:50:56 +0000 Subject: [PATCH 08/14] Don't need to document /pdu/ any more --- specification/server_server_api.rst | 9 --------- 1 file changed, 9 deletions(-) diff --git a/specification/server_server_api.rst b/specification/server_server_api.rst index 2365505e..66e50a1f 100644 --- a/specification/server_server_api.rst +++ b/specification/server_server_api.rst @@ -485,15 +485,6 @@ The destination name will be set to that of the receiving server itself. Each embedded PDU in the transaction body will be processed. -To fetch a particular PDU:: - - GET .../pdu/// - Response: JSON encoding of a single Transaction containing one PDU - -Retrieves a given PDU from the server. The response will contain a single new -Transaction, inside which will be the requested PDU. - - To fetch all the state of a given room:: GET .../state// From c9349b302aecdda5ba833010b9009369a75ef7d3 Mon Sep 17 00:00:00 2001 From: Mark Haines Date: Mon, 21 Dec 2015 17:17:28 +0000 Subject: [PATCH 09/14] Filters don't support wildcards for senders or room ids --- api/client-server/definitions/event_filter.yaml | 5 ++--- api/client-server/definitions/room_event_filter.yaml | 4 ++-- 2 files changed, 4 insertions(+), 5 deletions(-) diff --git a/api/client-server/definitions/event_filter.yaml b/api/client-server/definitions/event_filter.yaml index 23a2f11c..27cce87d 100644 --- a/api/client-server/definitions/event_filter.yaml +++ b/api/client-server/definitions/event_filter.yaml @@ -5,7 +5,7 @@ properties: not_senders: description: A list of sender IDs to exclude. If this list is absent then no senders are excluded. A matching sender will be excluded even if it is listed in the - 'senders' filter. A '*' can be used as a wildcard to match any sequence of characters. + 'senders' filter. items: type: string type: array @@ -19,8 +19,7 @@ properties: type: array senders: description: A list of senders IDs to include. If this list is absent then all - senders are included. A '*' can be used as a wildcard to match any sequence - of characters. + senders are included. items: type: string type: array diff --git a/api/client-server/definitions/room_event_filter.yaml b/api/client-server/definitions/room_event_filter.yaml index 4826693d..628e2ca8 100644 --- a/api/client-server/definitions/room_event_filter.yaml +++ b/api/client-server/definitions/room_event_filter.yaml @@ -4,13 +4,13 @@ properties: not_rooms: description: A list of room IDs to exclude. If this list is absent then no rooms are excluded. A matching room will be excluded even if it is listed in the 'rooms' - filter. A '*' can be used as a wildcard to match any sequence of characters. + filter. items: type: string type: array rooms: description: A list of room IDs to include. If this list is absent then all rooms - are included. A '*' can be used as a wildcard to match any sequence of characters. + are included. items: type: string type: array From a32ed96a0ff973ddddc316dd3ab0efb446e73a8f Mon Sep 17 00:00:00 2001 From: Mark Haines Date: Mon, 21 Dec 2015 20:31:02 +0000 Subject: [PATCH 10/14] Document filtering for account data --- api/client-server/definitions/sync_filter.yaml | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/api/client-server/definitions/sync_filter.yaml b/api/client-server/definitions/sync_filter.yaml index 4aa78e77..fc154f3a 100644 --- a/api/client-server/definitions/sync_filter.yaml +++ b/api/client-server/definitions/sync_filter.yaml @@ -20,6 +20,10 @@ properties: allOf: - $ref: event_filter.yaml description: The presence updates to include. + account_data: + allOf: + - $ref: event_filter.yaml + description: The user account data that isn't associated with rooms to include. room: properties: ephemeral: @@ -38,5 +42,9 @@ properties: allOf: - $ref: room_event_filter.yaml description: The message and state update events to include for rooms. + account_data: + allOf: + - $ref: room_event_filter.yaml + description: The per user account data to include for rooms. type: object type: object From 288c7ed35381de91c598ea82fed3f7a892b315d1 Mon Sep 17 00:00:00 2001 From: Mark Haines Date: Mon, 21 Dec 2015 20:44:06 +0000 Subject: [PATCH 11/14] Add top level filters for filtering by room id. --- api/client-server/definitions/sync_filter.yaml | 15 +++++++++++++++ 1 file changed, 15 insertions(+) diff --git a/api/client-server/definitions/sync_filter.yaml b/api/client-server/definitions/sync_filter.yaml index fc154f3a..1691ca31 100644 --- a/api/client-server/definitions/sync_filter.yaml +++ b/api/client-server/definitions/sync_filter.yaml @@ -26,6 +26,21 @@ properties: description: The user account data that isn't associated with rooms to include. room: properties: + not_rooms: + description: A list of room IDs to exclude. If this list is absent then no rooms + are excluded. A matching room will be excluded even if it is listed in the 'rooms' + filter. This filter is applied before the filters in ``ephemeral``, + ``state``, ``timeline`` or ``account_data`` + items: + type: string + type: array + rooms: + description: A list of room IDs to include. If this list is absent then all rooms + are included. This filter is applied before the filters in ``ephemeral``, + ``state``, ``timeline`` or ``account_data`` + items: + type: string + type: array ephemeral: allOf: - $ref: room_event_filter.yaml From 9b4a37f57b192fa962ae6722b9a9aa6d790fd4fc Mon Sep 17 00:00:00 2001 From: Mark Haines Date: Tue, 22 Dec 2015 11:44:05 +0000 Subject: [PATCH 12/14] backticks --- api/client-server/definitions/event_filter.yaml | 6 +++--- api/client-server/definitions/sync_filter.yaml | 2 +- 2 files changed, 4 insertions(+), 4 deletions(-) diff --git a/api/client-server/definitions/event_filter.yaml b/api/client-server/definitions/event_filter.yaml index 27cce87d..8bedacb2 100644 --- a/api/client-server/definitions/event_filter.yaml +++ b/api/client-server/definitions/event_filter.yaml @@ -5,14 +5,14 @@ properties: not_senders: description: A list of sender IDs to exclude. If this list is absent then no senders are excluded. A matching sender will be excluded even if it is listed in the - 'senders' filter. + ``'senders'`` filter. items: type: string type: array not_types: description: A list of event types to exclude. If this list is absent then no event types are excluded. A matching type will be excluded even if it is listed - in the 'types' filter. A '*' can be used as a wildcard to match any sequence + in the ``'types'`` filter. A '*' can be used as a wildcard to match any sequence of characters. items: type: string @@ -25,7 +25,7 @@ properties: type: array types: description: A list of event types to include. If this list is absent then all - event types are included. A '*' can be used as a wildcard to match any sequence + event types are included. A ``'*'`` can be used as a wildcard to match any sequence of characters. items: type: string diff --git a/api/client-server/definitions/sync_filter.yaml b/api/client-server/definitions/sync_filter.yaml index 1691ca31..09307c99 100644 --- a/api/client-server/definitions/sync_filter.yaml +++ b/api/client-server/definitions/sync_filter.yaml @@ -28,7 +28,7 @@ properties: properties: not_rooms: description: A list of room IDs to exclude. If this list is absent then no rooms - are excluded. A matching room will be excluded even if it is listed in the 'rooms' + are excluded. A matching room will be excluded even if it is listed in the ``'rooms'`` filter. This filter is applied before the filters in ``ephemeral``, ``state``, ``timeline`` or ``account_data`` items: From 4c4393997741911d88c4d53e04bb49b5cf624f79 Mon Sep 17 00:00:00 2001 From: Mark Haines Date: Tue, 22 Dec 2015 11:48:11 +0000 Subject: [PATCH 13/14] more backticks --- api/client-server/definitions/room_event_filter.yaml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/api/client-server/definitions/room_event_filter.yaml b/api/client-server/definitions/room_event_filter.yaml index 628e2ca8..0ad1ae33 100644 --- a/api/client-server/definitions/room_event_filter.yaml +++ b/api/client-server/definitions/room_event_filter.yaml @@ -3,7 +3,7 @@ allOf: properties: not_rooms: description: A list of room IDs to exclude. If this list is absent then no rooms - are excluded. A matching room will be excluded even if it is listed in the 'rooms' + are excluded. A matching room will be excluded even if it is listed in the ``'rooms'`` filter. items: type: string From 52bb20b1425dbc3d8fa6096aa91f9b800faa9402 Mon Sep 17 00:00:00 2001 From: Mark Haines Date: Tue, 22 Dec 2015 11:52:19 +0000 Subject: [PATCH 14/14] Document filtering for account data --- api/client-server/definitions/sync_filter.yaml | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/api/client-server/definitions/sync_filter.yaml b/api/client-server/definitions/sync_filter.yaml index 4aa78e77..fc154f3a 100644 --- a/api/client-server/definitions/sync_filter.yaml +++ b/api/client-server/definitions/sync_filter.yaml @@ -20,6 +20,10 @@ properties: allOf: - $ref: event_filter.yaml description: The presence updates to include. + account_data: + allOf: + - $ref: event_filter.yaml + description: The user account data that isn't associated with rooms to include. room: properties: ephemeral: @@ -38,5 +42,9 @@ properties: allOf: - $ref: room_event_filter.yaml description: The message and state update events to include for rooms. + account_data: + allOf: + - $ref: room_event_filter.yaml + description: The per user account data to include for rooms. type: object type: object