From 8fd5b15594ebe41f15334078507edc956c3bc8fb Mon Sep 17 00:00:00 2001 From: Travis Ralston Date: Wed, 5 Jun 2019 22:55:11 -0600 Subject: [PATCH 1/4] Reorganize event structure in c2s spec and clarify event capabilities Fixes https://github.com/matrix-org/matrix-doc/issues/1166 Fixes https://github.com/matrix-org/matrix-doc/issues/1527 Fixes https://github.com/matrix-org/matrix-doc/issues/1827 Note: In order to fix the "state events have the following fields: [no words]" bug (1827) we need to resolve references on common event types. When doing this we ultimately end up with more fields than may be required to explain the section, however this commit alters the section descriptions to just say "these fields" instead of "these additional fields". This is also preferable over trying to get the inheritance reversed in the common event types, as the `/sync` endpoint has a high amount of reliance on partial events definitions. --- .../newsfragments/2087.clarification | 1 + .../schema/core-event-schema/room_event.yaml | 3 +- .../schema/core-event-schema/state_event.yaml | 3 +- scripts/templating/matrix_templates/units.py | 1 + specification/client_server_api.rst | 57 +++++++++++++++ specification/events.rst | 73 ------------------- specification/index.rst | 5 ++ specification/targets.yaml | 1 - 8 files changed, 66 insertions(+), 78 deletions(-) create mode 100644 changelogs/client_server/newsfragments/2087.clarification delete mode 100644 specification/events.rst diff --git a/changelogs/client_server/newsfragments/2087.clarification b/changelogs/client_server/newsfragments/2087.clarification new file mode 100644 index 00000000..1974127d --- /dev/null +++ b/changelogs/client_server/newsfragments/2087.clarification @@ -0,0 +1 @@ +Reorganize information about events into a common section. diff --git a/event-schemas/schema/core-event-schema/room_event.yaml b/event-schemas/schema/core-event-schema/room_event.yaml index 007372a5..231d5c65 100644 --- a/event-schemas/schema/core-event-schema/room_event.yaml +++ b/event-schemas/schema/core-event-schema/room_event.yaml @@ -1,7 +1,6 @@ allOf: - $ref: sync_room_event.yaml -description: In addition to the Event fields, Room Events have the following additional - fields. +description: Room Events have the following fields. properties: room_id: description: |- diff --git a/event-schemas/schema/core-event-schema/state_event.yaml b/event-schemas/schema/core-event-schema/state_event.yaml index 37d4426f..816f925f 100644 --- a/event-schemas/schema/core-event-schema/state_event.yaml +++ b/event-schemas/schema/core-event-schema/state_event.yaml @@ -1,7 +1,6 @@ allOf: - $ref: room_event.yaml - $ref: sync_state_event.yaml -description: In addition to the Room Event fields, State Events have the following - additional fields. +description: State Events have the following fields. title: State Event type: object diff --git a/scripts/templating/matrix_templates/units.py b/scripts/templating/matrix_templates/units.py index ddb65efe..d44fb9c8 100644 --- a/scripts/templating/matrix_templates/units.py +++ b/scripts/templating/matrix_templates/units.py @@ -748,6 +748,7 @@ class MatrixUnits(Units): with open(filepath, encoding="utf-8") as f: event_schema = yaml.load(f, OrderedLoader) + event_schema = resolve_references(filepath, event_schema) schema_info = process_data_type( event_schema, diff --git a/specification/client_server_api.rst b/specification/client_server_api.rst index d9342a5b..80e4a2f4 100644 --- a/specification/client_server_api.rst +++ b/specification/client_server_api.rst @@ -1326,6 +1326,63 @@ the event ``type`` key SHOULD follow the Java package naming convention, e.g. ``com.example.myapp.event``. This ensures event types are suitably namespaced for each application and reduces the risk of clashes. +.. Note:: + Events are not limited to the types defined in this specification. New or custom + event types can be created on a whim using the Java package naming convention. + For example, a ``com.example.game.score`` event can be sent by clients and other + clients would receive it through Matrix. + +Note that the structure of these events may be different than those in the +server-server API. + +{{common_event_fields}} + +{{common_room_event_fields}} + +{{common_state_event_fields}} + + +Size limits +~~~~~~~~~~~ + +The complete event MUST NOT be larger than 65535 bytes, when formatted as a +`PDU for the Server-Server protocol <../server_server/%SERVER_RELEASE_LABEL%#pdus>`_, +including any signatures, and encoded as `Canonical JSON`_. + +There are additional restrictions on sizes per key: + +- ``sender`` MUST NOT exceed 255 bytes (including domain). +- ``room_id`` MUST NOT exceed 255 bytes. +- ``state_key`` MUST NOT exceed 255 bytes. +- ``type`` MUST NOT exceed 255 bytes. +- ``event_id`` MUST NOT exceed 255 bytes. + +Some event types have additional size restrictions which are specified in +the description of the event. Additional keys have no limit other than that +implied by the total 65 KB limit on events. + +Room Events +~~~~~~~~~~~ +.. NOTE:: + This section is a work in progress. + +This specification outlines several standard event types, all of which are +prefixed with ``m.`` + +{{m_room_aliases_event}} + +{{m_room_canonical_alias_event}} + +{{m_room_create_event}} + +{{m_room_join_rules_event}} + +{{m_room_member_event}} + +{{m_room_power_levels_event}} + +{{m_room_redaction_event}} + Syncing ~~~~~~~ diff --git a/specification/events.rst b/specification/events.rst deleted file mode 100644 index c5e4a288..00000000 --- a/specification/events.rst +++ /dev/null @@ -1,73 +0,0 @@ -.. Copyright 2016 OpenMarket Ltd -.. -.. Licensed under the Apache License, Version 2.0 (the "License"); -.. you may not use this file except in compliance with the License. -.. You may obtain a copy of the License at -.. -.. http://www.apache.org/licenses/LICENSE-2.0 -.. -.. Unless required by applicable law or agreed to in writing, software -.. distributed under the License is distributed on an "AS IS" BASIS, -.. WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -.. See the License for the specific language governing permissions and -.. limitations under the License. - -Event Structure -=============== - -All communication in Matrix is expressed in the form of data objects called -Events. These are the fundamental building blocks common to the client-server, -server-server and application-service APIs, and are described below. - -Note that the structure of these events may be different than those in the -server-server API. - -{{common_event_fields}} - -{{common_room_event_fields}} - -{{common_state_event_fields}} - - -Size limits ------------ - -The complete event MUST NOT be larger than 65535 bytes, when formatted as a -`PDU for the Server-Server protocol <../server_server/%SERVER_RELEASE_LABEL%#pdus>`_, -including any signatures, and encoded as `Canonical JSON`_. - -There are additional restrictions on sizes per key: - -- ``sender`` MUST NOT exceed 255 bytes (including domain). -- ``room_id`` MUST NOT exceed 255 bytes. -- ``state_key`` MUST NOT exceed 255 bytes. -- ``type`` MUST NOT exceed 255 bytes. -- ``event_id`` MUST NOT exceed 255 bytes. - -Some event types have additional size restrictions which are specified in -the description of the event. Additional keys have no limit other than that -implied by the total 65 KB limit on events. - -Room Events ------------ -.. NOTE:: - This section is a work in progress. - -This specification outlines several standard event types, all of which are -prefixed with ``m.`` - -{{m_room_aliases_event}} - -{{m_room_canonical_alias_event}} - -{{m_room_create_event}} - -{{m_room_join_rules_event}} - -{{m_room_member_event}} - -{{m_room_power_levels_event}} - -{{m_room_redaction_event}} - -.. _`Canonical JSON`: ../appendices.html#canonical-json diff --git a/specification/index.rst b/specification/index.rst index 33dff5a3..7aa6a672 100644 --- a/specification/index.rst +++ b/specification/index.rst @@ -348,6 +348,11 @@ pushed over federation to the participating servers in a room, currently using full mesh topology. Servers may also request backfill of events over federation from the other servers participating in a room. +.. Note:: + Events are not limited to the types defined in this specification. New or custom + event types can be created on a whim using the Java package naming convention. + For example, a ``com.example.game.score`` event can be sent by clients and other + clients would receive it through Matrix. Room Aliases ++++++++++++ diff --git a/specification/targets.yaml b/specification/targets.yaml index abcdc240..ed3dcee3 100644 --- a/specification/targets.yaml +++ b/specification/targets.yaml @@ -5,7 +5,6 @@ targets: client_server: files: - client_server_api.rst - - { 1: events.rst } - { 1: modules.rst } - { 2: feature_profiles.rst } - { 2: "group:modules" } # reference a group of files From 51698a5dd55232d39746269bfd164647ec56548f Mon Sep 17 00:00:00 2001 From: Travis Ralston Date: Mon, 10 Jun 2019 14:04:14 -0600 Subject: [PATCH 2/4] Clarify that people must own the namespace to use it --- specification/client_server_api.rst | 3 ++- specification/index.rst | 3 ++- 2 files changed, 4 insertions(+), 2 deletions(-) diff --git a/specification/client_server_api.rst b/specification/client_server_api.rst index 80e4a2f4..b67beca1 100644 --- a/specification/client_server_api.rst +++ b/specification/client_server_api.rst @@ -1330,7 +1330,8 @@ namespaced for each application and reduces the risk of clashes. Events are not limited to the types defined in this specification. New or custom event types can be created on a whim using the Java package naming convention. For example, a ``com.example.game.score`` event can be sent by clients and other - clients would receive it through Matrix. + clients would receive it through Matrix, assuming the client has access to the + ``com.example`` namespace. Note that the structure of these events may be different than those in the server-server API. diff --git a/specification/index.rst b/specification/index.rst index 7aa6a672..bf12a271 100644 --- a/specification/index.rst +++ b/specification/index.rst @@ -352,7 +352,8 @@ from the other servers participating in a room. Events are not limited to the types defined in this specification. New or custom event types can be created on a whim using the Java package naming convention. For example, a ``com.example.game.score`` event can be sent by clients and other - clients would receive it through Matrix. + clients would receive it through Matrix, assuming the client has access to the + ``com.example`` namespace. Room Aliases ++++++++++++ From 37e2d81d580083f8d7ed2f1a99e368231aab299c Mon Sep 17 00:00:00 2001 From: Travis Ralston Date: Mon, 10 Jun 2019 14:04:27 -0600 Subject: [PATCH 3/4] Hardcode the state event fields table into the spec See comment for why. --- specification/client_server_api.rst | 23 ++++++++++++++++++++++- 1 file changed, 22 insertions(+), 1 deletion(-) diff --git a/specification/client_server_api.rst b/specification/client_server_api.rst index b67beca1..ad58fd1a 100644 --- a/specification/client_server_api.rst +++ b/specification/client_server_api.rst @@ -1340,7 +1340,28 @@ server-server API. {{common_room_event_fields}} -{{common_state_event_fields}} +.. This is normally where we'd put the common_state_event_fields variable for the +.. magic table of what makes up a state event, however the table is verbose in our +.. custom rendering of swagger. To combat this, we just hardcode this particular +.. table. + +State Event Fields +++++++++++++++++++ + +In addition to the fields of a Room Event, State Events have the following fields. + ++-----------+--------+-------------------------------------------------------------+ +| Key | Type | Description | ++===========+========+=============================================================+ +| state_key | string | **Required.** A unique key which defines the overwriting | +| | | semantics for this piece of room state. This value is often | +| | | a zero-length string. The presence of this key makes this | +| | | event a State Event. State keys starting with an ``@`` are | +| | | reserved for referencing user IDs, such as room members. | +| | | With the exception of a few events, state events set with | +| | | a given user's ID as the state key MUST only be set by that | +| | | user. | ++-----------+--------+-------------------------------------------------------------+ Size limits From c9345ba6a32618ded7fbfd992469164f4f239496 Mon Sep 17 00:00:00 2001 From: Travis Ralston Date: Tue, 11 Jun 2019 09:53:36 -0600 Subject: [PATCH 4/4] Never forget the past --- specification/client_server_api.rst | 27 +++++++++++++++------------ 1 file changed, 15 insertions(+), 12 deletions(-) diff --git a/specification/client_server_api.rst b/specification/client_server_api.rst index ad58fd1a..948ddc22 100644 --- a/specification/client_server_api.rst +++ b/specification/client_server_api.rst @@ -1350,18 +1350,21 @@ State Event Fields In addition to the fields of a Room Event, State Events have the following fields. -+-----------+--------+-------------------------------------------------------------+ -| Key | Type | Description | -+===========+========+=============================================================+ -| state_key | string | **Required.** A unique key which defines the overwriting | -| | | semantics for this piece of room state. This value is often | -| | | a zero-length string. The presence of this key makes this | -| | | event a State Event. State keys starting with an ``@`` are | -| | | reserved for referencing user IDs, such as room members. | -| | | With the exception of a few events, state events set with | -| | | a given user's ID as the state key MUST only be set by that | -| | | user. | -+-----------+--------+-------------------------------------------------------------+ ++--------------+--------------+-------------------------------------------------------------+ +| Key | Type | Description | ++==============+==============+=============================================================+ +| state_key | string | **Required.** A unique key which defines the overwriting | +| | | semantics for this piece of room state. This value is often | +| | | a zero-length string. The presence of this key makes this | +| | | event a State Event. State keys starting with an ``@`` are | +| | | reserved for referencing user IDs, such as room members. | +| | | With the exception of a few events, state events set with | +| | | a given user's ID as the state key MUST only be set by that | +| | | user. | ++--------------+--------------+-------------------------------------------------------------+ +| prev_content | EventContent | Optional. The previous ``content`` for this event. If there | +| | | is no previous content, this key will be missing. | ++--------------+--------------+-------------------------------------------------------------+ Size limits