wholetrans/docs-matrix-spec
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity

Spec /relations and aggregations (#1062)

Browse source 
* Commit to show changes to rich replies section

* Move rich replies to a module

* Add remainder of MSC2674

* Pivot away from MSC3440: Threads

* Add changelog entries so far

* Make a note for why we have aggregations/relations if nothing uses it

* Outright remove threads references

Apparently this breaks the table of contents

* Define MSC2675

* Define MSC3666

* Add note for rich replies?

* Update content/client-server-api/_index.md

Co-authored-by: Patrick Cloke <clokep@users.noreply.github.com>

* Clarify how ignoring works for aggregations.

* Try to clarify redactions a bit

* Clarify using parent/child language

* Add missing bits of MSC2675

* Add changelog for aggregations

* Appease the linters

* Update data/api/client-server/relations.yaml

Co-authored-by: Patrick Cloke <clokep@users.noreply.github.com>

* Apply suggestions from code review

Co-authored-by: Richard van der Hoff <1389908+richvdh@users.noreply.github.com>

* Apply suggestions from code review

Co-authored-by: Richard van der Hoff <1389908+richvdh@users.noreply.github.com>

* Try to clarify the return of /relations

* Fix required attribute

* Fix wording round 1

* Try to fix pagination

* Copy/paste the endpoint to make Open API happy

* Fix code block examples for rich replies

* Apply suggestions from code review

Co-authored-by: Richard van der Hoff <1389908+richvdh@users.noreply.github.com>

* Apply suggestions on all 3 endpoints

* Fix description of relationships API

* Fix warning about server-side aggregation/bundling

Co-authored-by: Patrick Cloke <clokep@users.noreply.github.com>
Co-authored-by: Richard van der Hoff <1389908+richvdh@users.noreply.github.com>
This commit is contained in:
Travis Ralston 2022-06-08 15:32:55 -06:00 • committed by GitHub
parent f14e18131b
commit c4db688af8
No known key found for this signature in database
GPG key ID: 4AEE18F83AFDEB23
8 changed files with 875 additions and 163 deletions
Download patch file Download diff file Expand all files Collapse all files

222
content/client-server-api/_index.md
View file

@ -1798,6 +1798,16 @@ There are several APIs provided to `GET` events for a room:
### Sending events to a room
{{% boxes/note %}}
{{% added-in v="1.3" %}}
Servers might need to post-process some events if they
[relate to](#forming-relationships-between-events) another event. The event's
relationship type (`rel_type`) determines any restrictions which might apply,
such as the user only being able to send one event of a given type in relation
to another.
{{% /boxes/note %}}
{{% http-api spec="client-server" api="room_state" %}}
**Examples**
@ -1888,6 +1898,216 @@ the topic to be removed from the room.
{{% http-api spec="client-server" api="redaction" %}}
### Forming relationships between events
{{% changed-in v="1.3" %}}
In some cases it is desirable to logically associate one event's contents with
another event's contents — for example, when replying to a message, editing an
event, or simply looking to add context for an event's purpose.
Events are related to each other in a parent/child structure, where any event can
become a parent by simply having a child event point at it. Parent events do not
define their children, instead relying on the children to describe their parent.
The relationship between a child and its parent event is described in the child
event's `content` as `m.relates_to` (defined below). A child event can point at
any other event, including another child event, to build the relationship so long
as both events are in the same room, however additional restrictions might be imposed
by the type of the relationship (the `rel_type`).
{{% boxes/note %}}
Child events can point at other child events, forming a chain of events. These chains
can naturally take the shape of a tree if two independent children point at a single
parent event, for example.
{{% /boxes/note %}}
To allow the server to aggregate and find child events for a parent, the `m.relates_to`
key of an event MUST be included in the plaintext copy of the event. It cannot be
exclusively recorded in the encrypted payload as the server cannot decrypt the event
for processing.
{{% boxes/warning %}}
If an encrypted event contains an `m.relates_to` in its payload, it should be
ignored and instead favour the plaintext `m.relates_to` copy (including when there
is no plaintext copy). This is to ensure the client's behaviour matches the server's
capability to handle relationships.
{{% /boxes/warning %}}
Relationships which don't match the schema, or which break the rules of a relationship,
are simply ignored. An example might be the parent and child being in different
rooms, or the relationship missing properties required by the schema below. Clients
handling such invalid relationships should show the events independently of each
other, optionally with an error message.
{{% boxes/note %}}
While this specification describes an `m.relates_to` object containing a `rel_type`, there
is not currently any relationship type which uses this structure. Replies, described below,
form their relationship outside of the `rel_type` as a legacy type of relationship. Future
versions of the specification might change replies to better match the relationship structures.
Custom `rel_type`s can, and should, still use the schema described above for relevant
behaviour.
{{% /boxes/note %}}
`m.relates_to` is defined as follows:
{{% definition path="api/client-server/definitions/m.relates_to" %}}
#### Relationship types
This specification describes the following relationship types:
* [Rich replies](#rich-replies) (**Note**: does not use `rel_type`).
#### Aggregations
{{% added-in v="1.3" %}}
Some child events can be "aggregated" by the server, depending on their
`rel_type`. This can allow a set of child events to be summarised to the client without
the client needing the child events themselves.
An example of this might be that a `rel_type` requires an extra `key` field which, when
appropriately specified, would mean that the client receives a total count for the number
of times that `key` was used by child events.
The actual aggregation format depends on the `rel_type`.
{{% boxes/note %}}
This specification does not currently describe any `rel_type`s which require
aggregation. This functionality forms a framework for future extensions.
{{% /boxes/note %}}
Aggregations are sometimes automatically included by a server alongside the parent
event. This is known as a "bundled aggregation" or "bundle" for simplicity. The
act of doing this is "bundling".
When an event is served to the client through the APIs listed below, a `m.relations` property
is included under `unsigned` if the event has child events which can be aggregated and point
at it. The `m.relations` property is an object keyed by `rel_type` and value being the type-specific
aggregated format for that `rel_type`, also known as the bundle.
For example (unimportant fields not included):
```json
{
"event_id": "$my_event",
"unsigned": {
"m.relations": {
"org.example.possible_annotations": [
{
"key": "👍",
"origin_server_ts": 1562763768320,
"count": 3
},
{
"key": "👎",
"origin_server_ts": 1562763768320,
"count": 1
}
],
"org.example.possible_thread": {
"current_server_participated": true,
"count": 7,
"latest_event": {
"event_id": "$another_event",
"content": {
"body": "Hello world"
}
}
}
}
}
}
```
Note how the `org.example.possible_annotations` bundle is an array compared to the
`org.example.possible_thread` bundle where the server is summarising the state of
the relationship in a single object. Both are valid ways to aggregate, and their
exact types depend on the `rel_type`.
{{% boxes/warning %}}
State events do not currently receive bundled aggregations. This is not
necessarily a deliberate design decision, and MSCs which aim to fix this are welcome.
{{% /boxes/warning %}}
The endpoints where the server *should* include bundled aggregations are:
* [`GET /rooms/{roomId}/messages`](#get_matrixclientv3roomsroomidmessages)
* [`GET /rooms/{roomId}/context/{eventId}`](#get_matrixclientv3roomsroomidcontexteventid)
* [`GET /rooms/{roomId}/event/{eventId}`](#get_matrixclientv3roomsroomideventeventid)
* [`GET /rooms/{roomId}/relations/{eventId}`](#get_matrixclientv1roomsroomidrelationseventid)
* [`GET /rooms/{roomId}/relations/{eventId}/{relType}`](#get_matrixclientv1roomsroomidrelationseventidreltype)
* [`GET /rooms/{roomId}/relations/{eventId}/{relType}/{eventType}`](#get_matrixclientv1roomsroomidrelationseventidreltypeeventtype)
* [`GET /sync`](#get_matrixclientv3sync) when the relevant section has a `limited` value
of `true`.
* [`POST /search`](#post_matrixclientv3search) for any matching events under `room_events`.
{{% boxes/note %}}
The server is **not** required to return bundled aggregations on deprecated endpoints
such as `/initialSync`.
{{% /boxes/note %}}
While this functionality allows the client to see what was known to the server at the
time of handling, the client should continue to aggregate locally if it is aware of
the relationship type's behaviour. For example, a client might increment a `count`
on a parent event's bundle if it saw a new child event which referenced that parent.
The bundle provided by the server only includes child events which were known at the
time the client would receive the bundle. For example, in a single `/sync` response
with the parent and multiple child events the child events would have already been
included on the parent's `m.relations` field. Events received in future syncs would
need to be aggregated manually by the client.
{{% boxes/note %}}
Events from [ignored users](#ignoring-users) do not appear in the aggregation
from the server, however clients might still have events from ignored users cached. Like
with normal events, clients will need to de-aggregate child events sent by ignored users to
avoid them being considered in counts. Servers must additionally ensure they do not
consider child events from ignored users when preparing a bundle for the client.
{{% /boxes/note %}}
When a parent event is redacted, the child events which pointed to that parent remain, however
when a child event is redacted then the relationship is broken. Therefore, the server needs
to de-aggregate or disassociate the event once the relationship is lost. Clients with local
aggregation or which handle redactions locally should do the same.
It is suggested that clients perform local echo on aggregations — for instance, aggregating
a new child event into a bundle optimistically until the server returns a failure or the client
gives up on sending the event, at which point the event should be de-aggregated and an
error or similar shown. The client should be cautious to not aggregate an event twice if
it has already optimistically aggregated the event. Clients are encouraged to take this
a step further to additionally track child events which target unsent/pending events,
likely using the transaction ID as a temporary event ID until a proper event ID is known.
{{% boxes/warning %}}
Due to history visibility restrictions, child events might not be visible to the user
if they are in a section of history the user cannot see. This means any bundles which would
normally include those events will be lacking them and the client will not be able to
locally aggregate the events either — relating events of importance (such as votes) should
take into consideration history visibility.
Additionally, if the server is missing portions of the room history then it may not be
able to accurately aggregate the events.
{{% /boxes/warning %}}
#### Relationships API
{{% added-in v="1.3" %}}
To retrieve the child events for a parent from the server, the client can call the
following endpoint.
This endpoint is particularly useful if the client has lost context on the aggregation for
a parent event and needs to rebuild/verify it.
{{% boxes/note %}}
Because replies do not use `rel_type`, they will not be accessible via this API.
{{% /boxes/note %}}
{{% http-api spec="client-server" api="relations" %}}
## Rooms
### Types
@ -2294,6 +2514,7 @@ that profile.
| Module / Profile | Web | Mobile | Desktop | CLI | Embedded |
|------------------------------------------------------------|-----------|----------|----------|----------|----------|
| [Instant Messaging](#instant-messaging) | Required | Required | Required | Required | Optional |
| [Rich replies](#rich-replies) | Optional | Optional | Optional | Optional | Optional |
| [Direct Messaging](#direct-messaging) | Required | Required | Required | Required | Optional |
| [Mentions](#user-room-and-group-mentions) | Required | Required | Required | Optional | Optional |
| [Presence](#presence) | Required | Required | Required | Required | Optional |
@ -2373,6 +2594,7 @@ applications, they are not intended to be fully-fledged communication
systems.
{{% cs-module name="instant_messaging" %}}
{{% cs-module name="rich_replies" %}}
{{% cs-module name="voip_events" %}}
{{% cs-module name="typing_notifications" %}}
{{% cs-module name="receipts" %}}

163
content/client-server-api/modules/instant_messaging.md
View file

@ -287,169 +287,6 @@ when using the `m.heroes` to calculate the name. Clients SHOULD use
minimum 5 heroes to calculate room names where possible, but may use
more or less to fit better with their user experience.
##### Rich replies
In some cases, events may wish to reference other events. This could be
to form a thread of messages for the user to follow along with, or to
provide more context as to what a particular event is describing.
Currently, the only kind of relation defined is a "rich reply" where a
user may reference another message to create a thread-like conversation.
Relationships are defined under an `m.relates_to` key in the event's
`content`. If the event is of the type `m.room.encrypted`, the
`m.relates_to` key MUST NOT be covered by the encryption and instead be
put alongside the encryption information held in the `content`.
A rich reply is formed through use of an `m.relates_to` relation for
`m.in_reply_to` where a single key, `event_id`, is used to reference the
event being replied to. The referenced event ID SHOULD belong to the
same room where the reply is being sent. Clients should be cautious of
the event ID belonging to another room, or being invalid entirely. Rich
replies can only be constructed in the form of `m.room.message` events
with a `msgtype` of `m.text` or `m.notice`. Due to the fallback
requirements, rich replies cannot be constructed for types of `m.emote`,
`m.file`, etc. Rich replies may reference any other `m.room.message`
event, however. Rich replies may reference another event which also has
a rich reply, infinitely.
An `m.in_reply_to` relationship looks like the following:
```
{
...
"type": "m.room.message",
"content": {
"msgtype": "m.text",
"body": "<body including fallback>",
"format": "org.matrix.custom.html",
"formatted_body": "<HTML including fallback>",
"m.relates_to": {
"m.in_reply_to": {
"event_id": "$another:event.com"
}
}
}
}
```
##### Fallbacks for rich replies
Some clients may not have support for rich replies and therefore need a
fallback to use instead. Clients that do not support rich replies should
render the event as if rich replies were not special.
Clients that do support rich replies MUST provide the fallback format on
replies, and MUST strip the fallback before rendering the reply. Rich
replies MUST have a `format` of `org.matrix.custom.html` and therefore a
`formatted_body` alongside the `body` and appropriate `msgtype`. The
specific fallback text is different for each `msgtype`, however the
general format for the `body` is:
> <@alice:example.org> This is the original body
This is where the reply goes
The `formatted_body` should use the following template:
<mx-reply>
<blockquote>
<a href="https://matrix.to/#/!somewhere:example.org/$event:example.org">In reply to</a>
<a href="https://matrix.to/#/@alice:example.org">@alice:example.org</a>
<br />
<!-- This is where the related event's HTML would be. -->
</blockquote>
</mx-reply>
This is where the reply goes.
If the related event does not have a `formatted_body`, the event's
`body` should be considered after encoding any HTML special characters.
Note that the `href` in both of the anchors use a [matrix.to
URI](/appendices#matrixto-navigation).
###### Stripping the fallback
Clients which support rich replies MUST strip the fallback from the
event before rendering the event. This is because the text provided in
the fallback cannot be trusted to be an accurate representation of the
event. After removing the fallback, clients are recommended to represent
the event referenced by `m.in_reply_to` similar to the fallback's
representation, although clients do have creative freedom for their user
interface. Clients should prefer the `formatted_body` over the `body`,
just like with other `m.room.message` events.
To strip the fallback on the `body`, the client should iterate over each
line of the string, removing any lines that start with the fallback
prefix ("&gt; ", including the space, without quotes) and stopping when
a line is encountered without the prefix. This prefix is known as the
"fallback prefix sequence".
To strip the fallback on the `formatted_body`, the client should remove
the entirety of the `mx-reply` tag.
###### Fallback for `m.text`, `m.notice`, and unrecognised message types
Using the prefix sequence, the first line of the related event's `body`
should be prefixed with the user's ID, followed by each line being
prefixed with the fallback prefix sequence. For example:
> <@alice:example.org> This is the first line
> This is the second line
This is the reply
The `formatted_body` uses the template defined earlier in this section.
###### Fallback for `m.emote`
Similar to the fallback for `m.text`, each line gets prefixed with the
fallback prefix sequence. However an asterisk should be inserted before
the user's ID, like so:
> * <@alice:example.org> feels like today is going to be a great day
This is the reply
The `formatted_body` has a subtle difference for the template where the
asterisk is also inserted ahead of the user's ID:
<mx-reply>
<blockquote>
<a href="https://matrix.to/#/!somewhere:example.org/$event:example.org">In reply to</a>
* <a href="https://matrix.to/#/@alice:example.org">@alice:example.org</a>
<br />
<!-- This is where the related event's HTML would be. -->
</blockquote>
</mx-reply>
This is where the reply goes.
###### Fallback for `m.image`, `m.video`, `m.audio`, and `m.file`
The related event's `body` would be a file name, which may not be very
descriptive. The related event should additionally not have a `format`
or `formatted_body` in the `content` - if the event does have a `format`
and/or `formatted_body`, those fields should be ignored. Because the
filename alone may not be descriptive, the related event's `body` should
be considered to be `"sent a file."` such that the output looks similar
to the following:
> <@alice:example.org> sent a file.
This is the reply
<mx-reply>
<blockquote>
<a href="https://matrix.to/#/!somewhere:example.org/$event:example.org">In reply to</a>
<a href="https://matrix.to/#/@alice:example.org">@alice:example.org</a>
<br />
sent a file.
</blockquote>
</mx-reply>
This is where the reply goes.
For `m.image`, the text should be `"sent an image."`. For `m.video`, the
text should be `"sent a video."`. For `m.audio`, the text should be
`"sent an audio file"`.
##### Spoiler messages
{{% added-in v="1.1" %}}

182
content/client-server-api/modules/rich_replies.md Normal file
View file

@ -0,0 +1,182 @@
---
type: module
---
### Rich replies
{{% changed-in v="1.3" %}}
Rich replies are a
special kind of [relationship](#forming-relationships-between-events) which
effectively quotes the referenced event for the client to render/process how
it wishes. They are normally used with [`m.room.message`](#mroommessage) events.
{{% boxes/note %}}
Until v1.3 of the spec, rich replies were limited to `m.room.message` events
which could represent an HTML-formatted body. As of v1.3 this is now expanded
to *all* event types by dropping the requirement that an HTML-formatted body
be included.
Additionally, a rich reply can reference any other event type as of v1.3.
Previously, a rich reply could only reference another `m.room.message` event.
{{% /boxes/note %}}
When possible, events SHOULD include a [fallback representation](#fallbacks-for-rich-replies)
to allow clients which do not render rich replies to still see something which
appears to be a quoted reply.
Though rich replies form a relationship to another event, they do not
use `rel_type` to create this relationship. Instead, a subkey named `m.in_reply_to`
is used to describe the reply's relationship, leaving the other properties of
`m.relates_to` to describe the primary relationship of the event. This means
that if an event is simply in reply to another event, without further relationship,
the `rel_type` and `event_id` properties of `m.relates_to` become *optional*.
An example reply would be:
```json5
{
"content": {
"m.relates_to": {
"m.in_reply_to": {
"event_id": "$another_event"
}
},
"body": "That sounds like a great idea!"
},
// other fields as required by events
}
```
Note that the `event_id` of the `m.in_reply_to` object has the same requirements
as if it were to be under `m.relates_to` directly instead.
#### Fallbacks for rich replies
Some clients may not have support for rich replies and therefore need a
fallback to use instead. Clients that do not support rich replies should
render the event as if rich replies were not special.
Clients that do support rich replies SHOULD provide the fallback format on
replies, and MUST strip the fallback before rendering the reply. The
specific fallback text is different for each `msgtype`, however the
general format for the `body` is:
```text
> <@alice:example.org> This is the original body
This is where the reply goes
```
The `formatted_body`, if present and using an associated `format` of
`org.matrix.custom.html`, should use the following template:
```html
<mx-reply>
<blockquote>
<a href="https://matrix.to/#/!somewhere:example.org/$event:example.org">In reply to</a>
<a href="https://matrix.to/#/@alice:example.org">@alice:example.org</a>
<br />
<!-- This is where the related event's HTML would be. -->
</blockquote>
</mx-reply>
This is where the reply goes.
```
If the related event does not have a `formatted_body`, the event's
`body` should be considered after encoding any HTML special characters.
Note that the `href` in both of the anchors use a [matrix.to
URI](/appendices#matrixto-navigation).
##### Stripping the fallback
Clients which support rich replies MUST strip the fallback from the
event before rendering the event. This is because the text provided in
the fallback cannot be trusted to be an accurate representation of the
event. After removing the fallback, clients are recommended to represent
the event referenced by `m.in_reply_to` similar to the fallback's
representation, although clients do have creative freedom for their user
interface. Clients should prefer the `formatted_body` over the `body`,
just like with other `m.room.message` events.
To strip the fallback on the `body`, the client should iterate over each
line of the string, removing any lines that start with the fallback
prefix ("&gt; ", including the space, without quotes) and stopping when
a line is encountered without the prefix. This prefix is known as the
"fallback prefix sequence".
To strip the fallback on the `formatted_body`, the client should remove
the entirety of the `mx-reply` tag.
##### Fallback for `m.text`, `m.notice`, and unrecognised message types
Using the prefix sequence, the first line of the related event's `body`
should be prefixed with the user's ID, followed by each line being
prefixed with the fallback prefix sequence. For example:
```text
> <@alice:example.org> This is the first line
> This is the second line
This is the reply
```
The `formatted_body` uses the template defined earlier in this section.
##### Fallback for `m.emote`
Similar to the fallback for `m.text`, each line gets prefixed with the
fallback prefix sequence. However an asterisk should be inserted before
the user's ID, like so:
```text
> * <@alice:example.org> feels like today is going to be a great day
This is the reply
```
The `formatted_body` has a subtle difference for the template where the
asterisk is also inserted ahead of the user's ID:
```html
<mx-reply>
<blockquote>
<a href="https://matrix.to/#/!somewhere:example.org/$event:example.org">In reply to</a>
* <a href="https://matrix.to/#/@alice:example.org">@alice:example.org</a>
<br />
<!-- This is where the related event's HTML would be. -->
</blockquote>
</mx-reply>
This is where the reply goes.
```
##### Fallback for `m.image`, `m.video`, `m.audio`, and `m.file`
The related event's `body` would be a file name, which may not be very
descriptive. The related event should additionally not have a `format`
or `formatted_body` in the `content` - if the event does have a `format`
and/or `formatted_body`, those fields should be ignored. Because the
filename alone may not be descriptive, the related event's `body` should
be considered to be `"sent a file."` such that the output looks similar
to the following:
```text
> <@alice:example.org> sent a file.
This is the reply
```
```html
<mx-reply>
<blockquote>
<a href="https://matrix.to/#/!somewhere:example.org/$event:example.org">In reply to</a>
<a href="https://matrix.to/#/@alice:example.org">@alice:example.org</a>
<br />
sent a file.
</blockquote>
</mx-reply>
This is where the reply goes.
```
For `m.image`, the text should be `"sent an image."`. For `m.video`, the
text should be `"sent a video."`. For `m.audio`, the text should be
`"sent an audio file"`.