From 540b3a41ff69cbd0c7a7ca49432ef12e274f01f4 Mon Sep 17 00:00:00 2001 From: Kegsay Date: Wed, 14 Jan 2015 15:15:40 +0000 Subject: [PATCH] RSTify; typos --- drafts/application_services.rst | 261 ++++++++++++++++---------------- 1 file changed, 129 insertions(+), 132 deletions(-) diff --git a/drafts/application_services.rst b/drafts/application_services.rst index 9ec271ec..2c73107f 100644 --- a/drafts/application_services.rst +++ b/drafts/application_services.rst @@ -33,32 +33,29 @@ Example use cases for application services include: * Exposing existing communication services in Matrix - * Gateways to/from standards-based protocols (SIP, XMPP, IRC, RCS (MSRP), - SIMPLE, Lync, etc) - * Gateways to/from closed services (e.g. WhatsApp) - * Gateways could be architected as: + * Gateways to/from standards-based protocols (SIP, XMPP, IRC, RCS (MSRP), SIMPLE, Lync, etc) + * Gateways to/from closed services (e.g. WhatsApp) + * Gateways could be architected as: + + * Act as a virtual client on the non-Matrix network + (e.g. connect as multiple virtual clients to an IRC or XMPP server) + * Act as a server on the non-Matrix network + (e.g. speak s2s XMPP federation, or IRC link protocol) + * Act as an application service on the non-Matrix network + (e.g. link up as IRC services, or an XMPP component) + * Exposing a non-Matrix client interface listener from the AS + (e.g. listen on port 6667 for IRC clients, or port 5222 for XMPP clients) - * Act as a virtual client on the non-Matrix network - (e.g. connect as multiple virtual clients to an IRC or XMPP server) - * Act as a server on the non-Matrix network - (e.g. speak s2s XMPP federation, or IRC link protocol) - * Act as an application service on the non-Matrix network - (e.g. link up as IRC services, or an XMPP component) - * Exposing a non-Matrix client interface listener from the AS - (e.g. listen on port 6667 for IRC clients, or port 5222 for XMPP clients) - - * Bridging existing APIs into Matrix +* Bridging existing APIs into Matrix * e.g. SMS/MMS aggregator APIs * Domain-specific APIs such as SABRE - * Integrating more exotic content into Matrix - +* Integrating more exotic content into Matrix * e.g. MIDI<->Matrix gateway/bridge * 3D world <-> Matrix bridge - * Application services: - +* Application services: * Search engines (e.g. elasticsearch search indices) * Notification systems (e.g. send custom pushes for various hooks) * VoIP Conference services @@ -84,150 +81,150 @@ The application service registers with its host homeserver to offer its services In the registration process, the AS provides: - * Credentials to identify itself as an approved application service for that HS - * Details of the namespaces of users and rooms the AS is acting on behalf of and - "subscribing to" - * Namespaces are defined as a list of regexps against which to match room aliases, - room IDs, and user IDs. - * There is overlap between selecting events via the csv2 Filter API and subscribing - to events here - perhaps subscription involves passing a filter token into the - registration API. - * A URL base for receiving requests from the HS (as the AS is a server, - implementers expect to receive data via inbound requests rather than - long-poll outbound requests) +* Credentials to identify itself as an approved application service for that HS +* Details of the namespaces of users and rooms the AS is acting on behalf of and + "subscribing to" +* Namespaces are defined as a list of regexps against which to match room aliases, + room IDs, and user IDs. +* There is overlap between selecting events via the csv2 Filter API and subscribing + to events here - perhaps subscription involves passing a filter token into the + registration API. +* A URL base for receiving requests from the HS (as the AS is a server, + implementers expect to receive data via inbound requests rather than + long-poll outbound requests) On HS handling events to unknown users: - * If the HS receives an event for an unknown user who is in the namespace delegated to - the AS, then the HS queries the AS for the profile of that user. If the AS - confirms the existence of that user (from its perspective), then the HS - creates an account to represent the virtual user. - * The namespace of virtual user accounts should conform to a structure like - @.irc.freenode.Arathorn:matrix.org. This lets Matrix users communicate with - foreign users who are not yet mapped into Matrix via 3PID mappings or through - an existing non-virtual Matrix user by trying to talk to them via a gateway. - * The AS can alternatively preprovision virtual users using the existing CS API - rather than lazy-loading them in this manner. - * The AS may want to link the matrix ID of the sender through to their 3PID in - the remote ecosystem. E.g. a message sent from @matthew:matrix.org may wish - to originate from Arathorn on irc.freenode.net in the case of an IRC bridge. - It's left as an AS implementation detail as to how the user should authorise - the AS to act on its behalf. +* If the HS receives an event for an unknown user who is in the namespace delegated to + the AS, then the HS queries the AS for the profile of that user. If the AS + confirms the existence of that user (from its perspective), then the HS + creates an account to represent the virtual user. +* The namespace of virtual user accounts should conform to a structure like + ``@.irc.freenode.Arathorn:matrix.org``. This lets Matrix users communicate with + foreign users who are not yet mapped into Matrix via 3PID mappings or through + an existing non-virtual Matrix user by trying to talk to them via a gateway. +* The AS can alternatively preprovision virtual users using the existing CS API + rather than lazy-loading them in this manner. +* The AS may want to link the matrix ID of the sender through to their 3PID in + the remote ecosystem. E.g. a message sent from ``@matthew:matrix.org`` may wish + to originate from Arathorn on irc.freenode.net in the case of an IRC bridge. + It's left as an AS implementation detail as to how the user should authorise + the AS to act on its behalf. On HS handling events to unknown rooms: - * If the HS receives an invite to an unknown room which is in the namespace - delegated to the AS, then the HS queries the AS for the existence of that room. - If the AS confirms its existence (from its perspective), then the HS creates - the room. - * The initial state of the room may be populated by the AS by querying an - initialSync API (probably a subset of the CS initialSync API, to reuse the - same pattern for the equivalent function). As messages have to be signed - from the point of m.room.create, we will not be able to back-populate - arbitrary history for rooms which are lazy-created in this manner, and instead - have to chose the amount of history to be synchronised into the AS as a one-off. - * If exposing arbitrary history is required, then: +* If the HS receives an invite to an unknown room which is in the namespace + delegated to the AS, then the HS queries the AS for the existence of that room. + If the AS confirms its existence (from its perspective), then the HS creates + the room. +* The initial state of the room may be populated by the AS by querying an + initialSync API (probably a subset of the CS initialSync API, to reuse the + same pattern for the equivalent function). As messages have to be signed + from the point of ``m.room.create``, we will not be able to back-populate + arbitrary history for rooms which are lazy-created in this manner, and instead + have to chose the amount of history to be synchronised into the AS as a one-off. +* If exposing arbitrary history is required, then: - * either: the room history must be preemptively provisioned in the HS by the AS via - the CS API (TODO: meaning the CS API needs to support massaged - timestamps), resulting in conversation history being replicated between - the HS and the source store. - * or: the HS must delegate conversation storage entirely to the - AS using a Storage API (not defined here) which allows the existing - conversation store to back the HS, complete with all necessary Matrix - metadata (e.g. hashes, signatures, federation DAG, etc). This obviously - increases the burden of implementing an AS considerably, but is the only - option if the implementer wants to avoid duplicating conversation history - between the external data source and the HS. + * either: the room history must be preemptively provisioned in the HS by the AS via + the CS API (TODO: meaning the CS API needs to support massaged + timestamps), resulting in conversation history being replicated between + the HS and the source store. + * or: the HS must delegate conversation storage entirely to the + AS using a Storage API (not defined here) which allows the existing + conversation store to back the HS, complete with all necessary Matrix + metadata (e.g. hashes, signatures, federation DAG, etc). This obviously + increases the burden of implementing an AS considerably, but is the only + option if the implementer wants to avoid duplicating conversation history + between the external data source and the HS. On HS handling events to existing users and rooms: - * If the HS receives an event for a user or room that already exists (either - provisioned by the AS or by normal client interactions), then the message - is handled as normal. - * Events in the namespaces of rooms and users that the AS has subscribed to - are pushed to the AS using the same pattern as the federation API (without - any of the encryption or federation metadata). This serves precisely the - same purpose as the CS event stream and has the same data flow semantics - (and indeed an AS implementer could chose to use the CS event stream instead) - - * Events are linearised to avoid the AS having to handle the complexity of - linearisation, and because if linearisation is good enough for CS, it - should be good enough for AS. Should the AS require non-linearised events - from Matrix, it should implement the federation API rather than the AS API - instead. - * HS->AS event pushes are retried for reliability with sequence numbers - (or logical timestamping?) to presereve the linearisation order and ensure - a reliable event stream. - * Clustered HSes must linearise just as they do for the CS API. Clustered - ASes must loadbalance the inbound stream across the cluster as required. +* If the HS receives an event for a user or room that already exists (either + provisioned by the AS or by normal client interactions), then the message + is handled as normal. +* Events in the namespaces of rooms and users that the AS has subscribed to + are pushed to the AS using the same pattern as the federation API (without + any of the encryption or federation metadata). This serves precisely the + same purpose as the CS event stream and has the same data flow semantics + (and indeed an AS implementer could chose to use the CS event stream instead) + + * Events are linearised to avoid the AS having to handle the complexity of + linearisation, and because if linearisation is good enough for CS, it + should be good enough for AS. Should the AS require non-linearised events + from Matrix, it should implement the federation API rather than the AS API + instead. + * HS->AS event pushes are retried for reliability with sequence numbers + (or logical timestamping?) to preserve the linearisation order and ensure + a reliable event stream. + * Clustered HSes must linearise just as they do for the CS API. Clustered + ASes must loadbalance the inbound stream across the cluster as required. On AS relaying events from unknown-to-HS users: - * AS injects the event to the HS using the CS API, irrespective of whether the - target user or room is known to the HS or not. If the HS doesn't recognise - the target it goes through the same lazy-load provisioning as per above. - * The reason for not using a subset of the federation API here is because it - allows AS developers to reuse existing CS SDKs and benefit from the more - meaningful error handling of the CS API. The sending user ID must be - explicitly specified, as it cannot be inferred from the access_token, which - will be the same for all AS requests. +* AS injects the event to the HS using the CS API, irrespective of whether the + target user or room is known to the HS or not. If the HS doesn't recognise + the target it goes through the same lazy-load provisioning as per above. +* The reason for not using a subset of the federation API here is because it + allows AS developers to reuse existing CS SDKs and benefit from the more + meaningful error handling of the CS API. The sending user ID must be + explicitly specified, as it cannot be inferred from the access_token, which + will be the same for all AS requests. - * TODO: or do we maintain a separate access_token mapping? It seems like - unnecessary overhead for the AS developer; easier to just use a single - privileged access_token and just track which userid is emitting events? - * If the AS is spoofing the identity of a real (not virtual) matrix user, - we should actually let them log themselves in via OAuth2 to give permission - to the AS to act on their behalf. - * We can't auth gatewayed virtual users from 3rd party systems who are being - relayed into Matrix, as the relaying is happening whether the user likes it - or not. Therefore we do need to be able to spoof sender ID for virtual users. + * TODO: or do we maintain a separate ``access_token`` mapping? It seems like + unnecessary overhead for the AS developer; easier to just use a single + privileged ``access_token`` and just track which ``user_id`` is emitting events? + * If the AS is spoofing the identity of a real (not virtual) matrix user, + we should actually let them log themselves in via OAuth2 to give permission + to the AS to act on their behalf. + * We can't auth gatewayed virtual users from 3rd party systems who are being + relayed into Matrix, as the relaying is happening whether the user likes it + or not. Therefore we do need to be able to spoof sender ID for virtual users. On AS relaying events in unknown-to-HS rooms: - * See above. +* See above. On AS publishing aliases for virtual rooms: - * AS uses the normal alias management API to preemptively create/delete public - directory entries for aliases for virtual rooms provided by the AS. - * In order to create these aliases, the underlying room ID must also exist, so - at least the m.room.create of that room must also be prepopulated. It seems - sensible to prepopulate the required initial state and history of the room to - avoid a two-phase prepopulation process. +* AS uses the normal alias management API to preemptively create/delete public + directory entries for aliases for virtual rooms provided by the AS. +* In order to create these aliases, the underlying room ID must also exist, so + at least the ``m.room.create`` of that room must also be prepopulated. It seems + sensible to prepopulate the required initial state and history of the room to + avoid a two-phase prepopulation process. On unregistering the AS from the HS: - * An AS must tell the HS when it is going offline in order to stop receiving - requests from the HS. It does this by hitting an API on the HS. +* An AS must tell the HS when it is going offline in order to stop receiving + requests from the HS. It does this by hitting an API on the HS. AS Visibility: - * If an AS needs to sniff events in a room in order to operate on them (e.g. - to act as a search engine) but not inject traffic into the room, it should - do so by subscribing to the relevant events without actually joining the room. - * If the AS needs to participate in the room as a virtual user (e.g. an IVR - service, or a bot, or a gatewayed virtual user), it should join the room - normally. - * There are rare instances where an AS may wish to participate in a room - (including inserting messages), but be hidden from the room list - e.g. a - conferencing server focus bot may wish to join many rooms as the focus and - both listen to VoIP setups and inject its own VoIP answers, without ever - being physically seen in the room. In this scenario, the user should set - its presence to 'invisible', a state that HSes should only allow AS-authed - users to set. +* If an AS needs to sniff events in a room in order to operate on them (e.g. + to act as a search engine) but not inject traffic into the room, it should + do so by subscribing to the relevant events without actually joining the room. +* If the AS needs to participate in the room as a virtual user (e.g. an IVR + service, or a bot, or a gatewayed virtual user), it should join the room + normally. +* There are rare instances where an AS may wish to participate in a room + (including inserting messages), but be hidden from the room list - e.g. a + conferencing server focus bot may wish to join many rooms as the focus and + both listen to VoIP setups and inject its own VoIP answers, without ever + being physically seen in the room. In this scenario, the user should set + its presence to 'invisible', a state that HSes should only allow AS-authed + users to set. E2E Encryption - * The AS obviously has no visibility to E2E encrypted messages, unless it is - explicitly added to an encrypted room and participates in the group chat - itself. +* The AS obviously has no visibility to E2E encrypted messages, unless it is + explicitly added to an encrypted room and participates in the group chat + itself. Extensions to CS API ==================== - * Ability to assert the identity of the virtual user for all methods. - * Ability to massage timestamps when prepopulating historical state and - messages of virtual rooms (either by overriding origin_server_ts (preferred) or - adding an as_ts which we expect clients to honour) - * Ability to delete aliases (including from the directory) as well as create them. \ No newline at end of file +* Ability to assert the identity of the virtual user for all methods. +* Ability to massage timestamps when prepopulating historical state and + messages of virtual rooms (either by overriding ``origin_server_ts`` (preferred) or + adding an ``as_ts`` which we expect clients to honour) +* Ability to delete aliases (including from the directory) as well as create them.