wholetrans/docs-matrix-spec
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity

Only documents in matrix-doc!

Browse source
This commit is contained in:
Oddvar Lovaas 2015-10-08 19:27:08 +01:00
parent 2ab9372b16
commit 51ab51ae26
31 changed files with 0 additions and 2056 deletions
Download patch file Download diff file Expand all files Collapse all files

652
supporting-docs/guides/2015-08-10-client-server.rst Normal file
View file

@ -0,0 +1,652 @@
---
layout: post
title: Client-Server API
categories: guides
---
|
.. figure:: http://matrix.org/matrix.png
:width: 305px
:height: 130px
:alt: [matrix]
:align: center
How to use the client-server API
================================
.. NOTE::
The git version of this document is {% project_version %}
This guide focuses on how the client-server APIs *provided by the reference
home server* can be used. Since this is specific to a home server
implementation, there may be variations in relation to registering/logging in
which are not covered in extensive detail in this guide.
If you haven't already, get a home server up and running on
``http://localhost:8008``.
Accounts
========
Before you can send and receive messages, you must **register** for an account.
If you already have an account, you must **login** into it.
.. NOTE::
`Try out the fiddle`__
.. __: http://jsfiddle.net/gh/get/jquery/1.8.3/matrix-org/matrix-doc/tree/master/supporting-docs/howtos/jsfiddles/register_login
Registration
------------
The aim of registration is to get a user ID and access token which you will need
when accessing other APIs::
curl -XPOST -d '{"user":"example", "password":"wordpass", "type":"m.login.password"}' "http://localhost:8008/_matrix/client/api/v1/register"
{
"access_token": "QGV4YW1wbGU6bG9jYWxob3N0.AqdSzFmFYrLrTmteXc",
"home_server": "localhost",
"user_id": "@example:localhost"
}
NB: If a ``user`` is not specified, one will be randomly generated for you.
If you do not specify a ``password``, you will be unable to login to the account
if you forget the ``access_token``.
Implementation note: The matrix specification does not enforce how users
register with a server. It just specifies the URL path and absolute minimum
keys. The reference home server uses a username/password to authenticate user,
but other home servers may use different methods. This is why you need to
specify the ``type`` of method.
Login
-----
The aim when logging in is to get an access token for your existing user ID::
curl -XGET "http://localhost:8008/_matrix/client/api/v1/login"
{
"flows": [
{
"type": "m.login.password"
}
]
}
curl -XPOST -d '{"type":"m.login.password", "user":"example", "password":"wordpass"}' "http://localhost:8008/_matrix/client/api/v1/login"
{
"access_token": "QGV4YW1wbGU6bG9jYWxob3N0.vRDLTgxefmKWQEtgGd",
"home_server": "localhost",
"user_id": "@example:localhost"
}
Implementation note: Different home servers may implement different methods for
logging in to an existing account. In order to check that you know how to login
to this home server, you must perform a ``GET`` first and make sure you
recognise the login type. If you do not know how to login, you can
``GET /login/fallback`` which will return a basic webpage which you can use to
login. The reference home server implementation support username/password login,
but other home servers may support different login methods (e.g. OAuth2).
Communicating
=============
In order to communicate with another user, you must **create a room** with that
user and **send a message** to that room.
.. NOTE::
`Try out the fiddle`__
.. __: http://jsfiddle.net/gh/get/jquery/1.8.3/matrix-org/matrix-doc/tree/master/supporting-docs/howtos/jsfiddles/create_room_send_msg
Creating a room
---------------
If you want to send a message to someone, you have to be in a room with them. To
create a room::
curl -XPOST -d '{"room_alias_name":"tutorial"}' "http://localhost:8008/_matrix/client/api/v1/createRoom?access_token=YOUR_ACCESS_TOKEN"
{
"room_alias": "#tutorial:localhost",
"room_id": "!CvcvRuDYDzTOzfKKgh:localhost"
}
The "room alias" is a human-readable string which can be shared with other users
so they can join a room, rather than the room ID which is a randomly generated
string. You can have multiple room aliases per room.
.. TODO(kegan)
How to add/remove aliases from an existing room.
Sending messages
----------------
You can now send messages to this room::
curl -XPOST -d '{"msgtype":"m.text", "body":"hello"}' "http://localhost:8008/_matrix/client/api/v1/rooms/%21CvcvRuDYDzTOzfKKgh%3Alocalhost/send/m.room.message?access_token=YOUR_ACCESS_TOKEN"
{
"event_id": "YUwRidLecu"
}
The event ID returned is a unique ID which identifies this message.
NB: There are no limitations to the types of messages which can be exchanged.
The only requirement is that ``"msgtype"`` is specified. The Matrix
specification outlines the following standard types: ``m.text``, ``m.image``,
``m.audio``, ``m.video``, ``m.location``, ``m.emote``. See the specification for
more information on these types.
Users and rooms
===============
Each room can be configured to allow or disallow certain rules. In particular,
these rules may specify if you require an **invitation** from someone already in
the room in order to **join the room**. In addition, you may also be able to
join a room **via a room alias** if one was set up.
.. NOTE::
`Try out the fiddle`__
.. __: http://jsfiddle.net/gh/get/jquery/1.8.3/matrix-org/matrix-doc/tree/master/supporting-docs/howtos/jsfiddles/room_memberships
Inviting a user to a room
-------------------------
You can directly invite a user to a room like so::
curl -XPOST -d '{"user_id":"@myfriend:localhost"}' "http://localhost:8008/_matrix/client/api/v1/rooms/%21CvcvRuDYDzTOzfKKgh%3Alocalhost/invite?access_token=YOUR_ACCESS_TOKEN"
This informs ``@myfriend:localhost`` of the room ID
``!CvcvRuDYDzTOzfKKgh:localhost`` and allows them to join the room.
Joining a room via an invite
----------------------------
If you receive an invite, you can join the room::
curl -XPOST -d '{}' "http://localhost:8008/_matrix/client/api/v1/rooms/%21CvcvRuDYDzTOzfKKgh%3Alocalhost/join?access_token=YOUR_ACCESS_TOKEN"
NB: Only the person invited (``@myfriend:localhost``) can change the membership
state to ``"join"``. Repeatedly joining a room does nothing.
Joining a room via an alias
---------------------------
Alternatively, if you know the room alias for this room and the room config
allows it, you can directly join a room via the alias::
curl -XPOST -d '{}' "http://localhost:8008/_matrix/client/api/v1/join/%23tutorial%3Alocalhost?access_token=YOUR_ACCESS_TOKEN"
{
"room_id": "!CvcvRuDYDzTOzfKKgh:localhost"
}
You will need to use the room ID when sending messages, not the room alias.
NB: If the room is configured to be an invite-only room, you will still require
an invite in order to join the room even though you know the room alias. As a
result, it is more common to see a room alias in relation to a public room,
which do not require invitations.
Getting events
==============
An event is some interesting piece of data that a client may be interested in.
It can be a message in a room, a room invite, etc. There are many different ways
of getting events, depending on what the client already knows.
.. NOTE::
`Try out the fiddle`__
.. __: http://jsfiddle.net/gh/get/jquery/1.8.3/matrix-org/matrix-doc/tree/master/supporting-docs/howtos/jsfiddles/event_stream
Getting all state
-----------------
If the client doesn't know any information on the rooms the user is
invited/joined on, they can get all the user's state for all rooms::
curl -XGET "http://localhost:8008/_matrix/client/api/v1/initialSync?access_token=YOUR_ACCESS_TOKEN"
{
"end": "s39_18_0",
"presence": [
{
"content": {
"last_active_ago": 1061436,
"user_id": "@example:localhost"
},
"type": "m.presence"
}
],
"rooms": [
{
"membership": "join",
"messages": {
"chunk": [
{
"content": {
"@example:localhost": 10,
"default": 0
},
"event_id": "wAumPSTsWF",
"required_power_level": 10,
"room_id": "!MkDbyRqnvTYnoxjLYx:localhost",
"state_key": "",
"ts": 1409665585188,
"type": "m.room.power_levels",
"user_id": "@example:localhost"
},
{
"content": {
"join_rule": "public"
},
"event_id": "jrLVqKHKiI",
"required_power_level": 10,
"room_id": "!MkDbyRqnvTYnoxjLYx:localhost",
"state_key": "",
"ts": 1409665585188,
"type": "m.room.join_rules",
"user_id": "@example:localhost"
},
{
"content": {
"level": 10
},
"event_id": "WpmTgsNWUZ",
"required_power_level": 10,
"room_id": "!MkDbyRqnvTYnoxjLYx:localhost",
"state_key": "",
"ts": 1409665585188,
"type": "m.room.add_state_level",
"user_id": "@example:localhost"
},
{
"content": {
"level": 0
},
"event_id": "qUMBJyKsTQ",
"required_power_level": 10,
"room_id": "!MkDbyRqnvTYnoxjLYx:localhost",
"state_key": "",
"ts": 1409665585188,
"type": "m.room.send_event_level",
"user_id": "@example:localhost"
},
{
"content": {
"ban_level": 5,
"kick_level": 5
},
"event_id": "YAaDmKvoUW",
"required_power_level": 10,
"room_id": "!MkDbyRqnvTYnoxjLYx:localhost",
"state_key": "",
"ts": 1409665585188,
"type": "m.room.ops_levels",
"user_id": "@example:localhost"
},
{
"content": {
"avatar_url": null,
"displayname": null,
"membership": "join"
},
"event_id": "RJbPMtCutf",
"membership": "join",
"room_id": "!MkDbyRqnvTYnoxjLYx:localhost",
"state_key": "@example:localhost",
"ts": 1409665586730,
"type": "m.room.member",
"user_id": "@example:localhost"
},
{
"content": {
"body": "hello",
"hsob_ts": 1409665660439,
"msgtype": "m.text"
},
"event_id": "YUwRidLecu",
"room_id": "!MkDbyRqnvTYnoxjLYx:localhost",
"ts": 1409665660439,
"type": "m.room.message",
"user_id": "@example:localhost"
},
{
"content": {
"membership": "invite"
},
"event_id": "YjNuBKnPsb",
"membership": "invite",
"room_id": "!MkDbyRqnvTYnoxjLYx:localhost",
"state_key": "@myfriend:localhost",
"ts": 1409666426819,
"type": "m.room.member",
"user_id": "@example:localhost"
},
{
"content": {
"avatar_url": null,
"displayname": null,
"membership": "join",
"prev": "join"
},
"event_id": "KWwdDjNZnm",
"membership": "join",
"room_id": "!MkDbyRqnvTYnoxjLYx:localhost",
"state_key": "@example:localhost",
"ts": 1409666551582,
"type": "m.room.member",
"user_id": "@example:localhost"
},
{
"content": {
"avatar_url": null,
"displayname": null,
"membership": "join"
},
"event_id": "JFLVteSvQc",
"membership": "join",
"room_id": "!MkDbyRqnvTYnoxjLYx:localhost",
"state_key": "@example:localhost",
"ts": 1409666587265,
"type": "m.room.member",
"user_id": "@example:localhost"
}
],
"end": "s39_18_0",
"start": "t1-11_18_0"
},
"room_id": "!MkDbyRqnvTYnoxjLYx:localhost",
"state": [
{
"content": {
"creator": "@example:localhost"
},
"event_id": "dMUoqVTZca",
"required_power_level": 10,
"room_id": "!MkDbyRqnvTYnoxjLYx:localhost",
"state_key": "",
"ts": 1409665585188,
"type": "m.room.create",
"user_id": "@example:localhost"
},
{
"content": {
"@example:localhost": 10,
"default": 0
},
"event_id": "wAumPSTsWF",
"required_power_level": 10,
"room_id": "!MkDbyRqnvTYnoxjLYx:localhost",
"state_key": "",
"ts": 1409665585188,
"type": "m.room.power_levels",
"user_id": "@example:localhost"
},
{
"content": {
"join_rule": "public"
},
"event_id": "jrLVqKHKiI",
"required_power_level": 10,
"room_id": "!MkDbyRqnvTYnoxjLYx:localhost",
"state_key": "",
"ts": 1409665585188,
"type": "m.room.join_rules",
"user_id": "@example:localhost"
},
{
"content": {
"level": 10
},
"event_id": "WpmTgsNWUZ",
"required_power_level": 10,
"room_id": "!MkDbyRqnvTYnoxjLYx:localhost",
"state_key": "",
"ts": 1409665585188,
"type": "m.room.add_state_level",
"user_id": "@example:localhost"
},
{
"content": {
"level": 0
},
"event_id": "qUMBJyKsTQ",
"required_power_level": 10,
"room_id": "!MkDbyRqnvTYnoxjLYx:localhost",
"state_key": "",
"ts": 1409665585188,
"type": "m.room.send_event_level",
"user_id": "@example:localhost"
},
{
"content": {
"ban_level": 5,
"kick_level": 5
},
"event_id": "YAaDmKvoUW",
"required_power_level": 10,
"room_id": "!MkDbyRqnvTYnoxjLYx:localhost",
"state_key": "",
"ts": 1409665585188,
"type": "m.room.ops_levels",
"user_id": "@example:localhost"
},
{
"content": {
"membership": "invite"
},
"event_id": "YjNuBKnPsb",
"membership": "invite",
"room_id": "!MkDbyRqnvTYnoxjLYx:localhost",
"state_key": "@myfriend:localhost",
"ts": 1409666426819,
"type": "m.room.member",
"user_id": "@example:localhost"
},
{
"content": {
"avatar_url": null,
"displayname": null,
"membership": "join"
},
"event_id": "JFLVteSvQc",
"membership": "join",
"room_id": "!MkDbyRqnvTYnoxjLYx:localhost",
"state_key": "@example:localhost",
"ts": 1409666587265,
"type": "m.room.member",
"user_id": "@example:localhost"
}
]
}
]
}
This returns all the room information the user is invited/joined on, as well as
all of the presences relevant for these rooms. This can be a LOT of data. You
may just want the most recent event for each room. This can be achieved by
applying query parameters to ``limit`` this request::
curl -XGET "http://localhost:8008/_matrix/client/api/v1/initialSync?limit=1&access_token=YOUR_ACCESS_TOKEN"
{
"end": "s39_18_0",
"presence": [
{
"content": {
"last_active_ago": 1279484,
"user_id": "@example:localhost"
},
"type": "m.presence"
}
],
"rooms": [
{
"membership": "join",
"messages": {
"chunk": [
{
"content": {
"avatar_url": null,
"displayname": null,
"membership": "join"
},
"event_id": "JFLVteSvQc",
"membership": "join",
"room_id": "!MkDbyRqnvTYnoxjLYx:localhost",
"state_key": "@example:localhost",
"ts": 1409666587265,
"type": "m.room.member",
"user_id": "@example:localhost"
}
],
"end": "s39_18_0",
"start": "t10-30_18_0"
},
"room_id": "!MkDbyRqnvTYnoxjLYx:localhost",
"state": [
{
"content": {
"creator": "@example:localhost"
},
"event_id": "dMUoqVTZca",
"required_power_level": 10,
"room_id": "!MkDbyRqnvTYnoxjLYx:localhost",
"state_key": "",
"ts": 1409665585188,
"type": "m.room.create",
"user_id": "@example:localhost"
},
{
"content": {
"@example:localhost": 10,
"default": 0
},
"event_id": "wAumPSTsWF",
"required_power_level": 10,
"room_id": "!MkDbyRqnvTYnoxjLYx:localhost",
"state_key": "",
"ts": 1409665585188,
"type": "m.room.power_levels",
"user_id": "@example:localhost"
},
{
"content": {
"join_rule": "public"
},
"event_id": "jrLVqKHKiI",
"required_power_level": 10,
"room_id": "!MkDbyRqnvTYnoxjLYx:localhost",
"state_key": "",
"ts": 1409665585188,
"type": "m.room.join_rules",
"user_id": "@example:localhost"
},
{
"content": {
"level": 10
},
"event_id": "WpmTgsNWUZ",
"required_power_level": 10,
"room_id": "!MkDbyRqnvTYnoxjLYx:localhost",
"state_key": "",
"ts": 1409665585188,
"type": "m.room.add_state_level",
"user_id": "@example:localhost"
},
{
"content": {
"level": 0
},
"event_id": "qUMBJyKsTQ",
"required_power_level": 10,
"room_id": "!MkDbyRqnvTYnoxjLYx:localhost",
"state_key": "",
"ts": 1409665585188,
"type": "m.room.send_event_level",
"user_id": "@example:localhost"
},
{
"content": {
"ban_level": 5,
"kick_level": 5
},
"event_id": "YAaDmKvoUW",
"required_power_level": 10,
"room_id": "!MkDbyRqnvTYnoxjLYx:localhost",
"state_key": "",
"ts": 1409665585188,
"type": "m.room.ops_levels",
"user_id": "@example:localhost"
},
{
"content": {
"membership": "invite"
},
"event_id": "YjNuBKnPsb",
"membership": "invite",
"room_id": "!MkDbyRqnvTYnoxjLYx:localhost",
"state_key": "@myfriend:localhost",
"ts": 1409666426819,
"type": "m.room.member",
"user_id": "@example:localhost"
},
{
"content": {
"avatar_url": null,
"displayname": null,
"membership": "join"
},
"event_id": "JFLVteSvQc",
"membership": "join",
"room_id": "!MkDbyRqnvTYnoxjLYx:localhost",
"state_key": "@example:localhost",
"ts": 1409666587265,
"type": "m.room.member",
"user_id": "@example:localhost"
}
]
}
]
}
Getting live state
------------------
Once you know which rooms the client has previously interacted with, you need to
listen for incoming events. This can be done like so::
curl -XGET "http://localhost:8008/_matrix/client/api/v1/events?access_token=YOUR_ACCESS_TOKEN"
{
"chunk": [],
"end": "s39_18_0",
"start": "s39_18_0"
}
This will block waiting for an incoming event, timing out after several seconds.
Even if there are no new events (as in the example above), there will be some
pagination stream response keys. The client should make subsequent requests
using the value of the ``"end"`` key (in this case ``s39_18_0``) as the ``from``
query parameter e.g. ``http://localhost:8008/_matrix/client/api/v1/events?access
_token=YOUR_ACCESS_TOKEN&from=s39_18_0``. This value should be stored so when the
client reopens your app after a period of inactivity, you can resume from where
you got up to in the event stream. If it has been a long period of inactivity,
there may be LOTS of events waiting for the user. In this case, you may wish to
get all state instead and then resume getting live state from a newer end token.
NB: The timeout can be changed by adding a ``timeout`` query parameter, which is
in milliseconds. A timeout of 0 will not block.
Example application
-------------------
The following example demonstrates registration and login, live event streaming,
creating and joining rooms, sending messages, getting member lists and getting
historical messages for a room. This covers most functionality of a messaging
application.
.. NOTE::
`Try out the fiddle`__
.. __: http://jsfiddle.net/gh/get/jquery/1.8.3/matrix-org/matrix-doc/tree/master/supporting-docs/howtos/jsfiddles/example_app

125
supporting-docs/guides/2015-08-14-getting_involved.md Normal file
View file

@ -0,0 +1,125 @@
---
layout: post
title: Getting involved
categories: guides
---
# How can I get involved?
Matrix is an ecosystem consisting of several apps written by lots of people. We at Matrix.org have written one server and a few clients, and people in the community have also written several clients, servers, and Application Services. We are collecting [a list of all known Matrix-apps](https://matrix.org/blog/try-matrix-now/).
|
You have a few options when it comes to getting involved: if you just want to use Matrix, you can [register an account on a public server using a public webclient](#reg). If you have a virtual private server (VPS) or similar, you might want to [run a server and/or client yourself](#run). If you want to look under the hood, you can [checkout the code and modify it - or write your own client or server](#checkout). Or you can write an [Application Service](#as), for example a bridge to an existing ecosystem.
|
<a class="anchor" id="reg"></a>
## Access Matrix via a public webclient/server
The easiest thing to do if you want to just have a play, is to use our reference webclient and create a user on the matrix.org homeserver. You do that by visiting http://matrix.org/beta/, selecting "Create account" and choosing your userID and password on the next page. You can also add your email, but this is optional (adding it will make it easier for your friends to find your Matrix user in the future).
|
At the bottom of the account creation page, you can pick the homeserver and identity server you want to use. In this case, we are using matrix.org's homeserver, so just leave it as-is. Your full Matrix-userID will be formed partly from the hostname your server is running as, and partly from an userID you specify when you create the account. For example, if you put bob as your userID, your full Matrix-userID will be @bob:matrix.org ("bob on matrix.org").
|
You can use multiple clients with the same user, so you might want to also look at our [Android](https://github.com/matrix-org/matrix-android-console) or [iOS](https://github.com/matrix-org/matrix-ios-console) clients for your mobile phone!
|
<a class="anchor" id="run"></a>
## Run a server and/or client yourself
You can clone our open source projects and run clients and servers yourself. Here is how:
### Running your own client:
You can run your own Matrix client; there are [numerous clients available](https://matrix.org/blog/try-matrix-now/). You can take Matrix.org's [reference client](https://github.com/matrix-org/matrix-angular-sdk) and use it as-is - or modify it any way you want! Since it's written in JavaScript, running a client is [really easy](https://github.com/matrix-org/matrix-angular-sdk#running)!
|
### Running your own homeserver:
One of the core features of Matrix is that anyone can run a homeserver and join the federated network on equal terms (there is no hierarchy). If you want to set up your own homeserver, please see the relevant docs of the server you want to run. If you want to run Matrix.org's reference homeserver, please consult the [readme of the Synapse project](https://github.com/matrix-org/synapse/blob/master/README.rst).
|
Note that Synapse comes with a bundled Matrix.org webclient - but you can tell it to use your [development checkout snapshot instead](https://github.com/matrix-org/matrix-angular-sdk#matrix-angular-sdk) (or to not run a webclient at all via the "web_client: false" config option).
|
<a class="anchor" id="checkout"></a>
## Checkout our code - or write your own
As described above, you can clone our code and [run a server and/or client yourself](#run). Infact, all the code that we at Matrix.org write is available from [our github](http://github.com/matrix-org) - and other servers and clients may also be open sourced - see [our list of all known Matrix-apps](https://matrix.org/blog/try-matrix-now/).
|
You can also implement your own client or server - after all, Matrix is at its core "just" a specification of a protocol.
|
### Write your own client:
The [client-server API spec](http://matrix.org/docs/howtos/client-server.html) describes what API calls are available to clients, but a quick step-by-step guide would include:
|
1. Get a user either by registering your user in an existing client or running the [new-user script](https://github.com/matrix-org/synapse/blob/master/scripts/register_new_matrix_user) if you are running your own Synapse homeserver.
2. Assuming the homeserver you are using allows logins by password, log in via the login API:
```
curl -XPOST -d '{"type":"m.login.password", "user":"example", "password":"wordpass"}' "http://localhost:8008/_matrix/client/api/v1/login"
```
3. If successful, this returns the following, including an `access_token`:
{
"access_token": "QGV4YW1wbGU6bG9jYWxob3N0.vRDLTgxefmKWQEtgGd",
"home_server": "localhost",
"user_id": "@example:localhost"
}
4. This ``access_token`` will be used for authentication for the rest of your API calls. Potentially the next step you want is to make a call to the initialSync API and get the last x events from each room your user is in (via the limit parameter):
```
curl -XGET "http://localhost:8008/_matrix/client/api/v1/initialSync?limit=1&access_token=YOUR_ACCESS_TOKEN"
```
5. In addition to the last x events for all the rooms the user is interested in, this returns all the presences relevant for these rooms. Once you know which rooms the client has previously interacted with, you need to listen for incoming events. This can be done like so:
```
curl -XGET "http://localhost:8008/_matrix/client/api/v1/events?access_token=YOUR_ACCESS_TOKEN"
```
6. This request will block waiting for an incoming event, timing out after several seconds if there is no event, returning something like this:
{
"chunk": [],
"end": "s39_18_0",
"start": "s39_18_0"
}
7. Even if there are no new events (as in the example above), there will be some pagination stream response keys. The client should make subsequent requests using the value of the "end" key (in this case s39_18_0) as the from query parameter e.g.
```
http://localhost:8008/_matrix/client/api/v1/events?access _token=YOUR_ACCESS_TOKEN&from=s39_18_0
```
8. This ensures that you only get new events. Now you have initial rooms and presence, and a stream of events - a good client should be able to process all these events and present them to the user. And potentially you might want to add functionality to generate events as well (such as messages from the user, for example) - again please consult the [client-server API spec](http://matrix.org/docs/howtos/client-server.html)!
|
### Write your own server:
We are still working on the server-server spec, so the best thing to do if you are interested in writing a server, is to come talk to us in [#matrix:matrix.org](https://matrix.org/beta/#/room/%23matrix:matrix.org).
If you are interested in how federation works, please see the [federation API chapter in the spec](http://matrix.org/docs/spec/#federation-api).
|
<a class="anchor" id="as"></a>
## Write an Application Service:
Information about Application services and how they can be used can be found in the [Application services](./application_services.html) document! (This is based on Kegan's excellent blog posts, but now lives here so it can be kept up-to-date!)

593
supporting-docs/guides/2015-08-19-faq.md Normal file
View file

@ -0,0 +1,593 @@
---
layout: post
title: FAQ
date: 2015-08-19 16:58:07
categories: guides
---
<link href="/docs/guides/css/faq.css" type="text/css" rel="stylesheet" />
# FAQ
{:.no_toc}
Categories
----------
{:.no_toc}
[General](#general)
[Quick Start](#quick-start)
[Standard](#standard)
[Implementations](#implementations)
|
FAQ Content
-----------
{:.no_toc}
* TOC
{:toc .toc}
### General
##### What is Matrix?
Matrix is an open standard for interoperable, decentralised,
real-time communication over IP. It can be used to power Instant
Messaging, VoIP/WebRTC signalling, Internet of Things communication - or anywhere
you need a standard HTTP API for publishing and subscribing to
data whilst tracking the conversation history.
|
Matrix defines the standard, and provides open source reference implementations
of Matrix-compatible Servers, Clients, Client SDKs and Application Services
to help you create new communication solutions or extend the capabilities
and reach of existing ones.
##### What is Matrix's Mission?
Matrix's initial goal is to fix the problem of fragmented IP communications:
letting users message and call each other without having to care what app
the other user is on - making it as easy as sending an email.
|
The longer term goal is for Matrix to act as a generic HTTP messaging and data
synchronisation system for the whole web - allowing people, services and devices
to easily communicate with each other, empowering users to own and control their
data and select the services and vendors they want to use.
##### What does Matrix provide?
Matrix provides:
- [Open Standard](/docs/spec) HTTP APIs for transferring JSON messages (e.g. instant messages, WebRTC signalling), including:
- [Client\<-\>Server API](/docs/spec#client-server-api-v1) - defines how Matrix compatible clients communicate with Matrix homeservers.
- [Server\<-\>Server API](/docs/spec#federation-api) - defines how Matrix homeservers exchange messages and synchronise history with each other.
- [Application Service API](/docs/spec/#application-service-api) - defines how to extend the functionality of Matrix with 'integrations' and bridge to other networks.
- [Modules](/docs/spec/#modules) - specifies features that must be implemented by particular classes of clients.
- Open source reference implementations of:
- Clients (Web (React), iOS, Android)
- Client SDKs (Javascript, Web (React), iOS, Android)
- Homeservers (Synapse)
- Application Services (bridges to IRC, Slack, Skype, Lync and more...)
- The actual ecosystem and community of everyone running Matrix servers and services
- Loads of 3rd party contributions of clients, SDKs, servers and services.
You can find the full list of Matrix enabled projects at https://matrix.org/blog/try-matrix-now.
##### What does this mean for users?
The aim is to provide an analogous ecosystem to email - one where you
can communicate with pretty much anyone, without caring what app or
server they are using, using whichever app & server you chose to use,
and use a neutral identity system like an e-mail address or phone
number to discover people to talk to.
##### What kind of company is Matrix.org?
Matrix is an open initiative which acts as a neutral custodian of the
Matrix standard. It's not actually incorporated anywhere at the moment
but we are looking at the best legal structure for the future (and as
of October 2015 we have hopefully found one). Whatever the legal
structure, we are committed to keeping the Matrix project open.
##### Who is funding Matrix.org?
Most of the current core contributors to Matrix work at
[Amdocs](http://amdocs.com), who have kindly given us permission to work
on Matrix as an independent non-profit initiative. Other contributors
are funded by their own employers or donate their own time to the project.
##### Who is building Matrix?
The core team is ~10 people with extensive experience in building custom
VoIP and Messaging apps for mobile network operators. Most of us have
day jobs at [Amdocs](http://amdocs.com) or [OpenMarket](http://openmarket.com),
but there are an increasing number of contributors from other companies and
folks all over the internet.
##### Why are you called Matrix?
We are called Matrix because we provide a structure in which all
communication can be matrixed together.
|
No, it's nothing to do with the film (although you could go and build virtual
worlds on top of Matrix if you wanted :)
##### Why have you released this as open source?
We believe that any open standard defining interoperable communication
needs to be justified, demonstrated and validated with transparent open
source implementations. For Matrix to achieve its mission of making all
communications services interoperable we believe it needs to be truly
open, giving people access to take all the code we produce and to use
and build on top of it.
##### What do you mean by open?
Matrix is an open standard, meaning that we have freely published the
details for how to communicate interoperably using the Matrix set of
HTTP APIs. We encourage anyone and everyone to use the APIs and build
their own projects which implement them and so benefit from
interoperability with the rest of the Matrix ecosystem. We also
ensure the standard is not encumbered by any known patent licensing
requirements.
|
Matrix is also open source, meaning that we have released the source
code of the reference servers, clients and services to the public domain
under the [Apache Licence v2](http://www.apache.org/licenses/LICENSE-2.0.html), to
encourage anyone and everyone to run their own servers and clients, and
enhance them and contribute their enhancements as they see fit.
##### What does federated mean?
Federation allows separate deployments of a communication service to
communicate with each other - for instance a mail server run by Google
federates with a mail server run by Microsoft when you send email from
@gmail.com to @hotmail.com.
|
Federation is different to interoperability, as interoperable clients
may simply be running on the same deployment - whereas in federation the
deployments themselves are exchanging data in a compatible manner.
|
Matrix provides open federation - meaning that anyone on the internet
can join into the Matrix ecosystem by deploying their own server.
##### How is this like e-mail?
When email was first set up in the early ‘80s, companies like Compuserve
and AT&T and Sprint set up isolated email communities which only allowed
you to exchange mail with users on the same system.  If you got your
email from one service and your friend from another, then you couldn't
message each other.  This is basically the situation we're in today with
VoIP and IM.
##### Why has no-one done this before?
There have been several attempts before including SIP, XMPP and RCS.
 All of these have had some level of success, but many different
technological/usability/economic factors have ended up limiting their
success. Unfortunately, we've not ended up in a world where everyone
has a SIP URI or Jabber ID on their business card, or a phone that
actually uses RCS.
##### What is the difference between Matrix and IRC?
We love IRC.  In fact, as of today the core Matrix team still uses it as
our primary communication tool. Between us we've written IRCds, IRC bots
and admined dreamforge, UnrealIRCd, epona, ircservices and several
others. That said, it has some limitations that Matrix seeks to improve
on:
- Text only
- No history
- No multiple-device support
- No presence support
- Fragmented identity model
- No open federation
- No standard APIs, just a rather limited TCP line protocol
- Non-standardised federation protocol
- No built-in end-to-end encryption
- Disruptive net-splits
- Non-extensible
[IRCv3](http://ircv3.net) exists and is addressing some of issues;
this is great news and we wish them well. It's almost a contradiction
in terms to get competitive between openly interoperable communication
projects - we look forward to increasing the richness of Matrix\<-\>IRC
bridges as the project progresses.
##### What is the difference between Matrix and XMPP?
The Matrix team used XMPP (Openfire, ejabberd, spectrum, asmack,
XMPPFramework) for IM before starting to experiment with open HTTP APIs
as an alternative in around 2012. The main issues with XMPP that
drove us in this direction were:
- Not particularly web-friendly - you can't easily speak XMPP from a
web browser. (N.B. Nowadays you have options like XMPP-FTW and
Stanza.io that help loads with letting browsers talk XMPP).
- Single logical server per MUC is a single point of control and
availability. (MUCs can be distributed over multiple physical
servers, but they still sit behind a single logical JID and domain.
FMUC improves this with a similar approach to Matrix, but as of Oct
2015 there are no open source implementations.)
- History synchronisation is very much a second class citizen feature
- Stanzas aren't framed or reliably delivered without extensions. (See
[wiki.xmpp.org](http://wiki.xmpp.org/web/Myths#Myth_Four:_XMPP_is_unreliable_without_a_bunch_of_extensions.)
for an XMPP take on this)
- Multiple device support is limited. (Apparently Carbons and MAM help
with this)
- Baseline feature set is so minimal that fragmentation of features
between clients and servers is common, especially as interoperability
profiles for features have fallen behind (as of July 2015)
- No strong identity system (i.e. no standard E2E PKI, unless you
count X.509 certs, which [are
questionable](http://www.thoughtcrime.org/blog/ssl-and-the-future-of-authenticity/))
- Not particularly well designed for mobile use cases: push;
bandwidth-efficient transports. (Since the time of writing a [Push
XEP has appeared](http://xmpp.org/extensions/xep-0357.html), and
[wiki.xmpp.org](http://wiki.xmpp.org/web/Myths#Myth_Three:_It.27s_too_bandwidth-inefficient_for_mobile.)
claims that XMPP runs "fine" over a 9600bps + 30s latency link.)
The whole subject of XMPP vs Matrix seems to bring out the worst in
people. Rather than fighting over which open interoperable communication
standard works the best, we should just collaborate and bridge everything
together. The more federation and interoperability the better.
|
We think of Matrix and XMPP as being quite different; at its core
Matrix can be thought of as an eventually consistent global JSON db with
an HTTP API and pubsub semantics - whilst XMPP can be thought of as a
message passing protocol. You can use them both to build chat systems;
you can use them both to build pubsub systems; each comes with different
tradeoffs. Matrix has a deliberately extensive 'kitchen sink' baseline
of functionality; XMPP has a deliberately minimal baseline set of
functionality. If XMPP does what you need it to do, then we're genuinely
happy for you :) Meanwhile, rather than competing, an XMPP Bridge like
[Skaverat's xmpptrix beta](https://github.com/SkaveRat/xmpptrix) or
[jfred's matrix-xmpp-bridge](https://github.com/jfrederickson/matrix-xmpp-bridge)
or Matrix.org's own [matrix-appservice-purple](https://github.com/matrix-org/matrix-appservice-purple)
has potential to let both environments coexist and make the most of each
other's benefits.
##### What is the difference between Matrix and PSYC?
PSYC is a open federated messaging protocol loosely inspired by IRC.  In
version 1 it was a standalone protocol, and in version 2 it is being
reutilised as a messaging layer on top of GNUnet.  We honestly don't
know that much about it, beyond trying to use psycd as an XMPP\<-\>IRC
bridge in 2010. Matrix differentiates primarily by providing simple HTTP
APIs rather than the more exotic compact line protocol in PSYC v1 or the
comprehensive GNUnet stack in v2, and Matrix focuses more on decentralised
conversation history rather than just decentralised chat servers.
On the other hand, Matrix doesn't provide the metadata protection
guarantees that GNUnet/PSYC aims for.
|
See [http://about.psyc.eu/Matrix](http://about.psyc.eu/Matrix) for
PSYC's views on Matrix.
##### What is the difference between Matrix and Tox?
Tox.im looks to be a very cool clone of Skype - a fully decentralised
peer-to-peer network.  Matrix is deliberately not a 'pure' peer-to-peer
system; instead each user has a well-defined homeserver which stores
his data and that he can depend upon.  Matrix provides HTTP APIs;
Tox.im provides C APIs. As of October 2015 Tox doesn't seem to have an
answer yet for decentralised conversation history storage.
##### How does Matrix compare with something like Trillian or Pidgin?
Trillian and Pidgin and similar aggregating IM clients merge all your IM
activity into a single app.  However, your history and
identity is still fragmented across the networks.  People can't find you
easily, and your history is fragmented (other than on the device
where the client runs).   And rather than being able to chose the right
app for the job when communicating with people, you are pushed towards
relying on a specific aggregation app.
Matrix lets you get the best of both worlds by linking to all the
different networks (XMPP, AIM, ICQ, Lync, Skype etc) on the serverside,
using bridges which can be run by anyone. Matrix then provides a simple
standard HTTP API to access any of these networks, and lets you choose
whichever client you prefer (either as a 'native' Matrix client or using
a non-Matrix client from one of the networks which has been bridged in).
##### What Matrix compliant apps are there?
Quite a few, ranging from the glossy mass-market to the geeky command-line. There's even an emacs macro. Check out [https://matrix.org/blog/try-matrix-now] for the current
list of Matrix enabled projects.
##### What bridges to other networks are available?
The number of 'bridges' which integrate existing communication networks into
Matrix are growing on a daily basis - both written by the Matrix core team
and contributed by the wider community. The full list can be seen at
https://matrix.org/blog/try-matrix-now, but the core ones as of Oct 2015 include:
* [matrix-appservice-irc](https://github.com/matrix-org/matrix-appservice-irc) - an increasingly comprehensive Matrix\<-\>IRC bridge
* [matrix-appservice-verto](https://github.com/matrix-org/matrix-appservice-verto) - links from Matrix to FreeSWITCH via the Verto protocol
* [matrix-appservice-slack](https://github.com/matrix-org/matrix-appservice-slack) - a basic bridge to Slack
* [matrix-appservice-purple](https://github.com/matrix-org/matrix-appservice-purple) - lets you access any of the 20+ protocols supported by
[libpurple](https://developer.pidgin.im/wiki/WhatIsLibpurple), including
Skype, Lync, XMPP, etc)
* [matrix-appservice-bridge](https://github.com/matrix-org/matrix-appservice-bridge) - a general NodeJS framework for writing bridges
Writing new bridges is incredibly fun and easy - see the [matrix-appservice-bridge HOWTO](https://github.com/matrix-org/matrix-appservice-bridge/blob/master/HOWTO.md)
for an example of how to write a fully functional Slack bridge in less than 100 lines of code!
##### Why do you think existing apps will ever join this officially?
We firmly believe it is what is right for the consumer. As people begin
to use interoperable communications tools, service providers will see the
benefit and compete on quality of service, security and features rather
than relying on locking people into their walled garden. We believe as
soon as users see the availability and benefits of interoperable
services they will demand it.
##### Why aren't you doing this through the IETF? or W3C? or 3GPP?
We do recognise the advantages of working with existing standards
bodies. We have been focused on writing code and getting it out, and the standard has been evolving rapidly since initial release in September 2014.
Once the standard has matured sufficiently it may well be appropriate to work with an official
standard body to maintain it going forwards.
### Quick Start
##### How do I get an account and get started?
The quickest way is to pick a client from https://matrix.org/blog/try-matrix-now and sign up.
Please note that you can point clients to access any homeserver - you don't have to use matrix.org,
although as of day 1, matrix.org is the only communal homeserver
available.
##### What can I actually do with this?
A typical client provides a simple chatroom interface to Matrix -
letting the user interact with users and rooms anywhere within the
Matrix federation.  Text and image messages are supported, and basic
voice-only VoIP calling via WebRTC is supported in one-to-one rooms.
(As of October 2015, experimental multi-way calling is also available
on Vector.im).
##### How do I connect my homeserver to the public Matrix network?
See
[http://github.com/matrix-org/synapse](http://github.com/matrix-org/synapse)
for details
##### How do I Matrix-enable my existing app?
If your app doesn't have any communication capability already, you'll want
to use one of the Matrix client SDKs to add it in. These come in different
levels of sophistication - ranging from a simple HTTP API wrapper (like matrix-js-sdk, matrix-ios-sdk or matrix-android-sdk)
through to reusable UI components (like matrix-react-sdk and matrix-ios-kit). Pick
the one for your platform, or a 3rd party one if none of the above work for you,
and get plugging it in. You'll probably also want to read the [Client-Server API
HOWTO](http://matrix.org/docs/howtos/client-server.html) too.
If you already have communication infrastructure set up (XMPP, custom HTTP, or whatever),
then you'll want to run a bridge to expose it to the wider Matrix ecosystem.
See [matrix-appservice-bridge HOWTO](https://github.com/matrix-org/matrix-appservice-bridge/blob/master/HOWTO.md) for a
guide of how to write bridges using the matrix-appservice-bridge framework, or co-opt one
from the list at https://matrix.org/blog/try-matrix-now.
[Application Service API](/docs/spec/#application-service-api) gives the details of the API
that bridges have to implement.
##### How can I write a client on Matrix?
See the [Client-Server API
HOWTO](http://matrix.org/docs/howtos/client-server.html) and the [API
docs](/docs/api) and [the Spec](/docs/spec) for all the details you need
to write a client.
##### How can I help out with this?
Come say hi on #matrix:matrix.org! Install synapse and tell us how you get on. Critique the spec.  Write
clients. Write bridges! Run bridges! Nose around in [Jira](https://matrix.org/jira) and
send us some pull requests on github to fix some bugs or add some features! You could even
try to write a homeserver (but be warned, Matrix's architecture makes homeservers orders of
magnitude harder than clients or bridges.)
See [CONTRIBUTING.rst](http://github.com/matrix-org/synapse/tree/master/CONTRIBUTING.rst) for
full details on how to contribute to the project. All are welcome!
##### Where can I get support?
\#matrix:matrix.org aka \#matrix on irc.freenode.is your best bet.
##### How do I register custom matrix event types?
We're not yet managing a registry of custom matrix event types.  If you
have any particularly good ones you want to tell the world about, please
use the [mailing list](/mailman/listinfo/matrix-users) for now.
##### How mature is this?
We started working on Matrix in July 2014, and opened it to the
public in September 2014. We got all the core features in place in December 2014
and entered beta, and since then have been iterating away on the beta refining the
architecture and APIs, fixing bugs and scalability, and adding new features, clients,
bridges etc.
As of October 2015 (synapse 0.10) it's good for serious experimentation and
non-production services and can absolutely be used in the real world. However, we're
still in beta and we'll want to freeze the spec and implement clustering and other
nice features before we really declare it ready for production.
### Standard
##### What is a client?
Users in Matrix use one or more clients to communicate. This could be any combination of a web client, a command line client, a mobile client - or embedded clients built into existing apps. It could even be a piece of hardware (e.g. a drone) that is Matrix enabled.
##### Can I use Matrix without installing a Matrix client?
Sure. An ever increasing number of protocols are being bridged into Matrix, so if you use something like IRC on Freenode you may well be indirectly benefiting from Matrix, as others may be connected into the IRC channel via Matrix.
##### What is a home server?
A user's clients connect to a single homeserver, which stores the communication history and account information for that user, and shares data with the wider Matrix ecosystem by synchronising communication history with other homeservers.
##### What is an identity server?
Users in Matrix are identified internally via their matrix user ID (MXID). However, existing 3rd party ID (3PID) namespaces such as email addresses or phone numbers should be used publically to identify Matrix users, at least for invitation purposes. A Matrix "Identity" describes both the user ID and any other existing IDs from third party namespaces linked to their account.
|
Matrix users can link third-party IDs (3PIDs) to their user ID. Linking 3PIDs creates a mapping from a 3PID to a user ID. This mapping can then be used by Matrix users in order to discover the MXIDs of their contacts.
|
In order to ensure that the mapping from 3PID to user ID is genuine, a globally federated cluster of trusted "Identity Servers" (IS) are used to verify the 3PID and persist and replicate the mappings.
Usage of an IS is not required in order for a client application to be part of the Matrix ecosystem. However, without one clients will not be able to look up user IDs using 3PIDs.
|
The precise architecture of identity servers is currently in flux and subject to change as we work to fully decentralise them.
##### Where do my conversations get stored?
Each homeserver stores the communication history and account information for all of its clients, and shares data with the wider Matrix ecosystem by synchronising communication history with other homeservers and their clients. Clients typically communicate with each other by emitting events in the context of a virtual room. Room data is replicated across all of the homeservers *whose users are participating in a given room*.
##### What is a 3PID?
Third-party IDs (3PIDs) are IDs from other systems or contexts, such as email addresses, social network accounts and phone numbers.
##### How do you do VoIP calls on Matrix?
Voice (and video) over Matrix uses the WebRTC 1.0 standard to transfer call media (i.e. the actual voice and video traffic). Matrix is used to signal the establishment and termination of the call by sending call events, like any other event. Currently calls are only supported in rooms with exactly two participants - however, one of those participants may be a conferencing bridge. We're looking at better ways to do group calling.
##### Can I log into other homeservers with my username and password?
Currently, no. We are looking at options for decentralising or migrating user accounts between multiple servers, and might add this feature at a later stage.
##### Why Apache Licence?
The Apache Licence is a permissive licence. We want the Matrix protocol itself to be free and open, but people are free to create both free and commercial apps and services that uses the protocol. In our opinion, any Matrix-service only enhances the Matrix ecosystem.
##### Can I write a Matrix homeserver?
Yes. Matrix is just a spec, so implementations of the spec are very welcome! It should be noted that as of October 2015, changes are still being made to the spec, so if you want to write a Matrix homeserver, it is strongly recommended that you chat to the Matrix.org devs in #matrix:matrix.org first! You can also read about the [Federation API here]( https://github.com/matrix-org/matrix-doc/blob/master/specification/30_server_server_api.rst).
##### How secure is this?
Server-server traffic is mandatorily TLS from the outset. Server-client traffic mandates transport layer encryption other than for tinkering. Servers maintain a public/private key pair, and sign the integrity of all messages in the context of the historical conversation, preventing tampering. Server keys are distributed using a PERSPECTIVES-style system.
End-to-end encryption is coming shortly to clients for both 1:1 and group chats to protect user data stored on servers, using the [Olm](https://matrix.org/git/olm) cryptographic ratchet implementation. As of October 2015 this is blocked on implementing the necessary key distribution and fingerprint management.
Privacy of metadata is not currently protected from server administrators - a malicious homeserver administrator can see who is talking to who and when, but not what is being said (once E2E encryption is enabled). See [this presentation from Jardin Entropique](http://matrix.org/~matthew/2015-06-26%20Matrix%20Jardin%20Entropique.pdf) for a more comprehensive discussion of privacy in Matrix.
### Implementations
##### What is Synapse?
Synapse is a reference "homeserver" implementation of Matrix from the core development team at matrix.org, written in Python 2/Twisted. It is intended to showcase the concept of Matrix and let folks see the spec in the context of a codebase and let you run your own homeserver and generally help bootstrap the ecosystem.
##### How do I join the global Matrix federation?
You can download and run one of the available Matrix servers - please see [this guide](http://matrix.org/docs/guides/getting_involved.html#run) for details!
##### What ports do I have to open up to join the global Matrix federation?
We recommend servers use port 8448 for server\<-\>server HTTPS traffic. Look at ["Setting up Federation"](https://github.com/matrix-org/synapse#setting-up-federation) in the Synapse readme file for details.
Client\<-\>Server traffic can talk directly to Synapse via port 8448, but as by default Synapse creates a self-signed TLS certificate this can cause problems for clients which can't easily trust self-signed certificates (e.g. most web browsers). Instead, you can proxy access to Synapse's HTTP listener on port 8008 via an existing HTTPS proxy with a valid certificate (e.g. an nginx listening on port 443), or you can point Synapse at a valid X.509 signed TLS certificate. In future, Synapse will probably use letsencrypt to autogenerate valid certificates rather than self-signed ones during installation, simplifying this process enormously.
You can also put Synapse entirely behind an existing TLS load balancer and not expose port 8448 at all. In this situation, Synapse will need to be configured to share the same *public* TLS certificate as the load balancer (as Synapse uses the public certificate for identity in other areas too, and it has to match the certificate that other servers see when they connect).
##### How do I run my own homeserver?
Follow the instructions for the homeserver you want to run. If you want to run Synapse, the reference homeserver from Matrix.org, follow [these instructions](https://github.com/matrix-org/synapse#synapse-installation).
##### Can I run my own identity server?
Yes - the reference implementation is
[sydent](https://github.com/matrix-org/sydent) and you can run your own
ID server cluster that tracks 3rd party to Matrix ID mappings. This won't be very useful right now, though, and we don't recommend it.
If you want your server to participate in the global replicated Matrix ID
service then please get in touch with us. Meanwhile, we are looking at
ways of decentralising the 'official' Matrix identity service so that
identity servers are 100% decentralised and can openly federate with
each other. **N.B. that you can use Matrix without ever using the
identity service - it exists only to map 3rd party IDs (e.g. email
addresses) to matrix IDs to aid user discovery**.
##### What are Synapse's platform requirements?
Synapse will use as much RAM as you give it in order to cache conversations in RAM to avoid hitting the database. For small deployments (<50 active users) around 512MB of RAM is probably okay. You can configure the amount of RAM used by synapse with the event_cache_size config parameter - the more events in the cache, the more RAM required. Synapse itself requires relatively little diskspace other than for logging (which as of October 2015 is quite verbose for debugging purposes), but as it caches the content of all the file attachments (images, videos etc) viewed by its users, you may need to size storage appropriately. Synapse is currently effectively single threaded, and will never use more than 1 core.
|
For better performance, one should back Synapse with a Postgres database rather than the default SQLite - see https://github.com/matrix-org/synapse/tree/master/README.rst#using-postgresql for details.
##### Why is Synapse in Python/Twisted?
This is because both provide a mature and well known event-driven async IO framework for writing serverside code. Whilst this has been okay for our initial experimentation and proof of concept, it's likely that future homeserver work will be written in a more strongly typed language (e.g. Go).
##### Why aren't you using an ORM layer like SqlAlchemy in Synapse?
Synapse is *very* database dependent (as of Oct 2015; this is improving in the near future however), and we like having the flexibility to sculpt our own queries.
##### Where can I find a mobile app?
The "Matrix Console" reference apps (ugly, geeky and powerful - intended for early adopter powerusers) can be downloaded from the [Google Play store](https://play.google.com/store/apps/details?id=org.matrix.androidsdk.alpha)
and [Apple store](https://itunes.apple.com/gb/app/matrix-console/id970074271).
|
For the Android app, you can also install the latest development version
built by [Jenkins](http://www.matrix.org/jenkins/job/AndroidConsoleDevelop/lastBuild/artifact/console/build/outputs/apk/console-alpha-debug.apk).
##### Where can I find a web app?
As of Oct 2015, the best web app options are to use [Vector.im](https://vector.im) - a glossy web client written on top of [matrix-react-sdk](https://github.com/matrix-org/matrix-react-sdk), or the original [AngularJS based client](https://matrix.org/beta), which has serious performance problems and is not currently being maintained. In future a "Matrix Console" reference web app built on matrix-react-sdk will be released by matrix.org to complement the mobile apps above.
|
### QUESTIONS TO BE ANSWERED!
This FAQ is a constant work in progress - patches and pull requests are *very* welcome to help us improve it. Some of the frequent questions where we need to write an answer include:
* How do I rename servers?
* How do I change the TLS key of my server?
* How do I maintain my synapse's DB (e.g. prune old conversations)?
* How do I maintain my synapse's content repository (e.g. prune old content)?
* What are redactions?
* Why is the spec so big, especially relative to the XMPP baseline spec?
* How do I contribute to the spec?
* What is the privacy policy on Matrix.org?
* How precisely does E2E work?
* How does Matrix actually work architecturally?
* What IOT use cases are there for Matrix?
* Why is are the Matrix reference implementations written in so many different languages?
* How does push work?
* What's on the roadmap?
* How can I use Matrix to talk on Freenode or other IRC networks?
* Where can I learn more about Matrix? (link to PDFs of other presentations etc)
* Why HTTP? Doesn't HTTP suck?
* Why don't you use websockets?
* Why is synapse so resource intensive immediately after federating for the first time?
* \[your question goes here...\]
|
Any other questions? Please contact us in
[\#matrix:matrix.org](https://matrix.org/beta/#/room/%23matrix:matrix.org) or the [mailing
lists](/mailman/listinfo/matrix-users)!

143
supporting-docs/guides/2015-08-21-application_services.md Normal file
View file

@ -0,0 +1,143 @@
---
layout: post
title: Application services
categories: guides
---
# Application services
Application services are distinct modules which which sit alongside a home server providing arbitrary extensible functionality decoupled from the home server implementation. Just like the rest of Matrix, they communicate via HTTP using JSON. Application services function in a very similar way to traditional clients, but they are given much more power than a normal client. They can reserve entire namespaces of room aliases and user IDs for their own purposes. They can silently monitor events in rooms, or any events directed at any user ID. This power allows application services to have extremely useful abilities which overall enhance the end user experience.
|
One of the main use cases for application services is protocol bridges. Our Matrix server on Matrix.org links in to various IRC channels and networks. This functionality was initially implemented as a simple bot which resided as a user on the Matrix rooms we wanted to link to freenode channels (#matrix, #matrix-dev, #openwebrtc and #vuc etc). There was nothing special about this bot; it is just treated as a client. However, as we started to rely on it more and more though, we realised that there were things that were impossible for simple client-side bots to do by themselves - for example, the bot could not reserve the virtual user IDs it wanted to create, and could not lazily bridge arbitrary IRC rooms on-the-fly - and this spurred the development of Application Services.
|
### Some of the features of the IRC application service we have since implemented include:
- Specific channel-to-matrix room bridging : This is what the original IRC bot did. You can specify specific channels and specific room IDs, and messages will be bridged.
- Dynamic channel-to-matrix room bridging> : This allows Matrix users to join any channel on an IRC network, rather than being forced to use one of the specific channels configured.
- Two-way PM support : IRC users can PM the virtual "M-" users and private Matrix rooms will be created. Likewise, Matrix users can invite the virtual "@irc_Nick:domain" user IDs to a room and a PM to the IRC nick will be made.
- IRC nick changing support: Matrix users are no longer forced to use "M-" nicks and can change them by sending "!nick" messages directly to the bridge.
- Ident support: This allows usernames to be authenticated for virtual IRC clients, which means IRC bans can be targeted at the Matrix user rather than the entire application service.
|
### The use of the Application Services API means:
- The bot can reserve user IDs. This prevents humans from registering for @irc_... user IDs which would then clash with the operation of the bot.
- The bot can reserve room aliases. This prevents humans from register for #irc_... aliases which would then clash with the operation of the bot.
- The bot can trivially manage hundreds of users. Events are pushed to the application service directly. If you tried to do this as a client-side bot, you would need one event stream connection per virtual user.
- The bot can lazily create rooms on demand. This means Matrix users can join non-existent room aliases and have the application service quickly track an IRC channel and create a room with that alias, allowing the join request to succeed.
|
### Implementation details:
- Written in Node.js, designed to be run using <code>forever</code>.
- Built on the generic <a href="http://github.com/matrix-org/matrix-appservice-node">matrix-appservice-node</a> framework.
- Supports sending metrics in statsd format.
- Uses matrix-appservice-node to provide a standardised interface when writing application services, rather than an explicit web framework (though under the hood matrix-appservice-node is using Express).
|
At present, the IRC application service is in beta, and is being run on #matrix and #matrix-dev. If you want to give it a go, <a title="Matrix-IRC Application Service" href="https://github.com/matrix-org/matrix-appservice-irc">check it out on Github</a>!
|
# What Application services can do for you
Application services have enormous potential for creating new and exciting ways to transform and enhance the core Matrix protocol. For example, you could aggregate information from multiple rooms into a summary room, or create throwaway virtual user accounts to proxy messages for a fixed user ID on-the-fly. As you may expect, all of this power assumes a high degree of trust between application services and home servers. Only home server admins can allow an application service to link up with their home server, and the application service is in no way federated to other home servers. You can think of application services as additional logic on the home server itself, without messing around with the book-keeping that home servers have to do. This makes adding useful functionality very easy.
|
### Example
The application service (AS) API itself uses webhooks to communicate from the home server to the AS:
- Room Alias Query API : The home server hits a URL on your application server to see if a room alias exists.
- User Query API : The home server hits a URL on your application server to see if a user ID exists.
- Push API : The home server hits a URL on your application server to notify you of new events for your users and rooms.
A very basic application service may want to log all messages in rooms which have an alias starting with "#logged_" (side note: logging won't work if these rooms are using end-to-end encryption).
Here's an example of a very basic application service using Python (with Flask and Requests) which logs room activity:
# app_service.py:
import json, requests  # we will use this later
from flask import Flask, jsonify, request
app = Flask(__name__)
@app.route("/transactions/&lt;transaction&gt;", methods=["PUT"])
def on_receive_events(transaction):
events = request.get_json()["events"]
for event in events:
print "User: %s Room: %s" % (event["user_id"], event["room_id"])
print "Event Type: %s" % event["type"]
print "Content: %s" % event["content"]
return jsonify({})
if __name__ == "__main__":
app.run()
Set your new application service running on port 5000 with:
python app_service.py
The home server needs to know that the application service exists before it will send requests to it. This is done via a registration YAML file which is specified in Synapse's main config file e.g. <code>homeserver.yaml</code>. The server admin needs to add the application service registration configuration file as an entry to this file.
# homeserver.yaml
app_service_config_files:
- "/path/to/appservice/registration.yaml"
NB: Note the "-" at the start; this indicates a list element. The registration file <code>registration.yaml</code> should look like:
# registration.yaml
# this is the base URL of the application service
url: "http://localhost:5000"
# This is the token that the AS should use as its access_token when using the Client-Server API
# This can be anything you want.
as_token: wfghWEGh3wgWHEf3478sHFWE
# This is the token that the HS will use when sending requests to the AS.
# This can be anything you want.
hs_token: ugw8243igya57aaABGFfgeyu
# this is the local part of the desired user ID for this AS (in this case @logging:localhost)
sender_localpart: logging
namespaces:
users: []
rooms: []
aliases:
- exclusive: false
regex: "#logged_.*"
**You will need to restart the home server after editing the config file before it will take effect.**
|
To test everything is working correctly, go ahead and explicitly create a room with the alias "#logged_test:localhost" and send a message into the room: the HS will relay the message to the AS by PUTing to /transactions/&lt;tid&gt; and you should see your AS print the event on the terminal. This will monitor any room which has an alias prefix of "#logged_", but it won't lazily create room aliases if they don't already exist. This means it will only log messages in the room you created before: #logged_test:localhost. Try joining the room "#logged_test2:localhost" without creating it, and it will fail. Let's fix that and add in lazy room creation:
@app.route("/rooms/&lt;alias&gt;")
def query_alias(alias):
alias_localpart = alias.split(":")[0][1:]
requests.post(
# NB: "TOKEN" is the as_token referred to in registration.yaml
"http://localhost:8008/_matrix/client/api/v1/createRoom?access_token=TOKEN",
json.dumps({
"room_alias_name": alias_localpart
}),
headers={"Content-Type":"application/json"}
)
return jsonify({})
This makes the application service lazily create a room with the requested alias whenever the HS queries the AS for the existence of that alias (when users try to join that room), allowing any room with the alias prefix #logged_ to be sent to the AS. Now try joining the room "#logged_test2:localhost" and it will work as you'd expect.  You can see that if this were a real bridge, the AS would have checked for the existence of #logged_test2 in the remote network, and then lazily-created it in Matrix as required.
|
Application services are powerful components which extend the functionality of home servers, but they are limited. They can only ever function in a "passive" way. For example, you cannot implement an application service which censors swear words in rooms, because there is no way to prevent the event from being sent. Aside from the fact that censoring will not work when using end-to-end encryption, all federated home servers would also need to reject the event in order to stop developing an inconsistent event graph. To "actively" monitor events, another component called a "Policy Server" is required, which is beyond the scope of this post.  Also, Application Services can result in a performance bottleneck, as all events on the homeserver must be ordered and sent to the registered application services.  If you are bridging huge amounts of traffic, you may be better off having your bridge directly talk the Server-Server federation API rather than the simpler Application Service API.
I hope this demonstrates how easy it is to create an application service, along with a few ideas of the kinds of things you can do with them. Obvious uses include build protocol bridges, search engines, invisible bots, etc. For more information on the AS HTTP API, check out the new <a href="http://matrix.org/docs/spec/#application-service-api">Application Service API</a> section in the spec, or the raw drafts and spec in <a href="https://github.com/matrix-org/matrix-doc/" target="_blank">https://github.com/matrix-org/matrix-doc/</a>.