Merge branch 'master' into application-services

2015-02-09 09:12:58 +00:00 · 2015-02-09 09:12:58 +00:00 · 65b3cf3740
commit 65b3cf3740
parent e7c522b606 6c21f499e8
35 changed files with 3414 additions and 1913 deletions
--- a/drafts/client-server-v2.rst
+++ b/drafts/client-server-v2.rst
@ -1,297 +0,0 @@
 API Changes Summary
 ===================
 - Split up requesting tokens from delivering the data. This allows you to config many different tokens depending on the
  client. In v1, you could only hit /initialSync and get everything, even if that isn't what you wanted.
 - Introduction of an 'event filtering token'. This filters the event types returned from streams/pagination. Clients may
  have several of these in use at any moment, depending on the endpoint they are hitting.
 - All APIs which return events can optionally take a streaming token parameter. The server uses this to manage how events
  are sent to the client (either in response to the API call or via the stream, eg but not both for duplicate info reduction).
  This does mean each device would have a different streaming token.
 - New API: Scrollback API. This is designed to be used when you click on a room and want to display something. It is used
  in conjunction with a max # events limit. If you have some previous data, then you supply your streaming token. 
  If there are > max events between that event ID and now, it returns a brand new page of events and a new
  pagination token. If there are < max events, it just returns them incrementally. This supports both heavy/lightweight
  clients. This can use the event filtering token to allow only 'displayable' events to be shown.
 - New API: Pagination Overview API: This is designed for cases like forum threads, which tell you how many pages there are
  and allow you to jump around. This API returns an array of pagination tokens which represent each page. You tell it how
  many events per page. This can use the event filtering token to allow only 'displayable' events to be shown.
 Resolves issues:
 ----------------
 - You can't get events for a single room only as the event stream is global. FIX: You specify via the stream token.
 - You can't filter between "data" events (e.g. m.room.message) for display, and "metadata" events (e.g. power level changes).
  FIX: You specify via the event filter token.
 - There are race conditions when getting events via room initial sync and from the event stream. FIX: optional streaming
  token param allows intelligent suppression
 - You can't tell if an event you PUT is the same event when it comes down the event stream. FIX: optional streaming token 
  param allows intelligent suppression]
 - How do you obtain partial room state / handle large room state? FIX: You specify via the event filter token.
 - How do you sensibly do incremental updates? FIX: You give it a streaming token to return incremental updates.
 Outstanding Issues
 ------------------
 - Duplication of events in /initialSync is sub-optimal
 Issues not addressed
 --------------------
 These issues are more implementation specific (HTTP/JSON) and therefore haven't been addressed by this data model:
 - Naming of endpoints / keys isn't great
 - Can't set power levels incrementally.
 - State event PUTs are not consistent with other APIs.
 These issues are added features which have not been addressed:
 - Accessing federation level events (prev_pdus, signing keys, etc)
 - How do you reject an invite?
 - How do you delete state?
 - Paginating on global initial sync e.g. 10 most recently active rooms.
 - How do you determine the capabilities of a given HS?
 - Requesting context (read: events around) around an arbitrary event (which may have been 6 months ago)
 These issues are federation-related which have not been addressed:
 - Pagination can take a while for backfill. FIX: Add flag to say server_local vs backfill_yes_please? Given the client
  is best suited to say how long they are willing to wait.
 - Sending events may need to be multi-stage e.g. for signing. FIX: Extra 'Action API' added. Shouldn't be too invasive.
 - Handle rejection of events after the fact. e.g. HS later finds out that it shouldn't have accepted an event.
  TODO: Clarifiy if 'rejection' === redaction.
 These issues relate to events themselves which have not been addressed:
 - Distinguish between *room* EDUs (e.g. typing) and PDUs
 - Event timestamps (ISO8601?)
 Meta APIs (API calls used to configure other APIs)
 ==================================================
 Generating an event filtering token
 -----------------------------------
 Args:
 | Event Type Filter <[String]> (e.g. ["m.*", "org.matrix.custom.*", "my.specific.event.type"])
 Response:
 | Event Filter Token <String>
 Use Cases:
 | Picking out "displayable" events when paginating.
 | Reducing the amount of unhandled event types being sent to the client wasting bandwidth
 | Control whether presence is sent to the client (useless if they don't display it on the client!)
 Generating a streaming token
 ----------------------------
 Args:
 | Stream Config <Object> -> 
 | Room ID Filter <[String]> (e.g. ["!asd:foo.bar", "!dsf:foo.bar", ...]
 | User ID Filter <[String]> (e.g. ["@friend:foo.bar", "@boss:foo.bar", ...] or ["*"]) - e.g. control which user presence to get updates for
 Response:
 | Token <String>
 Use Cases:
 | Lightweight monitor-only-this-room-please
 | Heavyweight ALL THE THINGS
 | Middle of the road (all rooms joined with latest message + room names/aliases, rooms invited to + room names/aliases)
 Action APIs (performs some sort of action)
 ==========================================
 Create Room
 -----------
 Args:
 | Creation Config -> Join rules <String>, Visibility <String>
 | Response Config -> Events/filters/etc
 | Invitees <[String]>
 Response:
 | Room ID <String>
 | Response <Object> (Optional)
 Use Cases:
 | Create 1:1 PM room.
 | Create new private group chat
 | Create new public group chat with alias
 | Create new "forum thread"
 Send Message
 ------------
 Args:
 | Room ID <String>
 | Event Content <Object>
 | Event Type <String>
 | State Key <String> (Optional)
 Response:
 | ??? ACK ???
 Use Cases:
 | Sending message to a room.
 | Sending generic events to a room.
 | Sending state events to a room.
 | Send message in response to another message (commenting)
 Joining a room
 --------------
 Args:
 | Invite Event ID(?sufficient?) OR Room Alias <String> : This is how you accept an invite.
 | Response Config <Object> -> Events/filters/etc
 Response:
 | Room ID <String>
 | Response <Object> (Optional)
 Use Cases:
 | Joining a room from an invite
 | Joining a room from a room alias
 Invite/Leave/Kick/Ban
 ---------------------
 Args:
 | Room ID <String>
 | User ID <String>
 | Reason/Invitation Text <String> (Optional)
 Response:
 | ? ACK ?
 Syncing APIs
 ============
 Scrollback (aka I clicked a room and now want to display something)
 -------------------------------------------------------------------
 Args:
 | Room ID <String>
 | Max # Message Events <Integer>
 | Message Event Filter Token <String> (allows just 'displayable' events)
 | *Current* State Event Filter Token <String> (get member list, etc)
 | Streaming Token <String>
 Response:
 | Events <[Object]>
 | Incremental <Boolean> - True if the events are incremental from the streaming token provided. If false, there is > Max # events between NOW and the token provided.
 | Pagination Token <String> - The start token for the earliest message if not incremental.
 Use Cases:
 | Open a room and display messages (if no perm storage, supply no stream token to get the latest X events)
 | Open a room and get incremental (supply stream token and get either incremental messages or a new fresh lot depending on amount of events)
 Syncing (aka I want live data)
 ------------------------------
 NB: Does NOT provide any sort of 'catchup' service. This keeps the API simpler, and prevents potential attacks where people are dumb/maliciously request from ancient streaming tokens which then return 100000s of events, slowing down the HS. Alternatively, we could expire streaming tokens after a given time (but that doesn't help if 10000s of events come down really quickly). The general idea is to block all forms of historical data behind max events limits.
 Args:
 | Streaming Token <String>
 | Event Filtering Token <String> (optional)
 Response:
 | ??? EVENT STREAM DATA ???
 | Updated Streaming Token <String>
 Use Cases:
 | Getting events as they happen.
 Pagination (aka The user is infinite scrolling in a room)
 ---------------------------------------------------------
 Getting messages:
 Args:
 | Pagination Token <String>
 | Event Filter Token <String>
 | Room ID <String>
 | Max # Events <Integer>
 Response:
 | Events [<Object>]
 | New Pagination Token <String>
 Use Cases:
 | Infinite scrolling
 Requesting overview of pagination:
 Args:
 | Event Filter Token <String>
 | Room ID <String>
 | Max # Events per page <Integer>
 Response:
 | Pagination Tokens<[String]> - A snapshot of all the events *at that point* with the tokens you need to feed in to get each page. E.g. to get the 1st page, use token[0] into the "Getting messages" API.
 Use Cases:
 | Forum threads (page X of Y) - Allows jumping around.
 Initial Sync (aka I just booted up and want to know what is going on)
 ---------------------------------------------------------------------
 Args:
 | Message Events Event Filter Token <String> - the filter applied to message events f.e room
 | *Current* State Event Filter Token <String> - the filter applied to state events f.e room. Can specify nothing to not get ANY state events.
 | Max # events per room <Integer> - can be 0
 | Streaming Token <String> - A streaming token if you have one, to return incremental results
 Response:
 | Events per room [<Object>] - Up to max # events per room. NB: Still get duplicates for state/message events!
 | Pagination token per room (if applicable) [<String>] - Rooms which have > max events returns a fresh batch of events (See "Scrollback")
 Use Cases:
 | Populate recent activity completely from fresh.
 | Populate recent activity incrementally from a token.
 | Populate name/topic but NOT the name/topic changes when paginating (so m.room.name in state filter but not message filter)
 Examples
 ========
 Some examples of how these APIs could be used (emphasis on syncing since that is the main problem).
 #1 SYWEB recreation
 -------------------
 The aim of this is to reproduce the same data as v1, as a sanity check that this API is functional.
 - Login. POST streaming token (users["*"], rooms["*"]). Store token.
 - GET /initialSync max=30 with token. Returns all rooms (because rooms["*"]) and all state, and all presence (because users["*"]),
  and 30 messages. All event types returned since I didn't supply any event filter tokens. Since the streaming token 
  hasn't ever been used (I just made one), it returns the most recent 30 messages f.e room. This is semantically the same
  as v1's /initialSync.
 - GET /eventStream with streaming token. Starts blocking.
 - Click on room !foo:bar. Start infinite scrolling.
 - GET /paginate. Pagination token from initial sync. Get events and store new pagination token.
 #2 Less buggy SYWEB recreation
 ------------------------------
 The aim of this is to leverage the new APIs to fix some bugs.
 - Login. POST streaming token (users["*"], rooms["*"]). Store stream token.
 - POST event filter token (["m.room.message", "m.room.topic", "m.room.name", "m.room.member"]). Store as Message Event filter token.
 - POST event filter token (["m.room.name", "m.room.topic", "m.room.member"]). Store as Current State Event filter token.
 - GET /initialSync max=1 with all tokens. Returns all rooms (rooms["*"]), with name/topic/members (NOT all state), with
  max 1 m.room.message/topic/name/member (truly honouring max=1), with presence (users["*"]).
 - GET /eventStream with stream token. Blocks.
 - Click on room !foo:bar. Start infinite scrolling.
 - GET /paginate with Message Event filter token. Returns only m.room.message/name/topic/member events.
 #3 Mobile client (permanent storage)
 ------------------------------------
 The aim of this is to use the new APIs to get incremental syncing working.
 Initially:
 - Login. POST streaming token (users["*"], rooms["*"]). Store as stream token.
 - POST event filter token (["m.room.message"]). Store as Message Event filter token.
 - POST event filter token (["m.*"]). Store as Current State Event filter token.
 - GET /initialSync max=30 (we want a page worth of material) with all tokens. Returns all rooms (rooms["*"]), 
  with all m.* current state, with max 1 m.room.message, with presence (users["*"]).
 - GET /eventStream with stream token. Blocks.
 - Get some new events, new stream token. Quit app.
 Subsequently:
 - GET /initialSync max=30 with all tokens. Because the stream token has been used before, it tries to get the diff between
  then and now, with the filter tokens specified. If it finds > 30 events for a given room, it returns a brand new page 
  for that room. If it finds < 30 events, it returns those events. Any new rooms are also returned. Returns a new stream token.
 - GET /eventStream with new stream token. Blocks.
 #4 Lightweight client (super lazy loading, no permanent storage)
 ----------------------------------------------------------------
 The aim of this is to have a working app with the smallest amount of data transfer. Event filter tokens MAY be reused
 if the lightweight client persists them, reducing round-trips.
 - POST streaming token (rooms["*"] only, no presence). Store as streaming token.
 - POST event filter token (["m.room.message"]). Store message event filter token.
 - POST event filter token (["m.room.name"]). Store as current state event filter token.
 - POST event filter token (["m.room.message", "m.room.name", "m.room.member"]). Store as eventStream filter token.
 - GET /initialSync max=1 with all tokens. Returns all rooms (rooms["*"]), with 1 m.room.message, no presence, and just
  the current m.room.name if a room has it.
 - Click on room !foo:bar.
 - POST streaming token (rooms["!foo:bar"]), store as foo:bar token.
 - GET /eventStream with foo:bar token AND eventStream token. This will get new messages (m.room.message) and room name
  changes (m.room.name). It will also get me new room invites (m.room.member) and join/leave/kick/ban events (m.room.member),
  all JUST FOR THE ROOM !foo:bar.
--- a/drafts/client_server_use_cases.log
+++ b/drafts/client_server_use_cases.log
@ -1,303 +0,0 @@
 #1: Lightweight IM client (no perm storage)
 #2: Mobile IM client (perm storage)
 #3: MIDI client
 #4: Animatrix client
 #5: Unity object trees
 #6: Forum
 #7: Social Network ("Walls", PMs, groups)
 #8: Minecraft-clone
 #9: Bug Tracking Software
 #10: Global 'Like' widget, which links through to a room.
 ================
 #1: Lightweight IM client (no perm storage)
 -------------------------------------------
 Description: 
  An IM client (think web client) with no way of persisting data beyond
  a session (the instance a person is using the app).
 Features: 
  Recent activity, Room screen (member list, etc), User page, just like
  the web client.
 Actions:
 - Send a one-to-one message to someone.
 - Accept an invite.
 - Populate recent activity (all rooms joined with latest message + room names/aliases, rooms invited to + room names/aliases)
 - Populate scrollback if click on room
 - Populate member list if click on room + get presence updates for them
 - Populate room name / topic if click on room
 - Create an empty room.
 - Join a room from an alias.
 ---
 Action: 
  Send a one-to-one message to someone.
 How: 
  Enter their username and hit Message. Taken to room page with invited user.
  History displays that I've invited someone / joined the room. Enter a message
  and hit send. Message appears in window.
 ::
  Client                    Server
       -- @user:domain -->
       <--- room ID, ACK--
       <-historical msgs--
       -- msg,room ID --->
       <--- ACK ----------
 ---
 Action: 
  Accept an invite.
 How: 
  Get list of invites. Click one of them to 'accept' it. May or may not want
  room content.
 ::
  Client                    Server
       ---- req invites ->
       <--- [inv,inv] ----
       ---- accept inv -->
       <--- ACK ----------
       <--- room content-- (optional)
 ---
 Action: 
  Populate recent activity (all rooms joined with latest message + room names/aliases, rooms invited to + room names/aliases)
 How: 
  Request joined rooms with latest message and room name. Request rooms invited to. Possibly extra info like # joined members.
 ::
  Client                    Server
       ---- req sync ---->
       <---joined rooms--- {msg,name,alias,#members?}
       <---invited rooms-- {name,alias}
 ---
 Action: 
  Populate scrollback if click on room.
 How: 
  Request scrollback for room.
 ::
  Client                    Server
       ---- room id ----->
       <--- scrollback ---
 ---
 Action: 
  Populate member list if click on room + get presence updates for them.
 How: 
  Click on room. Member list with names/presence/pics appears. May not want
  pic.
 ::
  Client                    Server
       ---- req mem list ->
       <--- members ------- {name,pic,presence}
       - monitor presence->
              ...
       <- presence change--
       <- presence change--
              ...
       -- stop presence --->
 ---
 Action: 
  Populate room name / topic if click on room.
 How: 
  Click on room. Room name and topic with aliases appears. May not want topic
  (eg screen size).
 ::
  Client                    Server
       ---- req room info->
       <--- room info ----- {name,topic,aliases}
 ---
 Action: 
  Create an empty room.
 How: 
  Type in room config (desired name, public/private, etc). Hit Create. Room is
  created. Possibly get room info.
 ::
  Client                    Server
       ---- mkroom{config}->
       <--ACK{room_id}------
       <-- room info ------- (optional)
 ---
 Action: 
  Join a room from an alias.
 How: 
  Type in alias. Hit Join. Room is joined. Possibly get room info.
 ::
  Client                    Server
       -- join{alias} ----->
       <--ACK{room_id}------
       <--room info--------- (optional)
 ===========================
 #2: Mobile IM client (perm storage)
 -----------------------------------
 Description: 
  An IM client (think android/ios) which persists data on a database.
 Features: 
  Recent activity, Room screen (member list, etc), User page, just like
  the web client.
 Actions:
 - Send a one-to-one message to someone.
 - Accept a stored invite.
 - Populate recent activity (all rooms joined with latest message + room names/aliases, rooms invited to + room names/aliases)
 - Populate scrollback if click on room
 - Populate member list if click on room + get presence updates for them
 - Populate room name / topic if click on room
 - Create an empty room.
 - Join a room from an alias.
 ---
 Action: 
  Send a one-to-one message to someone (single room).
 How: 
  Enter their username and hit Message. Taken to room page with invited user if no room exists,
  else takes to existing room. History displays that I've invited someone or scrollback. Enter 
  a message and hit send. Message appears in window.
 ::
  Client                    Server
       -- @user:domain -->
       <--- room ID, ACK--
       <-historical msgs-- (optional; not if existing room)
       -- msg,room ID ---> 
       <--- ACK ----------
 ---
 Action: 
  Accept a stored invite.
 How: 
  Send invite to server. Get room content (or NO-OP if already joined).
 ::
  Client                    Server
       ---- accept inv -->
       <--- ACK ----------
       <--- room content-- (optional)
 ---
 Action: 
  Populate recent activity (all rooms joined with latest message + room names/aliases, rooms invited to + room names/aliases)
  incrementally.
 How: 
  Request recent activity diff. Get updated msg/name/#members for changed values only.
 ::
  Client                    Server
       - req sync{token}->
       <---diff{rooms}---- {msg,name,alias,#members?}
 ---
 Action: 
  Populate scrollback if click on room. 
 How: 
  Request scrollback for room. Either a diff or a page of scrollback
  depending on cached data.
 ::
  Client                        Server
       -room id{latest event}-> {max msgs}
       <--- scrollback -------- {fresh/incremental flag}
 ---
 Action: 
  Populate member list if click on room + get presence updates for them.
 How: 
  Click on room. Member list with names/presence/pics appears. May not want
  pic.
 ::
  Client                    Server
       ---- req mem list ->
       <--- members ------- {name,pic,presence}
       - monitor presence->
              ...
       <- presence change--
       <- presence change--
              ...
       -- stop presence --->
 ---
 Action: 
  Populate room name / topic if click on room.
 How: 
  Click on room. Room name and topic with aliases appears. May not want topic
  (eg screen size). Display cached info until updated.
 ::
  Client                    Server
       ---- req room info->
       <--- room info ----- {name,topic,aliases}
 ---
 Action: 
  Create an empty room.
 How: 
  Type in room config (desired name, public/private, etc). Hit Create. Room is
  created. Possibly get room info.
 ::
  Client                    Server
       ---- mkroom{config}->
       <--ACK{room_id}------
       <-- room info ------- (optional)
 ---
 Action: 
  Join a room from an alias.
 How: 
  Type in alias. Hit Join. Room is joined. Possibly get room info.
 ::
  Client                    Server
       -- join{alias} ----->
       <--ACK{room_id}------
       <--room info--------- (optional)
--- a/drafts/data_flows.rst
+++ b/drafts/data_flows.rst
@ -0,0 +1,222 @@
 Data flows for use cases
 ========================
 ::
 <- Data from server to client
 -> Data from client to server
 Instant Messaging
 -----------------
 Without storage
 ~~~~~~~~~~~~~~~
 ::
 Home screen
   Data required on load:
    <- For each room the user is joined: Name, topic, # members, last message, room ID, aliases
   Data required when new message arrives for a room:
    <- Room ID, message content, sender (user ID, display name, avatar url)
   Data required when someone invites you to a room:
    <- Room ID, sender (user ID, display name, avatar url), Room Name, Room Topic
   Data required when you leave a room on another device:
    <- Room ID
   Data required when you join a room on another device:
    <- Name, topic, # members, last message, room ID, aliases
   Data required when your profile info changes on another device:
    <- new profile info e.g. avatar, display name, etc.
   Creating a room
    -> Invitee list of user IDs, public/private, name of room, alias of room, topic of room
    <- Room ID
   Joining a room (and dumped into chat screen on success)
    -> Room ID / Room alias
    <- Room ID, Room aliases (plural), Name, topic, member list (f.e. member: user ID, 
       avatar, presence, display name, power level, whether they are typing), enough 
       messages to fill screen (and whether there are more)
 Chat Screen
   Data required when member name changes:
    <- new name, room ID, user ID, when in the context of the room did this occur
   Data required when the room name changes:
    <- new name, room ID, old room name?
   Invite a user:
    -> user ID, room ID
    <- display name / avatar of user invited (if known)
   Kick a user:
    -> user ID, room ID
    <- what message it came after
   Leave a room:
    -> room ID
    <- what message it came after
   Send a message
    -> Message content, room ID, message sequencing (eg sending my 1st, 2nd, 3rd msg)
    <- actual content sent (if server mods it), what message it comes after (to correctly
       display the local echo)
    Place a call (receive a call is just reverse)
     <- turn servers
     -> SDP offer
     -> Ice candidates (1 by 1; trickling)
     <- SDP answer
     <- Ice candidates
    Scrolling back (infinite scrolling)
     -> Identifier for the earliest message, # requested messages
     <- requested messages (f.e change in display name, what the old name was), whether
        there are more.
 With storage
 ~~~~~~~~~~~~
 ::
  Home Screen
    On Load
     -> Identifier which tells the server the client's current state (which rooms it is aware
        of, which messages it has, what display names for users, etc..)
     <- A delta from the client's current state to the current state on the server (e.g. the
        new rooms, the *latest* message if different, the changed display names, the new 
        invites, etc). f.e Room: Whether the cache of the room that you have has been replaced 
        with this new state.
    Pre-load optimisation (not essential for this screen)
      -> Number of desired messages f.e room to cache
      <- f.e Room: the delta OR the entire state
 Bug Tracking
 ------------
 ::
  Landing Page
    On Load
     <- Issues assigned to me, Issues I'm watching, Recent activity on other issues includes
        comments, list of projects
    Search for an issue (assume text)
     -> Search string
     <- List of paginated issues
       Request page 2:
        -> Page number requested
        <- Page of paginated issues
  Issue Page
    On Load
     -> Issue ID and Project ID (equiv to Room)
     <- Issue contents e.g. priority, resolution state, etc. All comments e.g. user ID, 
        comment text, timestamp. Entire issue history e.g. changes in priority
    Post a comment
     -> Issue ID, comment content, Project ID (equiv to Room)
     <- actual content sent (if modded), what comment it comes after
    Set issue priority
     -> Issue ID, Project ID, desired priority
     <- What action in the history it came after
    Someone else sets issue priority
     <- Issue ID, Project ID, new priority, where in the history
 Mapping model use cases to matrix models (Room, Message, etc)
 =============================================================
 To think about:
 - Do we want to support the idea of forking off new rooms from existing ones? This
   and forums could benefit from it.
 Bug tracking UI
 ---------------
 ::
 Projects => Rooms
 Issues => Message Events
 Comments => Message Events (relates_to key)
 Projects:
 - Unlikely that there will be 100,000s of issues, so having to pull in all the issues for a project is okay.
 - Permissions are usually per project and this Just Works.
 - New issues come in automatically and Just Work.
 - Can have read-only members
 Issues:
 - Don't really want 1 Room per Issue, else you can have thousands of Rooms PER PROJECT, hence choice for    
   Issues as Messages. Don't need to join a room for each issue.
 - Idea of issue owner is clear (sender of the message)
 - Updating issues requires an additional event similar to comments (with ``relates_to``)? Could possibly
   be state events? Don't really want all the history if say the priority was changed 1000 times, just want
   the current state of the key.
 Comments:
 - Additional event with ``relates_to`` key.
 Forum
 -----
 ::
 Forum => Room (with pointers to Board Rooms)
 Boards => Room (with pointers to Thread Rooms)
 Threads => Room
 Messages => Message Events
 Forum:
 - Contains 10s of Boards.
 - Contains special Message Events which point to different rooms f.e Board.
 Boards:
 - Contains 100s of Threads.
 - Contains special Message Events which point to different rooms f.e. Thread.
 Threads:
 - Contains 100s of Messages.
 Can't do this nicely with the current Federation API because you have loads of
 Rooms and what does posting a message look like? Creating a thread is done by..?
 The user who is posting cannot create the thread because otherwise they would be
 the room creator and have ultimate privileges. So it has to be created by a bot
 of some kind which ties into auth (Application services?). To follow a board,
 you need a bot to join the Board Room and then watch it for changes... 
 Fundamental problem with forums is that there is only 1 PDU graph per room and
 you either have to pull in lots of graphs separately or one graph and filter it
 separately to get to the desired sub set of data. You have to subscribe into a
 lot of graphs if you subscribe to a board... If you have the entire board...
 good luck scrollbacking a particular thread.
 Google+ Community
 -----------------
 ::
 Community => Room (with pointers to Category Rooms)
 Category => Room
 Post => Message Events
 Comment => Message Events (relates_to key)
 Community:
 - Contains 10s of categories.
 - Contains special Message Events which point to different rooms f.e Category.
 - Moderators of the community are mods in this room. They are in charge of making
   new categories and the subsequent rooms. Can get a bit funky if a mod creates a
   category room without the same permissions as the community room... but another
   mod can always delete the pointer to the buggy category room and make a new one.
 - Do we want to support the idea of forking off new rooms from existing ones? This
   and forums could benefit from it.
 Category:
 - Contains 1000s of posts.
 - Same permissions as the community room. How to enforce? Fork off the community
   room?
 Posts:
 - Contains 10s of comments.
 This is similar to forums but you can more reasonably say "screw it, pull in the
 entire community of posts."
--- a/drafts/design_workflow.rst
+++ b/drafts/design_workflow.rst
@ -0,0 +1,14 @@
 Matrix Spec Design Workflow
 ===========================
 1. Write use cases
 2. Design data flows for use cases
 3. Design generic API (factoring out commonalities where possible)
 4. Design transport-specific API with justifications
 5. Formalise transport-specific API as swagger or similar
 6. Evolve the generic API design doc and transport-specific API into the actual spec.
--- a/drafts/erikj_federation.rst
+++ b/drafts/erikj_federation.rst
@ -0,0 +1,408 @@
 Federation
 ==========
 .. sectnum::
 .. contents:: Table of Contents
 Authorization
 -------------
 When receiving new events from remote servers, or creating new events, a server 
 must know whether that event is allowed by the authorization rules. These rules
 depend solely on the state at that event. The types of state events that affect
 authorization are:
 - ``m.room.create``
 - ``m.room.member``
 - ``m.room.join_rules``
 - ``m.room.power_levels``
 Servers should not create new events that reference unauthorized events. 
 However, any event that does reference an unauthorized event is not itself
 automatically considered unauthorized. 
 Unauthorized events that appear in the event graph do *not* have any effect on 
 the state of the graph. 
 .. Note:: This is in contrast to redacted events which can still affect the 
          state of the graph. For example, a redacted *"join"* event will still
          result in the user being considered joined.
 Rules
 ~~~~~
 The following are the rules to determine if an event is authorized (this does
 include validation).
 **TODO**: What signatures do we expect?
 1. If type is ``m.room.create`` allow if and only if it has no prev events.
 #. If type is ``m.room.member``:
   a. If ``membership`` is ``join``:
      i. If the previous event is an ``m.room.create``, the depth is 1 and 
         the ``state_key`` is the creator, then allow.
      #. If the ``state_key`` does not match ``sender`` key, reject.
      #. If the current state has ``membership`` set to ``join``.
      #. If the ``sender`` is in the ``m.room.may_join`` list. [Not currently 
         implemented]
      #. If the ``join_rules`` is:
         - ``public``:  allow.
         - ``invite``: allow if the current state has ``membership`` set to 
           ``invite``
         - ``knock``: **TODO**.
         - ``private``: Reject.
      #. Reject
   #. If ``membership`` is ``invite`` then allow if ``sender`` is in room, 
      otherwise reject.
   #. If ``membership`` is ``leave``:
      i. If ``sender`` matches ``state_key`` allow.
      #. If ``sender``'s power level is greater than the the ``kick_level``
         given in the current ``m.room.power_levels`` state (defaults to 50),
         and the ``state_key``'s power level is less than or equal to the
         ``sender``'s power level, then allow.
      #. Reject.
   #. If ``membership`` is ``ban``:
      i. **TODO**.
   #. Reject.
 #. Reject the event if the event type's required power level is less that the
   ``sender``'s power level.
 #. If the ``sender`` is not in the room, reject.
 #. If the type is ``m.room.power_levels``:
   a. **TODO**.
 #. Allow.
 Definitions
 ~~~~~~~~~~~
 Required Power Level
  A given event type has an associated *required power level*. This is given
  by the current ``m.room.power_levels`` event, it is either listed explicitly
  in the ``events`` section or given by either ``state_default`` or 
  ``events_default`` depending on if the event type is a state event or not.
 Auth events
 ~~~~~~~~~~~
 The auth events of an event are the set of events used by the authorization 
 algorithm to accept the event. These should be a subset of the current state.
 A server is required to store the complete chain of auth events for all events
 it serves to remote servers.
 All auth events have type:
 - ``m.room.create``
 - ``m.room.power_levels``
 - ``m.room.member``
 .. todo
    We probably should probably give a lower band of how long auth events
    should be kept around for.
 Auth chain
 ~~~~~~~~~~
 The *auth chain* for an event is the recursive list of auth events and the auth
 chain for those auth events.
 .. Note:: The auth chain for an event gives all the information a server needs
          to accept an event. However, being given an auth chain for an event
          that appears valid does not mean that the event might not later be
          rejected. For example if we discover that the sender had been banned
          between the join event listed in the auth events and the event being
          authed.
 **TODO**: Clean the above explanations up a bit.
 Auth chain resolution
 ~~~~~~~~~~~~~~~~~~~~~
 If an auth check fails, or if we get told something we accepted should have
 been rejected, we need to try and determine who is right.
 If two servers disagree about the validity of the auth events, both should
 inform the other of what they think the current auth chain is. If either are
 missing auth events that they know are valid (through authorization and state
 resolution) they process the missing events as usual.
 If either side notice that the other has accepted an auth events we think
 should be rejected (for reasons *not* in their auth chain), that server should
 inform the other with suitable proof.
 The proofs can be:
 - An *event chain* that shows an auth event is *not* an ancestor of the event.
  This can be done by giving the full ancestor chains up to the depth of the
  invalid auth event.
 - Given an event (and event chain?) showing that authorization had been revoked.
 If a server discovers it cannot prove the other side is wrong, then it accepts
 that the other is correct; i.e. we always accept that the other side is correct
 unless we can prove otherwise.
 State Resolution
 ----------------
    **TODO**
 When two branches in the event graph merge, the state of those branches might
 differ, so a *state resolution* algorithm must be used to determine the current
 state of the resultant merge.
 The properties of the state resolution algorithm are:
 - Must only depend on the event graph, and not local server state.
 - When two state events are comparable, the descendant one should be picked.
 - Must not require the full event graph.
 The following algorithm satisfies these requirements; given two or more events,
 pick the one with the greatest:
 #. Depth.
 #. Hash of event_id.
 This works except in the case of auth events, where we need to mitigate against
 the attack where servers artificially netsplit to avoid bans or power level
 changes.
 We want the following rules to apply:
 #. If power levels have been changed on two different branches use the rules
   above, ensuring that the one picked is a valid change from the one not picked.
 #. Similarly handle membership changes (e.g. bans, kicks, etc.)
 #. Any state merged must be allowed by the newly merged auth events. If none of
   the candidate events for a given state are allowed, we pick the last event
   given by the ordering above (i.e. we pick one with the least depth).
 State Conflict Resolution
 -------------------------
 If a server discovers that it disagrees with another about the current state,
 it can follow the same process outlined in *Auth chain resolution* to resolve
 these conflicts.
 Constructing a new event
 ------------------------
    **TODO**
 When constructing a new event, the server should insert the following fields:
 - ``prev_events``: The list of event ids of what the server believes are the
  current leaf nodes of the event graph (i.e., nodes that have been received
  but are yet to be referenced by another event).
 - ``depth``: An integer one greater than the maximum depth of the event's
  previous events.
 - ``auth_events``: The list of event ids that authorizes this event. This
  should be a subset of the current state.
 - ``origin_server_ts``: The time the server created the event.
 - ``origin``: The name of the server.
 Signing and Hashes
 ~~~~~~~~~~~~~~~~~~
    **TODO**
 Validation
 ----------
    **TODO**
 Domain specific string
    A string of the form ``<prefix><localpart>:<domain>``, where <prefix> is a
    single character, ``<localpart>`` is an arbitrary string that does not
    include a colon, and `<domain>` is a valid server name.
 ``room_id``
    A domain specific string with prefix ``!`` that is static across all events
    in a graph and uniquely identifies it. The ``domain`` should be that of the
    home server that created the room (i.e., the server that generated the
    first ``m.room.create`` event).
 ``sender``
    The entity that logically sent the event. This is usually a user id, but
    can also be a server name.
 User Id
    A domain specific string with prefix ``@`` representing a user account. The
    ``domain`` is the home server of the user and is the server used to contact
    the user.
 Joining a room
 --------------
 If a user requests to join a room that the server is already in (i.e. the a
 user on that server has already joined the room) then the server can simply
 generate a join event and send it as normal.
 If the server is not already in the room it needs to will need to join via
 another server that is already in the room. This is done as a two step process.
 First, the local server requests from the remote server a skeleton of a join
 event. The remote does this as the local server does not have the event graph
 to use to fill out the ``prev_events`` key in the new event. Critically, the
 remote server does not process the event it responded with.
 Once the local server has this event, it fills it out with any extra data and
 signs it. Once ready the local server sends this event to a remote server
 (which could be the same or different from the first remote server), this
 remote server then processes the event and distributes to all the other
 participating servers in that room. The local server is told about the
 current state and complete auth chain for the join event. The local server
 can then process the join event itself.
 .. Note::
   Finding which server to use to join any particular room is not specified.
 Inviting a user
 ---------------
 To invite a remote user to a room we need their home server to sign the invite
 event. This is done by sending the event to the remote server, which then signs
 the event, before distributing the invite to other servers.
 Handling incoming events
 ------------------------
 When a server receives an event, it should:
 #. Check if it knows about the room. If it doesn't, then it should get the
   current state and auth events to determine whether the server *should* be in
   the room. If so continue, if not drop or reject the event
 #. If the server already knew about the room, check the prev events to see if
   it is missing any events. If it is, request them. Servers should limit how
   far back they will walk the event graph for missing events.
 #. If the server does not have all the prev events, then it should request the
   current state and auth events from a server.
 Failures
 --------
 A server can notify a remote server about something it thinks it has done
 wrong using the failures mechanism. For example, the remote accepted an event
 the local think it shouldn't have.
 A failure has a severity level depending on the action taken by the local
 server. These levels are:
 ``FATAL``
    The local server could not parse the event, for example due to a missing
    required field.
 ``ERROR``
    The local server *could* parse the event, but it was rejected. For example,
    the event may have failed an authorization check.
 ``WARN``
    The local server accepted the event, but something was unexpected about it.
    For example, the event may have referenced another event the local server
    thought should be rejected.
 A failure also includes several other fields:
 ``code``
    A numeric code (to be defined later) indicating a particular type of
    failure.
 ``reason``
    A short string indicating what was wrong, for diagnosis purposes on the
    remote server.
 ``affected``
    The event id of the event this failure is responding to. For example, if
    an accepted event referenced a rejected event, this would point to the
    accepted one.
 ``source``
    The event id of the event that was the source of this unexpected behaviour.
    For example, if an accepted event referenced a rejected event, this would
    point to the rejected one.
 Appendix
 ========
    **TODO**
 Example event:
 .. code::
    {
        "auth_events": [
            [
                "$14187571482fLeia:localhost:8480",
                {
                    "sha256": "kiZUclzzPetHfy0rVoYKnYXnIv5VxH8a4996zVl8xbw"
                }
            ],
            [
                "$14187571480odWTd:localhost:8480",
                {
                    "sha256": "GqtndjviW9yPGaZ6EJfzuqVCRg5Lhoyo4YYv1NFP7fw"
                }
            ],
            [
                "$14205549830rrMar:localhost:8480",
                {
                    "sha256": "gZmL23QdWjNOmghEZU6YjqgHHrf2fxarKO2z5ZTbkig"
                }
            ]
        ],
        "content": {
            "body": "Test!",
            "msgtype": "m.text"
        },
        "depth": 250,
        "event_id": "$14207181140uTFlx:localhost:8480",
        "hashes": {
            "sha256": "k1nuafFdFvZXzhb5NeTE0Q2Jkqu3E8zkh3uH3mqwIxc"
        },
        "origin": "localhost:8480",
        "origin_server_ts": 1420718114694,
        "prev_events": [
            [
                "$142071809077XNNkP:localhost:8480",
                {
                    "sha256": "xOnU1b+4LOVz5qih0dkNFrdMgUcf35fKx9sdl/gqhjY"
                }
            ]
        ],
        "room_id": "!dwZDafgDEFTtpPKpLy:localhost:8480",
        "sender": "@bob:localhost:8480",
        "signatures": {
            "localhost:8480": {
                "ed25519:auto": "Nzd3D+emFBJJ4LCTzQEZaKO0Sa3sSTR1fGpu8OWXYn+7XUqke9Q1jYUewrEfxb3lPxlYWm/GztVUJizLz1K5Aw"
            }
        },
        "type": "m.room.message",
        "unsigned": {
            "age": 500
        }
    }
--- a/drafts/event_schema.yaml
+++ b/drafts/event_schema.yaml
@ -0,0 +1,123 @@
 # JSON Schema for Matrix events. http://json-schema.org/
 title: "Matrix Event"
 type: object
 properties:
  auth_events:
    description: The events needed to authenticate this event
    $ref: "#/definitions/event_reference_list"
  content:
    type: object
    description: The body of the event.
  depth:
    type: integer
    description: A number one greater than that of any preceeding event.
    minimum: 0
  event_id:
    $ref: "#/definitions/event_id"
  hashes:
    $ref: "#/definitions/hashes"
  origin:
    $ref: "#/definitions/server_id"
  origin_server_ts:
    type: integer
    description: Posix timestamp on the originating server
    minimum: 0
  prev_events:
    description: The event(s) this event came after.
    $ref: "#/definitions/event_reference_list"
  prev_state:
    description: The state event(s) this event replaces.
    $ref: "#/definitions/event_reference_list"
  room_id:
    $ref: "#/definitions/room_id"
  sender:
    oneOf:
    - $ref: "#/definitions/user_id"
    - $ref: "#/definitions/server_id"
  state_key:
    type: string
  type:
    type: string
  unsigned:
    type: object
 required:
 - auth_events
 - content
 - depth
 - event_id
 - hashes
 - origin
 - origin_server_ts
 - prev_events
 - room_id
 - sender
 - type
 dependencies:
  state_key:
  - prev_state
  prev_state:
  - state_key
 definitions:
  server_id:
    type: string
    description: Identifies a server.
    pattern: "^(([a-zA-Z0-9]|[a-zA-Z0-9][a-zA-Z0-9\\-]*[a-zA-Z0-9])\\.)*\
             ([a-zA-Z0-9]|[a-zA-Z0-9][a-zA-Z0-9\\-]*[a-zA-Z0-9])\
             (:[0-9]*[0-9])?$"
  event_id:
    type: string
    description: Identifies an event.
    pattern: "^\\$[^:]+:\
             (([a-zA-Z0-9]|[a-zA-Z0-9][a-zA-Z0-9\\-]*[a-zA-Z0-9])\\.)*\
             ([a-zA-Z0-9]|[a-zA-Z0-9][a-zA-Z0-9\\-]*[a-zA-Z0-9])\
             (:[0-9]*[0-9])?$"
  room_id:
    type: string
    description: Identifies a room.
    pattern: "^\\![^:]+:\
             (([a-zA-Z0-9]|[a-zA-Z0-9][a-zA-Z0-9\\-]*[a-zA-Z0-9])\\.)*\
             ([a-zA-Z0-9]|[a-zA-Z0-9][a-zA-Z0-9\\-]*[a-zA-Z0-9])\
             (:[0-9]*[0-9])?$"
  user_id:
    type: string
    description: Identifies a user.
    pattern: "^\\@[^:]+:\
             (([a-zA-Z0-9]|[a-zA-Z0-9][a-zA-Z0-9\\-]*[a-zA-Z0-9])\\.)*\
             ([a-zA-Z0-9]|[a-zA-Z0-9][a-zA-Z0-9\\-]*[a-zA-Z0-9])\
             (:[0-9]*[0-9])?$"
  base64:
    type: string
    description: Base64 string without padding.
    pattern: "^[a-zA-Z0-9/+]*$"
  hashes:
    type: object
    description: Hashes
    minProperties: 1
    additionalProperties:
      $ref: "#/definitions/base64"
  signatures:
    type: object
    description: Signatures
    additionalProperties:
      type: object
      minProperties: 1
      additionalProperties:
        $ref: "#/definitions/base64"
  event_reference:
    type: array
    minItems: 2
    maxItems: 2
    items:
    - type: string
      description: Event id of the referenced event.
      $ref: "#/definitions/event_id"
    - type: object
      description: Reference hashes of the referenced event.
      $ref: "#/definitions/hashes"
  event_reference_list:
    type: array
    items:
      $ref: "#/definitions/event_reference"
--- a/drafts/general_api.rst
+++ b/drafts/general_api.rst
--- a/drafts/model/profiles.rst
+++ b/drafts/model/profiles.rst
@ -198,12 +198,11 @@ Requests that a client can make to its Home Server
 * get another user's Display Name / Avatar / metadata fields
-[[TODO(paul): At some later stage we should consider the API for:
+TODO(paul): At some later stage we should consider the API for:
 * get/set ACL permissions on my metadata fields
 * manage my ACL tokens
 ]]
 Server-Server
 -------------
--- a/drafts/model/protocol_examples.rst
+++ b/drafts/model/protocol_examples.rst
@ -3,7 +3,9 @@ Host: ...
 Content-Length: ...
 Content-Type: application/json
-{
+.. code :: javascript
  {
    "origin": "localhost:5000",
    "pdus": [
        {
@ -27,7 +29,7 @@ Content-Type: application/json
        "1404381396852"
    ],
    "ts": 1404381427823
-}
+  }
 HTTP/1.1 200 OK
 ...
@ -42,7 +44,9 @@ HTTP/1.1 200 OK
 Content-Length: ...
 Content-Type: application/json
-{
+.. code :: javascript
  {
    origin: ...,
    prev_ids: ...,
    data: [
@ -59,6 +63,6 @@ Content-Type: application/json
        },
        ...,
    ]
-}
+  }
--- a/drafts/model/rooms.rst
+++ b/drafts/model/rooms.rst
@ -125,13 +125,14 @@ m.invite_level
 m.join_rules
  Encodes the rules on how non-members can join the room. Has the following
  possibilities:
-    "public" - a non-member can join the room directly
+
-    "knock" - a non-member cannot join the room, but can post a single "knock"
+    - "public" - a non-member can join the room directly
-        message requesting access, which existing members may approve or deny
+    - "knock" - a non-member cannot join the room, but can post a single "knock"
-    "invite" - non-members cannot join the room without an invite from an
+      message requesting access, which existing members may approve or deny
-        existing member
+    - "invite" - non-members cannot join the room without an invite from an
-    "private" - nobody who is not in the 'may_join' list or already a member
+      existing member
-        may join by any mechanism
+    - "private" - nobody who is not in the 'may_join' list or already a member
      may join by any mechanism
  In any of the first three modes, existing members with sufficient permission
  can send invites to non-members if allowed by the "m.invite_level" key. A
@ -263,10 +264,11 @@ resolve this:
   that duplicate requests can be suppressed. On receipt of a room creation
   request that the HS thinks there already exists a room for, the invitation to
   join can be rejected if:
-      a) the HS believes the sending user is already a member of the room (and
+
-         maybe their HS has forgotten this fact), or
+      - a) the HS believes the sending user is already a member of the room (and
-      b) the proposed room has a lexicographically-higher ID than the existing
+        maybe their HS has forgotten this fact), or
-         room (to resolve true race condition conflicts)
+      - b) the proposed room has a lexicographically-higher ID than the existing
        room (to resolve true race condition conflicts)
 * The room ID for a private 1:1 chat has a special form, determined by
   concatenting the User IDs of both members in a deterministic order, such that
--- a/drafts/object_model.rst
+++ b/drafts/object_model.rst
@ -0,0 +1,28 @@
 ::
 +---------------+
 | Room          |
 |   "Room-ID"   |
 |   {State}     |         +----------------------+
 |     Name------|-------->| Event m.room.name    |
 |     Topic     |         |   "Name"             |
 |     [Aliases] |         +----------------------+      +-------------+
 |     [Members]-|---+     +----------------------+ <----| Start Token |
 |   [Messages]  |   |     | Event m.room.member  |      +-------------+
 |     |   |     |   +---->|   "invite/join/ban"  |
 +---------------+         |   "User-ID"          |
       |   |               +----------------------+
       |   |               +----------------------+
       |   | Message       | Event m.room.message |
       |   +-------------->|   {content}          |<--+
       |                   +----------------------+   |
       | Comment           +----------------------+   |
       +------------------>| Event m.room.message |   |
                           |   {content}          |   |
                           |   "relates-to"-------|---+  +-------------+
                           +----------------------+ <----|  End Token  |
                                                         +-------------+
--- a/drafts/old-client-server-specification.rst
+++ b/drafts/old-client-server-specification.rst
--- a/drafts/state_resolution.rst
+++ b/drafts/state_resolution.rst
@ -27,6 +27,7 @@ all state events eventually.
 Algorithm requirements
 ----------------------
 We want the algorithm to have the following properties:
 - Since we aren't guaranteed what order we receive state events in, except that
  we see parents before children, the state resolution algorithm must not depend
  on the order and must always come to the same result. 
--- a/drafts/typing_notifications.rst
+++ b/drafts/typing_notifications.rst
@ -7,7 +7,7 @@ Client APIs
 To set "I am typing for the next N msec"::
  PUT .../rooms/:room_id/typing/:user_id
  Content:  { "typing": true, "timeout": N }
-    # timeout is in msec; I suggest no more than 20 or 30 seconds
+  # timeout is in msec; I suggest no more than 20 or 30 seconds
 This should be re-sent by the client to continue informing the server the user
 is still typing; I suggest a safety margin of 5 seconds before the expected
--- a/drafts/use_cases.rst
+++ b/drafts/use_cases.rst
@ -0,0 +1,317 @@
 General UI/UX requirements:
 ===========================
 - Live updates
 - No flicker:
   * Sending message (local echo)
   * Receiving images (encoding w/h)
   * Scrollback
   * Resolving display names (from user ID)
 - Fast startup times
 - Fast "opening room" times (esp. when clicking through from a notification)
 - Low latency file transfer.
 Use cases
 ---------
 - #1: Lightweight IM client (no perm storage) - e.g. Web client
 - #2: Bug tracking software
 - #3: Forum
 - #4: Google + style communities
 - #5: Email style threading
 - #6: Multi-column threaded IM
 - #7: Mobile IM client (perm storage)
 - #8: MIDI client
 - #9: Animatrix client
 - #10: Unity object trees
 - #11: Social Network ("Walls", PMs, groups)
 - #12: Minecraft-clone
 - #13: Global 'Like' widget, which links through to a room.
 #1 Web client UI
 ================
 Model::
 Rooms ----< Messages 
 - name     - type (call/image)
 - topic    
 Home Screen
 What's visible:
  - Recent chats ordered by timestamp of latest event (with # users)
  - Your own display name, user ID and avatar url
  - A searchable list of public rooms (with # users and alias + room name + room topic)
 What you can do:
  - Create a room (public/private, with alias)
  - Join a room from alias
  - Message a user (with user ID)
  - Leave a recent room
  - Open a room
  - Open a chat history link.
  - Search for a public room.
 Chat Screen
 What's visible:
  - Enough scrollback to fill a "screen full" of content.
  - Each message: timestamp, user ID, display name at the time the message was
    sent, avatar URL at the time the message was sent, whether it was a bing message
    or not.
  - User list: for each user: presence, current avatar url in the room, current
    display name in the room, power level, ordered by when they were last speaking.
  - Recents list: (same as Home Screen)
  - Room name
  - Room topic
  - Typing notifications
  - Desktop/Push Notifications for messages
 What you can do:
  - Invite a user
  - Kick a user
  - Ban/Unban a user
  - Leave the room
  - Send a message (image/text/emote)
  - Change someone's power level
  - Change your own display name
  - Accept an incoming call
  - Make an outgoing call
  - Get older messages by scrolling up (scrollback)
  - Redact a message
  - Resend a message which was not sent
 Message sending:
  - Immediate local echo
  - Queue up messages which haven't been sent yet
  - Reordering local echo to where it actually happened
 VoIP:
  - One entry in your display for a call (which may contain duration, type, status)
  - Glare resolution
 Scrollback:
  - Display in reverse chronological order by the originating server's timestamp
  - Terminates at the start of the room (which then makes it impossible to request
    more scrollback)
 Local storage:
  - Driven by desire for fast startup times and minimal network traffic
  - Display messages from storage and from the network without any gaps in messages.
  - Persist scrollback if possible: Scrollback from storage first then from the 
    network.
 Notifications:
  - Receive notifications for rooms you're interested in (explicitly or from a default)
  - Maybe per device.
  - Maybe depending on presence (e.g. idle)
  - Maybe depending on message volume
  - Maybe depending on room config options.
 Message contents:
  - images
  - video
  - rich text
  - audio
  - arbitrary files
  - location
  - vcards (potentially)
 Chat History Screen
 What's visible:
  - The linked message and enough scrollback to fill a "screen full" of content.
  - Each message: timestamp, user ID, display name at the time the message was
    sent, avatar URL at the time the message was sent, whether it was a bing message
    or not.
  - The historical user list. *TODO: Is this taken at the linked message, or at
    wherever the user has scrolled to?*
 What you can do:
  - Get older messages by scrolling up (scrollback)
  - Get newer messages by scrolling down
 Public Room Search Screen
 What's visible:
  - The current search text.
  - The homeserver being searched (defaults to the HS the client is connected to).
  - The results of the current search with enough results to fill the screen
    with # users and alias + room name + room topic.
 What you can do:
  - Change what you are searching for.
  - Change the server that's being searched.
  - Scroll down to get more search results.
 User screen
 What's visible:
  - Display name
  - Avatar
  - User ID
 What you can do:
  - Start a chat with the user
 #2 Bug tracking UI
 ==================
 Model::
 Projects ----< Issues ---< Comments
 - key        - summary     - user
 - name       - ID          - message
  SYN         SYN-52       Fix it nooow!
 Landing page
 What's visible:
  - Issues assigned to me
  - Issues I'm watching
  - Recent activity on other issues (not refined to me)
  - List of projects
 What you can do:
  - View an issue
  - Create an issue
  - Sort issues
  - View a user
  - View a project
  - Search for issues (by name, time, priority, description contents, reporter, etc...)
 Issue page
 What's visible:
  - Summary of issue
  - Issue key
  - Project affected
  - Description
  - Comments
  - Priority, labels, type, purpose, etc..
  - Reporter/assignee
  - Creation and last updated times
  - History of issue changes
 What you can do:
  - Comment on issue
  - Change issue info (labels, type, purpose, etc..)
  - Open/Close/Resolve the issue
  - Edit the issue
  - Watch/Unwatch the issue
 #3 Forum UI
 ===========
 Model::
 Forum ----< Boards ----< Threads ----< Messages
 - Matrix   - Dev        - HALP!        - please halp!
 Main page
 What's visible:
  - Categories (containing boards)
  - Boards (with names and # posts and tagline and latest post)
 What you can do:
  - View a board
  - View the latest message on a board
 Board page
 What's visible:
  - Threads (titles, OP, latest post date+author, # replies, # upvotes, whether 
    the OP contains an image or hyperlink (small icon on title))
  - Whether the thread is answered (with link to the answer)
  - Pagination for posts within a thread (1,2,3,4,5...10)
  - Pagination for threads within a board 
  - List of threads in chronological order
  - Stickied threads
 What you can do:
  - View a user
  - View a thread on a particular page
  - View the latest message on a thread
  - View older threads (pagination)
  - Search the board
 Thread page
 What's visible:
  - Messages in chronological order
  - For each message: author, timestamp, # posts by author, avatar, registration
    date, status message, message contents, # views of message
 What you can do:
  - Upvote the message
  - Flag the message for a mod
  - Reply to the message
  - Subscribe to thread or message's RSS feed
  - Go to previous/next thread
 #4 Google+ community
 ====================
 Model::
 Community -----< Categories ----< Posts ---< Comments
 Kerbal SP       Mods, Help        Text        Text
                                 (no title!)
 Communities page
 What's visible:
  - List of communities
  - For each community: # users, # posts, group pic, title
 What you can do:
  - Join a community
  - View a community
 Community Page
 What's visible:
  - Title, pic
  - List of categories
  - List of members with avatars (+ total #)
  - Most recent posts with comments (most recent comment if >1)
 What you can do:
  - Join the group
  - Post a post (with voting and options)
  - Report abuse
  - View member
  - Expand comments
  - Infinite scrolling
  - Add a comment to a post
  - Share a post
  - +1 a post
 #5 Email style threading
 ========================
 Chat Screen
 What's visible:
  - Enough scrollback to fill a "screen full" of content.
  - Threads:
    - Initially will only display the timestamp and user ID of the *first*
      message. But can expand to show the entire tree.
    - Tree of messages indicating which message is a reply to which.
    - Ordered by the arbitrary field (timestamp of oldest message in thread;
      newest message in thread; sender id; sender display name; etc)
    - Each message: timestamp, user ID, display name at the time of the message
  - Room name
  - Room topic
  - Typing notifications
  - Desktop/Push Notifications for messages
 What you can do:
  - Send a message in reply to another message:
    - Immediate local echo, may cause messages to re-order
    - Messages that haven't reached the server are queued.
    - Thread is displayed where it should be in the thread order once the
      message is sent.
  - Start a new thread by sending a message.
 #6 Multi-threaded IM
 ====================
 Chat Screen
 What's visible:
  - A multi-column grid of threads from a number of chatrooms
    Each concurrent thread is displayed in a different column.
    The columns start and end as threads split and rejoin the main conversation
    The messages for each thread are ordered by how recent they are::
      Room #1        Room # 2           Room # 2
      +------------+ +----------------+ Side thread.
      | * Message1 | | * Root         | +--------------+
      | * Message2 | | * A1 -> Root   | | * B1 -> Root |
      +------------+ | * A2 -> A1     | | * B2 -> B1   |
                     | * M -> A2, B2  | +--------------+
                     +----------------+
  - Typing notifications. Displayed within the correct thread/column.
 What you can do:
   - Send a message into a particular thread/column.
   - Move an *existing* message into a new thread creating a new column
   - Move an existing message into an existing thread, causing the threads to
     reconverge (i.e. provide a route from the sidebar back into the existing
     thread).  This does not imply terminating the thread, which can continue
     independently of the merge.
--- a/jsfiddles/create_room_send_msg/demo.css
+++ b/jsfiddles/create_room_send_msg/demo.css
@ -0,0 +1,17 @@
 .loggedin {
    visibility: hidden;
 }
 p {
    font-family: monospace;
 }
 table
 {
    border-spacing:5px;
 }
 th,td
 {
    padding:5px;
 }
--- a/jsfiddles/create_room_send_msg/demo.html
+++ b/jsfiddles/create_room_send_msg/demo.html
@ -0,0 +1,30 @@
 <div>
    <p>This room creation / message sending demo requires a home server to be running on http://localhost:8008</p>
 </div>
 <form class="loginForm">
    <input type="text" id="userLogin" placeholder="Username"></input>
    <input type="password" id="passwordLogin" placeholder="Password"></input>
    <input type="button" class="login" value="Login"></input>
 </form>
 <div class="loggedin">
    <form class="createRoomForm">
        <input type="text" id="roomAlias" placeholder="Room alias (optional)"></input>
        <input type="button" class="createRoom" value="Create Room"></input>
    </form>
    <form class="sendMessageForm">
        <input type="text" id="roomId" placeholder="Room ID"></input>
        <input type="text" id="messageBody" placeholder="Message body"></input>
        <input type="button" class="sendMessage" value="Send Message"></input>
    </form>
    <table id="rooms">
        <tbody>
            <tr>
                <th>Room ID</th>
                <th>My state</th>
                <th>Room Alias</th>
                <th>Latest message</th>
            </tr>
        </tbody>
    </table>
 </div>
--- a/jsfiddles/create_room_send_msg/demo.js
+++ b/jsfiddles/create_room_send_msg/demo.js
@ -0,0 +1,113 @@
 var accountInfo = {};
 var showLoggedIn = function(data) {
    accountInfo = data;
    getCurrentRoomList();
    $(".loggedin").css({visibility: "visible"});
 };
 $('.login').live('click', function() {
    var user = $("#userLogin").val();
    var password = $("#passwordLogin").val();
    $.ajax({
        url: "http://localhost:8008/_matrix/client/api/v1/login",
        type: "POST",
        contentType: "application/json; charset=utf-8",
        data: JSON.stringify({ user: user, password: password, type: "m.login.password" }),
        dataType: "json",
        success: function(data) {
            showLoggedIn(data);
        },
        error: function(err) {
            var errMsg = "To try this, you need a home server running!";
            var errJson = $.parseJSON(err.responseText);
            if (errJson) {
                errMsg = JSON.stringify(errJson);   
            }
            alert(errMsg);  
        }
    }); 
 });
 var getCurrentRoomList = function() {
    var url = "http://localhost:8008/_matrix/client/api/v1/initialSync?access_token=" + accountInfo.access_token + "&limit=1";
    $.getJSON(url, function(data) {
        var rooms = data.rooms;
        for (var i=0; i<rooms.length; ++i) {
            rooms[i].latest_message = rooms[i].messages.chunk[0].content.body;
            addRoom(rooms[i]);   
        }
    }).fail(function(err) {
        alert(JSON.stringify($.parseJSON(err.responseText)));
    });
 };
 $('.createRoom').live('click', function() {
    var roomAlias = $("#roomAlias").val();
    var data = {};
    if (roomAlias.length > 0) {
        data.room_alias_name = roomAlias;   
    }
    $.ajax({
        url: "http://localhost:8008/_matrix/client/api/v1/createRoom?access_token="+accountInfo.access_token,
        type: "POST",
        contentType: "application/json; charset=utf-8",
        data: JSON.stringify(data),
        dataType: "json",
        success: function(data) {
            data.membership = "join"; // you are automatically joined into every room you make.
            data.latest_message = "";
            addRoom(data);
        },
        error: function(err) {
            alert(JSON.stringify($.parseJSON(err.responseText)));  
        }
    }); 
 });
 var addRoom = function(data) {
    row = "<tr>" +
        "<td>"+data.room_id+"</td>" +
        "<td>"+data.membership+"</td>" +
        "<td>"+data.room_alias+"</td>" +
        "<td>"+data.latest_message+"</td>" +
        "</tr>";
    $("#rooms").append(row);
 };
 $('.sendMessage').live('click', function() {
    var roomId = $("#roomId").val();
    var body = $("#messageBody").val();
    var msgId = $.now();
    if (roomId.length === 0 || body.length === 0) {
        return;
    }
    var url = "http://localhost:8008/_matrix/client/api/v1/rooms/$roomid/send/m.room.message?access_token=$token";
    url = url.replace("$token", accountInfo.access_token);
    url = url.replace("$roomid", encodeURIComponent(roomId));
    var data = {
        msgtype: "m.text",
        body: body
    };
    $.ajax({
        url: url,
        type: "POST",
        contentType: "application/json; charset=utf-8",
        data: JSON.stringify(data),
        dataType: "json",
        success: function(data) {
            $("#messageBody").val("");
            // wipe the table and reload it. Using the event stream would be the best
            // solution but that is out of scope of this fiddle.
            $("#rooms").find("tr:gt(0)").remove();
            getCurrentRoomList();
        },
        error: function(err) {
            alert(JSON.stringify($.parseJSON(err.responseText)));  
        }
    }); 
 });
--- a/jsfiddles/event_stream/demo.css
+++ b/jsfiddles/event_stream/demo.css
@ -0,0 +1,17 @@
 .loggedin {
    visibility: hidden;
 }
 p {
    font-family: monospace;
 }
 table
 {
    border-spacing:5px;
 }
 th,td
 {
    padding:5px;
 }
--- a/jsfiddles/event_stream/demo.html
+++ b/jsfiddles/event_stream/demo.html
@ -0,0 +1,23 @@
 <div>
    <p>This event stream demo requires a home server to be running on http://localhost:8008</p>
 </div>
 <form class="loginForm">
    <input type="text" id="userLogin" placeholder="Username"></input>
    <input type="password" id="passwordLogin" placeholder="Password"></input>
    <input type="button" class="login" value="Login"></input>
 </form>
 <div class="loggedin">
    <form class="sendMessageForm">
        <input type="button" class="sendMessage" value="Send random message"></input>
    </form>
    <p id="streamErrorText"></p>
    <table id="rooms">
        <tbody>
            <tr>
                <th>Room ID</th>
                <th>Latest message</th>
            </tr>
        </tbody>
    </table>
 </div>
--- a/jsfiddles/event_stream/demo.js
+++ b/jsfiddles/event_stream/demo.js
@ -0,0 +1,145 @@
 var accountInfo = {};
 var eventStreamInfo = {
    from: "END"
 };
 var roomInfo = [];
 var longpollEventStream = function() {
    var url = "http://localhost:8008/_matrix/client/api/v1/events?access_token=$token&from=$from";
    url = url.replace("$token", accountInfo.access_token);
    url = url.replace("$from", eventStreamInfo.from);
    $.getJSON(url, function(data) {
        eventStreamInfo.from = data.end;
        var hasNewLatestMessage = false;
        for (var i=0; i<data.chunk.length; ++i) {
            if (data.chunk[i].type === "m.room.message") {
                for (var j=0; j<roomInfo.length; ++j) {
                    if (roomInfo[j].room_id === data.chunk[i].room_id) {
                        roomInfo[j].latest_message = data.chunk[i].content.body;
                        hasNewLatestMessage = true;
                    }
                }
            }
        }
        if (hasNewLatestMessage) {
           setRooms(roomInfo);
        }
        $("#streamErrorText").text("");
        longpollEventStream();
    }).fail(function(err) {
        $("#streamErrorText").text("Event stream error: "+JSON.stringify($.parseJSON(err.responseText)));
        setTimeout(longpollEventStream, 5000);
    });
 };
 var showLoggedIn = function(data) {
    accountInfo = data;
    longpollEventStream();
    getCurrentRoomList();
    $(".loggedin").css({visibility: "visible"});
 };
 $('.login').live('click', function() {
    var user = $("#userLogin").val();
    var password = $("#passwordLogin").val();
    $.ajax({
        url: "http://localhost:8008/_matrix/client/api/v1/login",
        type: "POST",
        contentType: "application/json; charset=utf-8",
        data: JSON.stringify({ user: user, password: password, type: "m.login.password" }),
        dataType: "json",
        success: function(data) {
            $("#rooms").find("tr:gt(0)").remove();
            showLoggedIn(data);
        },
        error: function(err) {
            var errMsg = "To try this, you need a home server running!";
            var errJson = $.parseJSON(err.responseText);
            if (errJson) {
                errMsg = JSON.stringify(errJson);   
            }
            alert(errMsg);  
        }
    }); 
 });
 var getCurrentRoomList = function() {
    $("#roomId").val("");
    var url = "http://localhost:8008/_matrix/client/api/v1/initialSync?access_token=" + accountInfo.access_token + "&limit=1";
    $.getJSON(url, function(data) {
        var rooms = data.rooms;
        for (var i=0; i<rooms.length; ++i) {
            if ("messages" in rooms[i]) {
                rooms[i].latest_message = rooms[i].messages.chunk[0].content.body;   
            }
        }
        roomInfo = rooms;
        setRooms(roomInfo);  
    }).fail(function(err) {
        alert(JSON.stringify($.parseJSON(err.responseText)));
    });
 };
 $('.sendMessage').live('click', function() {
    if (roomInfo.length === 0) {
        alert("There is no room to send a message to!");
        return;
    }
    var index = Math.floor(Math.random() * roomInfo.length);
    sendMessage(roomInfo[index].room_id);
 });
 var sendMessage = function(roomId) {
    var body = "jsfiddle message @" + $.now();
    if (roomId.length === 0) {
        return;
    }
    var url = "http://localhost:8008/_matrix/client/api/v1/rooms/$roomid/send/m.room.message?access_token=$token";
    url = url.replace("$token", accountInfo.access_token);
    url = url.replace("$roomid", encodeURIComponent(roomId));
    var data = {
        msgtype: "m.text",
        body: body
    };
    $.ajax({
        url: url,
        type: "POST",
        contentType: "application/json; charset=utf-8",
        data: JSON.stringify(data),
        dataType: "json",
        success: function(data) {
            $("#messageBody").val("");
        },
        error: function(err) {
            alert(JSON.stringify($.parseJSON(err.responseText)));  
        }
    }); 
 };
 var setRooms = function(roomList) {
    // wipe existing entries
    $("#rooms").find("tr:gt(0)").remove();
    var rows = "";
    for (var i=0; i<roomList.length; ++i) {
        row = "<tr>" +
            "<td>"+roomList[i].room_id+"</td>" +
            "<td>"+roomList[i].latest_message+"</td>" +
            "</tr>";  
        rows += row;
    }
    $("#rooms").append(rows);
 };
--- a/jsfiddles/example_app/demo.css
+++ b/jsfiddles/example_app/demo.css
@ -0,0 +1,43 @@
 .roomListDashboard, .roomContents, .sendMessageForm {
    visibility: hidden;
 }
 .roomList {
    background-color: #909090;
 }
 .messageWrapper {
    background-color: #EEEEEE;
    height: 400px;
    overflow: scroll;
 }
 .membersWrapper {
    background-color: #EEEEEE;
    height: 200px;
    width: 50%;
    overflow: scroll;
 }
 .textEntry {
    width: 100%
 }
 p {
    font-family: monospace;
 }
 table
 {
    border-spacing:5px;
 }
 th,td
 {
    padding:5px;
 }
 .roomList tr:not(:first-child):hover {
    background-color: orange;
    cursor: pointer;
 }
--- a/jsfiddles/example_app/demo.details
+++ b/jsfiddles/example_app/demo.details
@ -0,0 +1,7 @@
 name: Example Matrix Client
 description: Includes login, live event streaming, creating rooms, sending messages and viewing member lists.
 authors:
   - matrix.org
 resources:
   - http://matrix.org
 normalize_css: no
--- a/jsfiddles/example_app/demo.html
+++ b/jsfiddles/example_app/demo.html
@ -0,0 +1,56 @@
 <div class="signUp">
    <p>Matrix example application: Requires a local home server running at http://localhost:8008</p>
    <form class="registrationForm">
        <p>No account? Register:</p>
        <input type="text" id="userReg" placeholder="Username"></input>
        <input type="password" id="passwordReg" placeholder="Password"></input>
        <input type="button" class="register" value="Register"></input>
    </form>
    <form class="loginForm">
        <p>Got an account? Login:</p>
        <input type="text" id="userLogin" placeholder="Username"></input>
        <input type="password" id="passwordLogin" placeholder="Password"></input>
        <input type="button" class="login" value="Login"></input>
    </form>
 </div>
 <div class="roomListDashboard">
    <form class="createRoomForm">
        <input type="text" id="roomAlias" placeholder="Room alias"></input>
        <input type="button" class="createRoom" value="Create Room"></input>
    </form>
    <table id="rooms" class="roomList">
        <tbody>
            <tr>
                <th>Room</th>
                <th>My state</th>
                <th>Latest message</th>
            </tr>
        </tbody>
    </table>
 </div>
 <div class="roomContents">
    <p id="roomName">Select a room</p>
    <div class="messageWrapper">
        <table id="messages">
            <tbody>
            </tbody>
        </table>
    </div>
    <form class="sendMessageForm">
        <input type="text" class="textEntry" id="body" placeholder="Enter text here..." onkeydown="javascript:if (event.keyCode == 13) document.getElementById('sendMsg').focus()"></input>
        <input type="button" class="sendMessage" id="sendMsg" value="Send"></input>
    </form>
 </div>
 <div>
    <p>Member list:</p>
    <div class="membersWrapper">
        <table id="members">
            <tbody>
            </tbody>
        </table>
    </div>
 </div>
--- a/jsfiddles/example_app/demo.js
+++ b/jsfiddles/example_app/demo.js
@ -0,0 +1,327 @@
 var accountInfo = {};
 var eventStreamInfo = {
    from: "END"
 };
 var roomInfo = [];
 var memberInfo = [];
 var viewingRoomId;
 // ************** Event Streaming **************
 var longpollEventStream = function() {
    var url = "http://localhost:8008/_matrix/client/api/v1/events?access_token=$token&from=$from";
    url = url.replace("$token", accountInfo.access_token);
    url = url.replace("$from", eventStreamInfo.from);
    $.getJSON(url, function(data) {
        eventStreamInfo.from = data.end;
        var hasNewLatestMessage = false;
        var updatedMemberList = false;
        var i=0;
        var j=0;
        for (i=0; i<data.chunk.length; ++i) {
            if (data.chunk[i].type === "m.room.message") {
                console.log("Got new message: " + JSON.stringify(data.chunk[i]));
                if (viewingRoomId === data.chunk[i].room_id) {
                    addMessage(data.chunk[i]);
                }
                for (j=0; j<roomInfo.length; ++j) {
                    if (roomInfo[j].room_id === data.chunk[i].room_id) {
                        roomInfo[j].latest_message = data.chunk[i].content.body;
                        hasNewLatestMessage = true;
                    }
                }
            }
            else if (data.chunk[i].type === "m.room.member") {
                if (viewingRoomId === data.chunk[i].room_id) {
                    console.log("Got new member: " + JSON.stringify(data.chunk[i]));
                    addMessage(data.chunk[i]);
                    for (j=0; j<memberInfo.length; ++j) {
                        if (memberInfo[j].state_key === data.chunk[i].state_key) {
                            memberInfo[j] = data.chunk[i];
                            updatedMemberList = true;
                            break;
                        }
                    }
                    if (!updatedMemberList) {
                        memberInfo.push(data.chunk[i]);  
                        updatedMemberList = true;
                    }
                }
                if (data.chunk[i].state_key === accountInfo.user_id) {
                    getCurrentRoomList(); // update our join/invite list
                }
            }
            else {
                console.log("Discarding: " + JSON.stringify(data.chunk[i]));
            }
        }
        if (hasNewLatestMessage) {
           setRooms(roomInfo);
        }
        if (updatedMemberList) {
            $("#members").empty();
            for (i=0; i<memberInfo.length; ++i) { 
                addMember(memberInfo[i]);
            }
        }
        longpollEventStream();
    }).fail(function(err) {
        setTimeout(longpollEventStream, 5000);
    });
 };
 // ************** Registration and Login **************
 var onLoggedIn = function(data) {
    accountInfo = data;
    longpollEventStream();
    getCurrentRoomList();
    $(".roomListDashboard").css({visibility: "visible"});
    $(".roomContents").css({visibility: "visible"});
    $(".signUp").css({display: "none"});
 };
 $('.login').live('click', function() {
    var user = $("#userLogin").val();
    var password = $("#passwordLogin").val();
    $.ajax({
        url: "http://localhost:8008/_matrix/client/api/v1/login",
        type: "POST",
        contentType: "application/json; charset=utf-8",
        data: JSON.stringify({ user: user, password: password, type: "m.login.password" }),
        dataType: "json",
        success: function(data) {
            onLoggedIn(data);
        },
        error: function(err) {
            alert("Unable to login: is the home server running?");  
        }
    }); 
 });
 $('.register').live('click', function() {
    var user = $("#userReg").val();
    var password = $("#passwordReg").val();
    $.ajax({
        url: "http://localhost:8008/_matrix/client/api/v1/register",
        type: "POST",
        contentType: "application/json; charset=utf-8",
        data: JSON.stringify({ user: user, password: password, type: "m.login.password" }),
        dataType: "json",
        success: function(data) {
            onLoggedIn(data);
        },
        error: function(err) {
            var msg = "Is the home server running?";
            var errJson = $.parseJSON(err.responseText);
            if (errJson !== null) {
                msg = errJson.error;   
            }
            alert("Unable to register: "+msg);  
        }
    });
 });
 // ************** Creating a room ******************
 $('.createRoom').live('click', function() {
    var roomAlias = $("#roomAlias").val();
    var data = {};
    if (roomAlias.length > 0) {
        data.room_alias_name = roomAlias;   
    }
    $.ajax({
        url: "http://localhost:8008/_matrix/client/api/v1/createRoom?access_token="+accountInfo.access_token,
        type: "POST",
        contentType: "application/json; charset=utf-8",
        data: JSON.stringify(data),
        dataType: "json",
        success: function(response) {
            $("#roomAlias").val("");
            response.membership = "join"; // you are automatically joined into every room you make.
            response.latest_message = "";
            roomInfo.push(response);
            setRooms(roomInfo);
        },
        error: function(err) {
            alert(JSON.stringify($.parseJSON(err.responseText)));  
        }
    }); 
 });
 // ************** Getting current state **************
 var getCurrentRoomList = function() {
    var url = "http://localhost:8008/_matrix/client/api/v1/initialSync?access_token=" + accountInfo.access_token + "&limit=1";
    $.getJSON(url, function(data) {
        var rooms = data.rooms;
        for (var i=0; i<rooms.length; ++i) {
            if ("messages" in rooms[i]) {
                rooms[i].latest_message = rooms[i].messages.chunk[0].content.body;   
            }
        }
        roomInfo = rooms;
        setRooms(roomInfo);  
    }).fail(function(err) {
        alert(JSON.stringify($.parseJSON(err.responseText)));
    });
 };
 var loadRoomContent = function(roomId) {
    console.log("loadRoomContent " + roomId);
    viewingRoomId = roomId;
    $("#roomName").text("Room: "+roomId);
    $(".sendMessageForm").css({visibility: "visible"});
    getMessages(roomId);
    getMemberList(roomId);
 };
 var getMessages = function(roomId) {
    $("#messages").empty();
    var url = "http://localhost:8008/_matrix/client/api/v1/rooms/" + 
              encodeURIComponent(roomId) + "/messages?access_token=" + accountInfo.access_token + "&from=END&dir=b&limit=10";
    $.getJSON(url, function(data) {
        for (var i=data.chunk.length-1; i>=0; --i) {
            addMessage(data.chunk[i]);   
        }
    });
 };
 var getMemberList = function(roomId) {
    $("#members").empty();
    memberInfo = [];
    var url = "http://localhost:8008/_matrix/client/api/v1/rooms/" + 
              encodeURIComponent(roomId) + "/members?access_token=" + accountInfo.access_token;
    $.getJSON(url, function(data) {
        for (var i=0; i<data.chunk.length; ++i) {
            memberInfo.push(data.chunk[i]);
            addMember(data.chunk[i]);   
        }
    });
 };
 // ************** Sending messages **************
 $('.sendMessage').live('click', function() {
    if (viewingRoomId === undefined) {
        alert("There is no room to send a message to!");
        return;
    }
    var body = $("#body").val();
    sendMessage(viewingRoomId, body);
 });
 var sendMessage = function(roomId, body) {
    var msgId = $.now();
    var url = "http://localhost:8008/_matrix/client/api/v1/rooms/$roomid/send/m.room.message?access_token=$token";
    url = url.replace("$token", accountInfo.access_token);
    url = url.replace("$roomid", encodeURIComponent(roomId));
    var data = {
        msgtype: "m.text",
        body: body
    };
    $.ajax({
        url: url,
        type: "POST",
        contentType: "application/json; charset=utf-8",
        data: JSON.stringify(data),
        dataType: "json",
        success: function(data) {
            $("#body").val("");
        },
        error: function(err) {
            alert(JSON.stringify($.parseJSON(err.responseText)));  
        }
    });
 };
 // ************** Navigation and DOM manipulation **************
 var setRooms = function(roomList) {
    // wipe existing entries
    $("#rooms").find("tr:gt(0)").remove();
    var rows = "";
    for (var i=0; i<roomList.length; ++i) {
        row = "<tr>" +
              "<td>"+roomList[i].room_id+"</td>" +
              "<td>"+roomList[i].membership+"</td>" +
              "<td>"+roomList[i].latest_message+"</td>" +
              "</tr>";  
        rows += row;
    }
    $("#rooms").append(rows);
    $('#rooms').find("tr").click(function(){
        var roomId = $(this).find('td:eq(0)').text();
        var membership = $(this).find('td:eq(1)').text();
        if (membership !== "join") {
            console.log("Joining room " + roomId); 
            var url = "http://localhost:8008/_matrix/client/api/v1/rooms/$roomid/join?access_token=$token";
            url = url.replace("$token", accountInfo.access_token);
            url = url.replace("$roomid", encodeURIComponent(roomId));
            $.ajax({
                url: url,
                type: "POST",
                contentType: "application/json; charset=utf-8",
                data: JSON.stringify({membership: "join"}),
                dataType: "json",
                success: function(data) {
                    loadRoomContent(roomId);
                    getCurrentRoomList();
                },
                error: function(err) {
                    alert(JSON.stringify($.parseJSON(err.responseText)));  
                }
            });
        }
        else {
            loadRoomContent(roomId);
        }
    });
 };
 var addMessage = function(data) {
    var msg = data.content.body;
    if (data.type === "m.room.member") {
        if (data.content.membership === undefined) {
            return;
        }
        if (data.content.membership === "invite") {
            msg = "<em>invited " + data.state_key + " to the room</em>";
        }
        else if (data.content.membership === "join") {
            msg = "<em>joined the room</em>";
        }
        else if (data.content.membership === "leave") {
            msg = "<em>left the room</em>";
        }
        else if (data.content.membership === "ban") {
            msg = "<em>was banned from the room</em>";
        }
    }
    if (msg === undefined) {
        return;
    }
    var row = "<tr>" +
              "<td>"+data.user_id+"</td>" +
              "<td>"+msg+"</td>" +
              "</tr>"; 
    $("#messages").append(row);
 };
 var addMember = function(data) {
    var row = "<tr>" +
              "<td>"+data.state_key+"</td>" +
              "<td>"+data.content.membership+"</td>" +
              "</tr>"; 
    $("#members").append(row);
 };
--- a/jsfiddles/register_login/demo.css
+++ b/jsfiddles/register_login/demo.css
@ -0,0 +1,7 @@
 .loggedin {
    visibility: hidden;
 }
 p {
    font-family: monospace;
 }
--- a/jsfiddles/register_login/demo.html
+++ b/jsfiddles/register_login/demo.html
@ -0,0 +1,20 @@
 <div>
    <p>This registration/login demo requires a home server to be running on http://localhost:8008</p>
 </div>
 <form class="registrationForm">
    <input type="text" id="user" placeholder="Username"></input>
    <input type="password" id="password" placeholder="Password"></input>
    <input type="button" class="register" value="Register"></input>
 </form>
 <form class="loginForm">
    <input type="text" id="userLogin" placeholder="Username"></input>
    <input type="password" id="passwordLogin" placeholder="Password"></input>
    <input type="button" class="login" value="Login"></input>
 </form>
 <div class="loggedin">
    <p id="welcomeText"></p>
    <input type="button" class="testToken" value="Test token"></input>
    <input type="button" class="logout" value="Logout"></input>
    <p id="imSyncText"></p>
 </div>
--- a/jsfiddles/register_login/demo.js
+++ b/jsfiddles/register_login/demo.js
@ -0,0 +1,79 @@
 var accountInfo = {};
 var showLoggedIn = function(data) {
    accountInfo = data;
    $(".loggedin").css({visibility: "visible"});
    $("#welcomeText").text("Welcome " + accountInfo.user_id+". Your access token is:  " +
                           accountInfo.access_token);    
 };
 $('.register').live('click', function() {
    var user = $("#user").val();
    var password = $("#password").val();
    $.ajax({
        url: "http://localhost:8008/_matrix/client/api/v1/register",
        type: "POST",
        contentType: "application/json; charset=utf-8",
        data: JSON.stringify({ user: user, password: password, type: "m.login.password" }),
        dataType: "json",
        success: function(data) {
            showLoggedIn(data);
        },
        error: function(err) {
            var errMsg = "To try this, you need a home server running!";
            var errJson = $.parseJSON(err.responseText);
            if (errJson) {
                errMsg = JSON.stringify(errJson);   
            }
            alert(errMsg);   
        }
    });
 });
 var login = function(user, password) {
    $.ajax({
        url: "http://localhost:8008/_matrix/client/api/v1/login",
        type: "POST",
        contentType: "application/json; charset=utf-8",
        data: JSON.stringify({ user: user, password: password, type: "m.login.password" }),
        dataType: "json",
        success: function(data) {
            showLoggedIn(data);
        },
        error: function(err) {
            var errMsg = "To try this, you need a home server running!";
            var errJson = $.parseJSON(err.responseText);
            if (errJson) {
                errMsg = JSON.stringify(errJson);   
            }
            alert(errMsg);  
        }
    });  
 };
 $('.login').live('click', function() {
    var user = $("#userLogin").val();
    var password = $("#passwordLogin").val();
    $.getJSON("http://localhost:8008/_matrix/client/api/v1/login", function(data) {
        if (data.flows[0].type !== "m.login.password") {
            alert("I don't know how to login with this type: " + data.type);
            return;
        }
        login(user, password);
    });
 });
 $('.logout').live('click', function() {
    accountInfo = {};
    $("#imSyncText").text("");
    $(".loggedin").css({visibility: "hidden"});
 });
 $('.testToken').live('click', function() {
    var url = "http://localhost:8008/_matrix/client/api/v1/initialSync?access_token=" + accountInfo.access_token + "&limit=1";
    $.getJSON(url, function(data) {
         $("#imSyncText").text(JSON.stringify(data, undefined, 2));
    }).fail(function(err) {
        $("#imSyncText").text(JSON.stringify($.parseJSON(err.responseText)));
    });
 });
--- a/jsfiddles/room_memberships/demo.css
+++ b/jsfiddles/room_memberships/demo.css
@ -0,0 +1,17 @@
 .loggedin {
    visibility: hidden;
 }
 p {
    font-family: monospace;
 }
 table
 {
    border-spacing:5px;
 }
 th,td
 {
    padding:5px;
 }
--- a/jsfiddles/room_memberships/demo.html
+++ b/jsfiddles/room_memberships/demo.html
@ -0,0 +1,37 @@
 <div>
    <p>This room membership demo requires a home server to be running on http://localhost:8008</p>
 </div>
 <form class="loginForm">
    <input type="text" id="userLogin" placeholder="Username"></input>
    <input type="password" id="passwordLogin" placeholder="Password"></input>
    <input type="button" class="login" value="Login"></input>
 </form>
 <div class="loggedin">
    <form class="createRoomForm">
        <input type="button" class="createRoom" value="Create Room"></input>
    </form>
    <form class="changeMembershipForm">
        <input type="text" id="roomId" placeholder="Room ID"></input>
        <input type="text" id="targetUser" placeholder="Target User ID"></input>
        <select id="membership">
            <option value="invite">invite</option>
            <option value="join">join</option>
            <option value="leave">leave</option>
        </select>
        <input type="button" class="changeMembership" value="Change Membership"></input>
    </form>
    <form class="joinAliasForm">
        <input type="text" id="roomAlias" placeholder="Room Alias (#name:domain)"></input>
        <input type="button" class="joinAlias" value="Join via Alias"></input>
    </form>
    <table id="rooms">
        <tbody>
            <tr>
                <th>Room ID</th>
                <th>My state</th>
                <th>Room Alias</th>
            </tr>
        </tbody>
    </table>
 </div>
--- a/jsfiddles/room_memberships/demo.js
+++ b/jsfiddles/room_memberships/demo.js
@ -0,0 +1,141 @@
 var accountInfo = {};
 var showLoggedIn = function(data) {
    accountInfo = data;
    getCurrentRoomList();
    $(".loggedin").css({visibility: "visible"});
    $("#membership").change(function() {
    if ($("#membership").val() === "invite") {
        $("#targetUser").css({visibility: "visible"});
    }
    else {
        $("#targetUser").css({visibility: "hidden"});
    }
 });
 };
 $('.login').live('click', function() {
    var user = $("#userLogin").val();
    var password = $("#passwordLogin").val();
    $.ajax({
        url: "http://localhost:8008/_matrix/client/api/v1/login",
        type: "POST",
        contentType: "application/json; charset=utf-8",
        data: JSON.stringify({ user: user, password: password, type: "m.login.password" }),
        dataType: "json",
        success: function(data) {
            $("#rooms").find("tr:gt(0)").remove();
            showLoggedIn(data);
        },
        error: function(err) {
            var errMsg = "To try this, you need a home server running!";
            var errJson = $.parseJSON(err.responseText);
            if (errJson) {
                errMsg = JSON.stringify(errJson);   
            }
            alert(errMsg); 
        }
    }); 
 });
 var getCurrentRoomList = function() {
    $("#roomId").val("");
    // wipe the table and reload it. Using the event stream would be the best
    // solution but that is out of scope of this fiddle.
    $("#rooms").find("tr:gt(0)").remove();
    var url = "http://localhost:8008/_matrix/client/api/v1/initialSync?access_token=" + accountInfo.access_token + "&limit=1";
    $.getJSON(url, function(data) {
        var rooms = data.rooms;
        for (var i=0; i<rooms.length; ++i) {
            addRoom(rooms[i]);   
        }
    }).fail(function(err) {
        alert(JSON.stringify($.parseJSON(err.responseText)));
    });
 };
 $('.createRoom').live('click', function() {
    var data = {};
    $.ajax({
        url: "http://localhost:8008/_matrix/client/api/v1/createRoom?access_token="+accountInfo.access_token,
        type: "POST",
        contentType: "application/json; charset=utf-8",
        data: JSON.stringify(data),
        dataType: "json",
        success: function(data) {
            data.membership = "join"; // you are automatically joined into every room you make.
            data.latest_message = "";
            addRoom(data);
        },
        error: function(err) {
            alert(JSON.stringify($.parseJSON(err.responseText)));  
        }
    }); 
 });
 var addRoom = function(data) {
    row = "<tr>" +
        "<td>"+data.room_id+"</td>" +
        "<td>"+data.membership+"</td>" +
        "<td>"+data.room_alias+"</td>" +
        "</tr>";
    $("#rooms").append(row);
 };
 $('.changeMembership').live('click', function() {
    var roomId = $("#roomId").val();
    var member = $("#targetUser").val();
    var membership = $("#membership").val();
    if (roomId.length === 0) {
        return;
    }
    var url = "http://localhost:8008/_matrix/client/api/v1/rooms/$roomid/$membership?access_token=$token";
    url = url.replace("$token", accountInfo.access_token);
    url = url.replace("$roomid", encodeURIComponent(roomId));
    url = url.replace("$membership", membership);
    var data = {};
    if (membership === "invite") {
        data = {
            user_id: member
        };
    }
    $.ajax({
        url: url,
        type: "POST",
        contentType: "application/json; charset=utf-8",
        data: JSON.stringify(data),
        dataType: "json",
        success: function(data) {
            getCurrentRoomList();
        },
        error: function(err) {
            alert(JSON.stringify($.parseJSON(err.responseText)));  
        }
    }); 
 });
 $('.joinAlias').live('click', function() {
    var roomAlias = $("#roomAlias").val();
    var url = "http://localhost:8008/_matrix/client/api/v1/join/$roomalias?access_token=$token";
    url = url.replace("$token", accountInfo.access_token);
    url = url.replace("$roomalias", encodeURIComponent(roomAlias));
    $.ajax({
        url: url,
        type: "POST",
        contentType: "application/json; charset=utf-8",
        data: JSON.stringify({}),
        dataType: "json",
        success: function(data) {
            getCurrentRoomList();
        },
        error: function(err) {
            alert(JSON.stringify($.parseJSON(err.responseText)));  
        }
    }); 
 });
--- a/scripts/gendoc.sh
+++ b/scripts/gendoc.sh
@ -18,6 +18,6 @@ perl -pi -e 's#<head>#<head><link rel="stylesheet" href="/site.css">#' $MATRIXDO
 perl -MFile::Slurp -pi -e 'BEGIN { $nav = read_file("'$MATRIXDOTORG'/includes/nav.html") } s#<body>#<body><div id="header"><div id="headerContent">$nav</div></div><div id="page"><div id="wrapper"><div style="text-align: center; padding: 40px;"><a href="/"><img src="/matrix.png" width="305" height="130" alt="[matrix]"/></a></div>#' $MATRIXDOTORG/docs/spec/index.html $MATRIXDOTORG/docs/howtos/client-server.html
-perl -pi -e 's#</body>#</div></div><div id="footer"><div id="footerContent">&copy 2014 Matrix.org</div></div></body>#' $MATRIXDOTORG/docs/spec/index.html $MATRIXDOTORG/docs/howtos/client-server.html
+perl -pi -e 's#</body>#</div></div><div id="footer"><div id="footerContent">&copy 2014-2015 Matrix.org</div></div></body>#' $MATRIXDOTORG/docs/spec/index.html $MATRIXDOTORG/docs/howtos/client-server.html
-scp -r $MATRIXDOTORG/docs matrix@ldc-prd-matrix-001:/sites/matrix
+scp -r $MATRIXDOTORG/docs matrix@ldc-prd-matrix-001:/sites/matrix
--- a/specification/00_basis.rst
+++ b/specification/00_basis.rst
@ -134,7 +134,7 @@ federate with other HSes.  It is typically responsible for multiple clients.
 more home servers.
 Events
-++++++
+~~~~~~
 Data in Matrix is encapsulated in an "event". An event is an action within the
 system. Typically each action (e.g. sending a message) correlates with exactly
@ -148,7 +148,7 @@ is the event type for instant messages. Events are usually sent in the context
 of a "Room".
 Event Graphs
-++++++++++++
+~~~~~~~~~~~~
 Each event has a list of zero or more `parent` events. These relations form
 directed acyclic graphs of events called `event graphs`. Every event graph has a single root event, and each event graph forms the 
@ -247,7 +247,7 @@ participating in a room.
 Room Aliases
-~~~~~~~~~~~~
++++++++++++
 Each room can also have multiple "Room Aliases", which looks like::
@ -282,7 +282,7 @@ that are in the room that can be used to join via.
   |________________________________|
 Identity
-~~~~~~~~
++++++++
 Users in Matrix are identified via their matrix user ID (MXID). However,
 existing 3rd party ID namespaces can also be used in order to identify Matrix
@ -303,7 +303,7 @@ the Matrix ecosystem. However, without one clients will not be able to look up
 user IDs using 3PIDs.
 Presence
-~~~~~~~~
++++++++
 Each user has the concept of presence information. This encodes the
 "availability" of that user, suitable for display on other user's clients. This
@ -343,7 +343,7 @@ the other direction it will not). This timestamp is presented via a key called
 message is generated/emitted that the user was last seen active.
 Presence List
-+++++++++++++
+~~~~~~~~~~~~~
 Each user's home server stores a "presence list". This stores a list of user IDs
 whose presence the user wants to follow.
@ -353,7 +353,7 @@ and accept the invitation. Once accepted, both user's HSes track the
 subscription.
 Presence and Permissions
-++++++++++++++++++++++++
+~~~~~~~~~~~~~~~~~~~~~~~~
 For a viewing user to be allowed to see the presence information of a target
 user, either:
@ -367,7 +367,7 @@ presence information in a user list for a room.
 Profiles
-~~~~~~~~
++++++++
 .. TODO-spec
  - Metadata extensibility
--- a/specification/10_events.rst
+++ b/specification/10_events.rst
@ -330,6 +330,22 @@ outlined below:
  Example:
    ``{ "msgtype": "m.emote", "body": "tries to come up with a witty explanation" }``
 ``m.notice``
  Required keys:
    - ``body`` : "string" - The body of the message.
  Optional keys:
    None.
  Example:
    ``{ "msgype": "m.notice", "body": "some kind of automated announcement" }``
  A ``m.notice`` message should be considered similar to a plain ``m.text``
  message except that clients should visually distinguish it in some way. It is
  intended to be used by automated clients, such as bots, bridges, and other
  entities, rather than humans. Additionally, such automated agents which watch
  a room for messages and respond to them ought to ignore ``m.notice`` messages.
  This helps to prevent infinite-loop situations where two automated clients
  continuously exchange messages, as each responds to the other.
 ``m.image``
  Required keys:
    - ``url`` : "string" - The URL to the image.
@ -428,7 +444,7 @@ outlined below:
        "mimetype" : "string (e.g. image/jpeg)",
      }
-.. TODO-spec::
+.. TODO-spec
    Make the definitions "inherit" from FileInfo where necessary...
--- a/specification/11_event_signing.rst
+++ b/specification/11_event_signing.rst
@ -38,7 +38,7 @@ using this representation.
     ).encode("UTF-8")
 Grammar
-+++++++
+~~~~~~~
 Adapted from the grammar in http://tools.ietf.org/html/rfc7159 removing
 insignificant whitespace, fractions, exponents and redundant character escapes
@ -76,7 +76,7 @@ the signature for that sequence and then adding the signature to the original
 JSON object.
 Signing Details
-+++++++++++++++
+~~~~~~~~~~~~~~~
 JSON is signed by encoding the JSON object without ``signatures`` or keys grouped
 as ``unsigned``, using the canonical encoding described above. The JSON bytes are then signed using the
@ -133,7 +133,7 @@ and additional signatures.
      return json_object
 Checking for a Signature
-++++++++++++++++++++++++
+~~~~~~~~~~~~~~~~~~~~~~~~
 To check if an entity has signed a JSON object a server does the following