Add instant_messaging module; modify batesian section rules

Previously, all `m.room.*` events were wodged into `{{room_events}}` which
isn't great when you want to pull specific ones out. Batesian had a 1:1
mapping of `render_foo()` to a section `{{foo}}`, and having to constantly
add functions for new types is a PITA. Batesian now supports returning a
`dict` instead of a section `string` where the keys are the `{{foo}}` and
the value is what will be inserted. Also add conflicting section key checks
to avoid multiple definitions of the same `{{foo}}`. Define dicts for
event schemata and swagger HTTP APIs.

Using this new feature, split out the instant messaging stuff from the events
section, and replace `{{room_events}}` with a list of specific events e.g.
`{{m_room_member_event}}`.

specification/0-events.rst

@@ -20,71 +20,19 @@ Room Events
 This specification outlines several standard event types, all of which are
 prefixed with ``m.``
 
-{{room_events}}
+{{m_room_aliases_event}}
 
-m.room.message msgtypes
-~~~~~~~~~~~~~~~~~~~~~~~
+{{m_room_canonical_aliases_event}}
 
-.. TODO-spec
-   How a client should handle unknown message types.
+{{m_room_create_event}}
 
+{{m_room_history_visibility_event}}
 
-Each `m.room.message`_ MUST have a ``msgtype`` key which identifies the type
-of message being sent. Each type has their own required and optional keys, as
-outlined below.
+{{m_room_join_rules_event}}
 
-{{msgtype_events}}
+{{m_room_member_event}}
 
-Presence Events
-~~~~~~~~~~~~~~~
+{{m_room_power_levels_event}}
 
-{{presence_events}}
- 
-Each user has the concept of presence information. This encodes the
-"availability" of that user, suitable for display on other user's clients.
-This is transmitted as an ``m.presence`` event and is one of the few events
-which are sent *outside the context of a room*. The basic piece of presence
-information is represented by the ``presence`` key, which is an enum of one
-of the following:
-
-      - ``online`` : The default state when the user is connected to an event
-        stream.
-      - ``unavailable`` : The user is not reachable at this time.
-      - ``offline`` : The user is not connected to an event stream.
-      - ``free_for_chat`` : The user is generally willing to receive messages
-        moreso than default.
-      - ``hidden`` : Behaves as offline, but allows the user to see the client
-        state anyway and generally interact with client features. (Not yet
-        implemented in synapse).
-
-In addition, the server maintains a timestamp of the last time it saw a
-pro-active event from the user; either sending a message to a room, or
-changing presence state from a lower to a higher level of availability
-(thus: changing state from ``unavailable`` to ``online`` counts as a
-proactive event, whereas in the other direction it will not). This timestamp
-is presented via a key called ``last_active_ago``, which gives the relative
-number of milliseconds since the message is generated/emitted that the user
-was last seen active.
-    
-
-Events on Change of Profile Information
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-Because the profile displayname and avatar information are likely to be used in
-many places of a client's display, changes to these fields cause an automatic
-propagation event to occur, informing likely-interested parties of the new
-values. This change is conveyed using two separate mechanisms:
-
- - a ``m.room.member`` event is sent to every room the user is a member of,
-   to update the ``displayname`` and ``avatar_url``.
- - a ``m.presence`` presence status update is sent, again containing the new values of the
-   ``displayname`` and ``avatar_url`` keys, in addition to the required
-   ``presence`` key containing the current presence state of the user.
-
-Both of these should be done automatically by the home server when a user
-successfully changes their displayname or avatar URL fields.
-
-Additionally, when home servers emit room membership events for their own
-users, they should include the displayname and avatar URL fields in these
-events so that clients already have these details to hand, and do not have to
-perform extra roundtrips to query it.
+{{m_room_redaction_event}}
 

specification/modules/instant_messaging.rst

@@ -0,0 +1,27 @@
+Instant Messaging
+=================
+
+Events
+------
+
+{{m_room_message_event}}
+
+{{m_room_message_feedback_event}}
+
+{{m_room_name_event}}
+
+{{m_room_topic_event}}
+
+m.room.message msgtypes
+-----------------------
+
+.. TODO-spec
+   How a client should handle unknown message types.
+
+
+Each `m.room.message`_ MUST have a ``msgtype`` key which identifies the type
+of message being sent. Each type has their own required and optional keys, as
+outlined below.
+
+{{msgtype_events}}
+

specification/modules/presence.rst

@@ -1,7 +1,5 @@
-Presence Events
-===============
-
-{{presence_events}}
+Presence
+========
  
 Each user has the concept of presence information. This encodes the
 "availability" of that user, suitable for display on other user's clients.
@@ -29,6 +27,11 @@ is presented via a key called ``last_active_ago``, which gives the relative
 number of milliseconds since the message is generated/emitted that the user
 was last seen active.
 
+Events
+------
+
+{{presence_events}}
+
 Presence HTTP API
 -----------------
 .. TODO-spec

specification/targets.yaml

@@ -14,6 +14,7 @@ targets:
       - 6-appendices.rst
 groups:  # reusable blobs of files when prefixed with 'group:'
   modules:
+    - modules/instant_messaging.rst
     - modules/voip_events.rst
     - modules/typing_notifications.rst
     - modules/receipts.rst

templating/batesian/sections.py

@@ -27,12 +27,38 @@ class Sections(object):
             section_key = func_name[len("render_"):]
             self.log("Generating section '%s'" % section_key)
             section = func()
-            if not isinstance(section, basestring):
+            if isinstance(section, basestring):
+                if section_key in section_dict:
                     raise Exception(
-                    "Section function '%s' didn't return a string!" % func_name
+                        ("%s : Section %s already exists. It must have been " +
+                        "generated dynamically. Check which render_ methods " +
+                        "return a dict.") %
+                        (func_name, section_key)
                     )
                 section_dict[section_key] = section
                 self.log(
                     "  Generated. Snippet => %s" % section[:60].replace("\n","")
                 )
+            elif isinstance(section, dict):
+                self.log("  Generated multiple sections:")
+                for (k, v) in section.iteritems():
+                    if not isinstance(k, basestring) or not isinstance(v, basestring):
+                        raise Exception(
+                            ("Method %s returned multiple sections as a dict but " +
+                            "expected the dict elements to be strings but they aren't.") %
+                            (func_name, )
+                        )
+                    if k in section_dict:
+                        raise Exception(
+                            "%s tried to produce section %s which already exists." %
+                            (func_name, k)
+                        )
+                    section_dict[k] = v
+                    self.log(
+                        "  %s  =>  %s" % (k, v[:60].replace("\n",""))
+                    )
+            else:
+                raise Exception(
+                    "Section function '%s' didn't return a string/dict!" % func_name
+                )
         return section_dict

templating/matrix_templates/sections.py

@@ -76,37 +76,36 @@ class MatrixSections(Sections):
             ))
         return "\n\n".join(sections)
 
-    def render_profile_http_api(self):
-        return self._render_http_api_group(
-            "profile", 
-            sortFnOrPathList=["displayname", "avatar_url"]
-        )
 
-    def render_sync_http_api(self):
-        return self._render_http_api_group(
-            "sync"
-        )
-
-    def render_presence_http_api(self):
-        return self._render_http_api_group(
-            "presence",
+    # Special function: Returning a dict will specify multiple sections where
+    # the key is the section name and the value is the value of the section
+    def render_group_http_apis(self):
+        # map all swagger_apis to the form $GROUP_http_api
+        swagger_groups = self.units.get("swagger_apis").keys()
+        renders = {}
+        for group in swagger_groups:
+            sortFnOrPathList = None
+            if group == "presence":
                 sortFnOrPathList = ["status"]
+            elif group == "profile":
+                sortFnOrPathList=["displayname", "avatar_url"]
+            renders[group + "_http_api"] = self._render_http_api_group(
+                group, sortFnOrPathList
             )
+        return renders
 
-    def render_membership_http_api(self):
-        return self._render_http_api_group(
-            "membership"
-        )
-
-    def render_login_http_api(self):
-        return self._render_http_api_group(
-            "login"
-        )
-
-    def render_rooms_http_api(self):
-        return self._render_http_api_group(
-            "rooms"
+    # Special function: Returning a dict will specify multiple sections where
+    # the key is the section name and the value is the value of the section
+    def render_group_events(self):
+        # map all event schemata to the form $EVENTTYPE_event with s/./_/g
+        # e.g. m_room_topic_event
+        schemas = self.units.get("event_schemas")
+        renders = {}
+        for event_type in schemas:
+            renders[event_type.replace(".", "_") + "_event"] = self._render_events(
+                lambda x: x == event_type, sorted
             )
+        return renders
 
     def render_room_events(self):
         def filterFn(eventType):
@@ -180,3 +179,4 @@ class MatrixSections(Sections):
 
     def render_common_state_event_fields(self):
         return self._render_ce_type("state_event")
+