Lets consider a related bit of information, what version of a dependency (e.g. subvertpy for bzr-svn) is present when the test run is started.

Further, consider a distributed testing case, where we have two machines A and B both running tests (partitioned from the whole suite). In this scenario we have three streams: A – tests from A, B – tests from B, and C- the multiplexed stream once testrepository has combined the streams. Ideally we could recover this dependency information for any test, regardless of source stream A or B. So, ‘stream’ is the wrong granularity.

In V2 we have the route_code (or tags could be used for V1 compat) which can uniquely group all the tests for the same context within a stream. So we could associate some structured data with that.

A slightly weaker thing though, that doesn’t need any test support, is to just use an attachment.

Given a subunit v2 serialiser, something like:

serializer.status(file_name=”version-info”, file_bytes=bzrlib.version_info….().encode(‘utf8′), mime_type=”text/json; charset=utf8″)

emitted from the test runner will capture that data, and you can recover it at any later point by starting with the test, and then looking for the version-info that matches the route code within the combined stream.

The extensability thing is a mixed blessing. As far as malicious input, there are lots of ways to get things wrong – see the msgpack issue above, which is arguably intrinsic to the msgpack definition.

I don’t see anything here that would conflict with using protobufs (or Thrift, or probably Datomic.)

If you want to write your own byte-packing code yourself (maybe to run on a teeny microcomputer), producing protobufs will not be much harder than the format you allude to here.

On the other hand protobufs seem easier in some ways:

- the bottom layer of encoding is already done for people who want to read/write it from a new language
- importantly, probably more safely against malicious input than a by-hand parser
- you get a formal grammar essentially for free
- there is a clear systematic way to add new fields in future, without breaking old peers, which seems likely to be useful

backend and runner and runner and supervisor are ambiguous because testrepository is a runner runner . The supervisor is a process that has a UI on one side and a number of subunit emitting processes on the other. Those are backends from the supervisor perspective. When the supervisor’s UI is subunit, then the supervisor can act as a backend for another supervisor…

disttrial is indeed interesting, but AFAIK it doesn’t concern itself with muxing, by having a single depth hub and spoke network. Got a link to an english description of its workings?

- I don’t understand the protocol you are describing. If you could present a packet diagram or a grammar or something that would help a lot. Maybe some example packets

- Are you developing your own binary protocol anyway? Why? The “Dependencies” section makes it clear that it wasn’t an option for v1, but there seems to be no reason now.

- I’m struggling to follow the “Selecting” case, I think it’s because ‘supervisor’, ‘backend’, and ‘runner’ are terms that now carry weight but aren’t defined.

- `disttrial` has some prior art here. It’s built using AMP, a bidirectional asynchronous binary protocol, which might break a couple of requirements here, but it’s worth mentioning.