formatter design

[gnecula]

I do not quite like how formatters work. Right now, they modify the observation dictionary in place. This makes it easy for the formatter, because you do not have to copy the observation, but has unpleasant consequences:

• the side-effects in the observation are visible to downstream functions, but not to filter and ignore functions, which are processed before
• the side-effects make it impossible to write one-line formatters using 'lambda' (but that is really a weakness in Python). Even without side-effects it would be hard to write one-liners.

What I would really like is to specify places in the observation where I want to make some changes. For example, I want to split the lines for key1[].foo, or I want to replace all strings that match a date in key2.dates[].

[gnecula]

Also, I am proposing that only agents specify formatters. The spy function does not really need them because you can just as well format the value before passing it to spy. The spy_point should not have it because we want to discourage writing bond-specific code in production code.

[xkrogen]

Agree that I don't really like the way they work right now. I'm sure we can find something better.

As for formatters in spy_point... I can imagine a situation where being able to specify a formatter right next to the function could be useful, e.g. when you know that there's one argument you'll never want to observe, or something along those lines. I agree that minimizing bond code in production is important but it's something to consider.

[gnecula]

For now I have moved the formatter than I had in bond_reconcile._print to the testing code.

[gnecula]

Here is a proposal for helping the writing of formatters, without writing a new JSON serializer. Essentially we specify a list of pairs (location_regexp, action) where the location_regexp says what keys in the observation should be acted on, with the keys being of the form "key1.key2[xxx]", and the actions being things like "split_lines", "search_replace(regexp, replacement)"

So we can write things like:

formatter=formatters.split_lines().at('.*\.what$'), 
...
formatter=formatters.search_replace('random=\d+', 'random=...').at('.*\.what\[\d+\]$')

This allows one to write simple formatters without defining auxiliary functions. Another advantage is that most of the time we use the standard JSON formatter and we do not need to re-traverse the object.

This notation should allow one to write new actions.

Using regexp to match keys is fast, but the notation is awkward because the key language contains "." and "[ ]" which have special meaning as regular expressions.

[xkrogen]

Yeah, I like that direction. We could probably allow regex for full flexibility, but also provide easier ways, e.g. split_lines().atKey('what') would achieve the same thing as your first line and search_replace(...).atKeyWithValue('what', '\d+') would achieve the same as the second line (or something like this - generally I just I feel that most of the time full regex would be unnecessary and a simpler interface would be nice)

[gnecula]

Another useful formatter would be a rounding function for floating point values.