What I’m Up To

I have a vision: The way we write computer software needs to change. We need to build-in continuous validation and verification. It’s not enough to write unit tests or let something “bake” in a production environment; it needs to be built from the start to be easy to verify as correct.

In my free time, I work toward this vision in project blue. Blue encompasses the following projects (in topologically-sorted order):

  • biometrics encompasses counters, gauges, and moments. These are all algebraic types. Concretely, that means that taking a measure at any two points allows one to summarize the value between the two points. Counters and moments preserve this property perfectly.
  • sig_fig_histogram provides a histogram type with bounded inaccuracy. Specify one signficant figure (e.g. 9e1) for at most 10% inaccuracy. Two significant figures (e.g, 99.) for at most 1% inaccuracy. Three significant figures (e.g. 99.9) gives 0.1% inaccuracy. sig_fig_histogram is a form of exponential histogram, except it also matches the exponential that humans expect: base 10.
  • buffertk provides utilities for packing and unpacking binary data including varints.
  • prototk is a protocol buffers library for rust that works via derive macros.
  • guacamole provides tools for controlled random data. It provides a set of combinators that compose to allow fast generation of random data.
  • armnod provides tools for building random strings.
  • setsum is a checksum that operates on sets of byte strings. Two sets that contain the same elements will have the same setsum even if they are enumerated in different orders. It supports precise addition and removal from the set.
  • scrunch provides a library for performing full-text search over compressed text documents.
  • indicio provides STRUCT-ured logging.
  • analogize marries scrunch with indicio for doing full-text search across structured logs. Core to indicio is a primitve for finding exemplars within a document.
  • tatl Hey, Listen! That’s the sound of alerting. I’m experimenting with some new APIs for alerting in a production environment.
  • mani is a text-editor friendly write-ahead log.
  • sst is a Sorted String Table library that incorporates setsum.
  • lsmtk is an LSM tree that provides end-to-end checksumming via setsum.

As is probably apparent, I’m writing in rust at the moment.