mani

This post introduces mani, a text-based write-ahead log for maintaining a manifest of strings. Mani’s text-based format makes it easy to support reading in other languages, and writing in any language that supports crc32c.

Concretely, a manifest edit looks like this:

dcab9d28+thing one
a4e79c62+thing two
05a03b0d1thing one metadata
bc9dae362thing two metadata
--------

This edit adds the strings "thing one" and "thing two" to the manifest and overwrites the metadata for slots 1 and 2 to be “thing one metadata” and “thing two metadata” respectively. This shows the two fundamental data structures of mani. Concretely, mani maintains two data structures: A manifest consisting of a set of strings and a dictionary of information consisting of symbols that map to strings.

The core of the mani manifest API is the following functions:

impl Manifest {
    pub fn strs(&self) -> impl Iterator<Item = &str> { ... }
    pub fn info(&self, c: char) -> Option<&str> { ... }
    pub fn apply(&self, edit: Edit) -> Result<(), Error> { ... }
}

The last function allows application of an edit. An edit patches the manifest and overrides info strings when it is applied. Edit looks like this:

impl Edit {
    pub fn add(&mut self, s: &str);
    pub fn rm(&mut self, s: &str);
    pub fn info(&mut self, c: char, s: &str);
}

An edit will remove from the manifest any string that is passed to rm—-it won’t appear in the list of strs anymore. Similarly, any string passed to add will appear in subsequent calls to strs. Lastly, the most recent s passed for a given c is available from the info call.

mani is in use within the lsmtk crate.