Motivation and Status

Nathan Marz recently released Specter, a Clojure library for manipulating nested data structures:

Most of Clojure programming involves creating, manipulating, and transforming immutable values. However, as soon as your values become more complicated than a simple map or list - like a list of maps of maps - transforming these data structures becomes extremely cumbersome.

Specter is a library for querying and updating nested data structures. One way to think of it is get-in and assoc-in on steroids, though Specter works on any data structure, not just maps. It is similar to the concept of a "lens" in functional programming, though it has some important extensions.

-- Specter (GH)

I would love to have similar tooling in Elixir, so I posted a note to elixir-lang-talk. Although José Valim was enthusiastic, he was concerned that porting efforts could run afoul of upcoming changes:

This is really nice Rich, thanks for sharing!

The truth is that you can already do all this in Elixir. From get_in/3 docs:

iex> users = [%{name: "john", age: 27}, %{name: "meg", age: 23}]
iex> all = fn :get, data, next -> Enum.map(data, next) end
iex> get_in(users, [all, :age])
[27, 23]

The issue is that nobody knows about it because we don't have built-in "selectors" yet, so doing a simple traversal like above gets complicated quickly. Selectors are actually planned for Elixir 1.3. With the Access protocol being deprecated on 1.1 and 1.2, the goal is to use the Access module for having a variety of selectors, and push more performance improvements too. So the example above could be rewritten as:

get_in(users, [Access.all, :age])

And we could allow other selectors too:

get_in(users, [Access.only(&is_map/1), :age])

In any case, it is already possible today, it is just inconvenient. :)

Nonetheless, it seems to me that an Elixir version of Specter could be useful and provide an interesting testbed for API experimentation while we're waiting for selectors to appear. Indeed, we might well find out some things that could make selectors better when they do appear.



AFAICT, Specter's key benefits include the following:

  • compact: ~500 LOC (mostly functions)

  • convenient: handles values and indices

  • extensible: based on a single protocol

  • flexible: uses higher order functions

  • homomorphic: preserves data structures

  • performant: rivals hand-optimized code


I'm still trying to understand Specter's implementation. At Nathan's suggestion, I'm documenting both an early prototype and the current (optimized) version of Specter:

To be continued...

This wiki page is maintained by Rich Morin, an independent consultant specializing in software design, development, and documentation. Please feel free to email comments, inquiries, suggestions, etc!

Topic revision: r62 - 12 Jun 2015, RichMorin
This site is powered by Foswiki Copyright © by the contributing authors. All material on this wiki is the property of the contributing authors.
Foswiki version v2.1.6, Release Foswiki-2.1.6, Plugin API version 2.4
Ideas, requests, problems regarding CFCL Wiki? Send us email