Ecto

Ecto is an Elixir-based domain specific language (DSL) for database access. At present, Ecto targets only relational databases. However, it seems plausible that it could be adapted and extended to handle graph databases (e.g., Neo4j) and/or related tools (e.g., Gremlin).

Motivation

Many knowledge bases (e.g., DBpedia) are structured as graphs. Although Neo4j is good at storing and traversing these structures, it provides little support for graph algorithms (e.g., spreading activation). Elixir could be an interesting tool for exploring these algorithms (e.g., by modeling graph nodes and edges as actors), but it needs convenient and efficient access to the data.

Approach

Ecto provides a supported and well-considered API for relational databases. If it could be adapted and/or extended to handle graph databases, I could rely on it to handle repository access, queries, etc. I plan to give this a try, using the Phoenix Framework's Simple Chat Example as a check of basic functionality.

Details

Status: These pages and a few mailing list submissions constitute most of the visible progress on the project. FWIW, I'm currently trying to determine requirements and find gotchas.

Ecto Overview

Ecto is an Elixir-based domain-specific language for communicating with Relational Database Management Systems. Drivers are currently available for MSSQL, MySQL, and PostgreSQL.

Ecto is split into three main components:

  • Ecto.Repo - repositories are wrappers around the database. Via the repository, we can create, update, destroy, and query existing entries. A repository needs an adapter and a URL to communicate to the database.

  • Ecto.Model - models provide a set of functionalities for defining data structures, how changes are performed in the storage, life-cycle callbacks, and more.

  • Ecto.Query - written in Elixir syntax, queries are used to retrieve information from a given repository. Queries in Ecto are secure, avoiding common problems like SQL Injection, and also provide type safety. Queries are composable via the Ecto.Queryable protocol.

-- Ecto documentation

Ecto has several other components, including:

  • Ecto.Adaptor - specifies the API that an adapter is required to implement

  • Ecto.Changeset - allow filtering, casting, and validation of model changes

  • Ecto.Storage - convenience functions around the data store of a repository

  • Ecto.Type - functions and behaviour for implementing custom types

Ecto Components

Let's take a (slightly) deeper dive into some of Ecto's components:

Ecto.Adaptor

Cyphex will certainly need its own set of adaptor functions. So, let's dig a little deeper, showing (partial) calling trees.

all/4

Fetches all results from the data store based on the given query.

__MODULE__.Adaptors.SQL.all/4
| __MODULE__.Connection.all/1
| Ecto.Adaptors.SQL.all/5
| | Ecto.Adaptors.SQL.query/4
| | Ecto.Adaptors.SQL.extract_fields/2

delete/4

Deletes a single model with the given filters.

__MODULE__.Adaptors.SQL.all/4
| __MODULE__.Connection.delete/3
| Ecto.Adaptors.SQL.model/5
| | Ecto.Adaptors.SQL.query/4

delete_all/4

Deletes all entities matching the given query.

__MODULE__.Adaptors.SQL.delete_all/4
| __MODULE__.Connection.delete_all/1
| Ecto.Adaptors.SQL.count_all/4
| | Ecto.Adaptors.SQL.query/4

insert/5

Inserts a single new model in the data store.

__MODULE__.Adaptors.SQL.insert/5
| __MODULE__.Connection.insert/3
| Ecto.Adaptors.SQL.model/5
| | Ecto.Adaptors.SQL.query/4

Starts any connection pooling or supervision and returns {:ok, pid} or just :ok if nothing needs to be done.

__MODULE__.Adaptors.SQL.start_link/2
| Ecto.Adaptors.SQL.start_link/4
| | Ecto.Adaptors.SQL.split_opts/2
| | :poolboy.start_link/2

stop/1

Stops any connection pooling or supervision started with start_link/1.

__MODULE__.Adaptors.SQL.stop/1
| Ecto.Adaptors.SQL.stop/1

update/6

Updates a single model with the given filters.

__MODULE__.Adaptors.SQL.update/6
| __MODULE__.Connection.update/4
| Ecto.Adaptors.SQL.model/5

update_all/5

Updates all entities matching the given query with the values given.

__MODULE__.Adaptors.SQL.update_all/5
| __MODULE__.Connection.update_all/2
| Ecto.Adaptors.SQL.count_all/4
| | Ecto.Adaptors.SQL.query/4

Ecto.Model

  • assoc(model_or_models, assoc)
    Builds a query for the association in the given model or models.

  • build(struct, assoc, attributes \\ %{})
    Builds a struct from the given assoc in model.

  • primary_key(struct) - Returns primary keys as a keyword list.

  • primary_key!(struct)
    Similar to get/3, but raises an exception if the model has no primary key.

Ecto.Query

  • distinct(query, binding, expr)
    A "distinct" query expression. When true, only keeps distinct values.

  • exclude(query, field)
    Resets a previously set field on a query.

  • from(expr, kw \\ [])
    Creates a query.

  • group_by(query, binding, expr)
    A "group by" query expression. Groups together rows
    that have the same values in the given fields.

  • having(query, binding, expr)
    A "having" query expression. Filters rows from the model.

  • join(query, qual, binding, expr, on \\ nil)
    A "join" query expression. Receives a model that is to be
    joined to the query and a condition to do the joining on.

  • limit(query, binding, expr)
    A "limit" query expression. Limits the number of rows selected.

  • lock(query, expr)
    A "lock" query expression. Provides support for row-level locking.

  • offset(query, binding, expr)
    An "offset" query expression. Offsets the number of rows selected.

  • preload(query, bindings \\ [], expr)
    Pre-loads the associations into the given model.

  • select(query, binding, expr)
    A "select" query expression. Specifies which fields will be selected.

  • where(query, binding, expr)
    A "where" query expression. Used to filter the result set.

Ecto.Repo

  • adapter/0
    Returns the adapter tied to the repository.

  • all/2
    Fetches all entries matching the given query.

  • config/0
    Returns the adapter configuration stored in :otp_app.

  • delete/2
    Deletes a model using its primary key.

  • delete_all/2
    Deletes all entries matching the given query.

  • get/3
    Fetches a single model from the data store
    where the primary key matches the given id.

  • get!/3
    Similar to get/3, but raises an exception if no record found.

  • insert/2
    Inserts a model or a changeset.

  • log/2
    Enables logging of adapter actions such as sending queries.

  • one/2
    Fetches a single result from the query.

  • one!/2
    Similar to one/3, but raises an exception if no record found.

  • preload/2
    Preloads all associations on the given model or models.

  • rollback/1
    Rolls back the current transaction.

  • start_link/0
    Starts any connection pooling or supervision and returns
    {:ok, pid} or just :ok if nothing needs to be done.

  • stop/0
    Stops any connection pooling or supervision started with start_link/1.

  • transaction/2
    Runs the given function inside a transaction.

  • update/2
    Updates a model or changeset using its primary key.

Neo4j Overview

Neo4j is a high-performance graph database, based on fixed-length, memory-resident data structures. Cypher, Neo4j's declarative query language, combines graph patterns (expressed as ASCII art) with SQL-like verbs.

Neo4j's Transactional Cypher HTTP endpoint provides a network API for Cypher, based on HTTP and JSON. It is basically a REST API, save that it supports multi-request transactions via retained state.

Note: Although the main thrust of the project is concerned with Cypher, other Neo4j interfaces (e.g., unmanaged extensions) may be employed, as well as related technologies such as Gremlin.


This wiki page is maintained by Rich Morin, an independent consultant specializing in software design, development, and documentation. Please feel free to email comments, inquiries, suggestions, etc!

Topic revision: r3 - 04 Apr 2016, RichMorin
This site is powered by Foswiki Copyright © by the contributing authors. All material on this wiki is the property of the contributing authors.
Foswiki version v2.1.6, Release Foswiki-2.1.6, Plugin API version 2.4
Ideas, requests, problems regarding CFCL Wiki? Send us email