A Rack application from scratch, Part 4: Models and Persistence

9 Oct 2016

This is the fourth and last part of my guide on how to build a simple Rails clone from scratch, using Rack. In this post we will focus on the last missing component of our framework: models and pesistence. You can check part one for an intro to Rack, part two for the implementation of request routing and controllers, and part three for dynamic template rendering.

The usual disclaimer applies here: because this is a naive Rails clone, the code doesn’t aim to be performant or secure, but rather explicit and easy to understand. Think twice before using it directly in production.

Where we left off and what to do next

In the previous part we added view templates that are rendering values extracted from @instance variables from the controllers. Even though at that point we didn’t yet have model or record objects to pass around, we used OpenStruct instances to build objects that could then be rendered in the templates.

We basically used the same outside in approach already described in part 2. Each time we’ve worked on an element of the application, we’ve stubbed the desired interface of its collaborators. Implementing the template rendering logic of our MVC framework required a way to pass structured data from the Controller to the View, and we did so by defining what the API of our future Model should look like. Now that the rest of the moving parts are there and working fine, all we have to do is provide a real Model implementation for that OpenStruct-backed interface.

Models, persistence and interfaces

Before diving into the code, let’s stop for a moment and think about what we need.

We already have pages to display a single dog object and a list of dogs. The interface of a Dog is already defined, and we don’t want to modify those templates. The template for the HTML form to create a new dog is still a placeholder and we will implement it later.
We know that dog objects should respond to an id method returning an integer and a name method returning a string, and we know that we need a way to collect them in an Enumerable. We also need a way to create new dog objects populated with user input (just the name, to keep things simple) and we need to persist them somewhere and load them later.

Rails deals with these requirements by using a relational database (e.g. PostgreSQL, MySQL or SQLite) and the Active Record pattern, so that model classes provide a repository-like interface to load and search for records, and each model instance privides an API to persist (create or update) and delete its data.

Let’s try to implement the same interface, but without using a relational database.
Using a full-blown DB like PostgreSQL or MySQL would be too complex for our simple exercise, and while SQLite would be a nice fit, I’d still prefer something simpler.
With any of the mentioned SQL stores, we would have to hand over the persistence logic to the adapter library; even though we would write the SQL statements to support our modeling, I would rather use a simpler (and more limited) solution without “black box” parts.
Also, I would like to take the opportunity to show that it’s possible to implement an Active Record interface and simple CRUD features without an underlying SQL storage. This is important because it shows how clear abstractions (the active record interface) allow us to later replace the implementation.

With this in mind, and to use something available in the Standard Library, we’re going to build the persistence functionality on the PStore class. PStore allows us to store arbitrary Ruby objects in a file and retrieve them by key, as long as they can be marshaled.

The Model base class

Even though in software design the concept of “model” is not associated to “persistence”, in a vanilla Rails application the models directory is reserved for ActiveRecord::Base subclasses.
Since so far we have mimicked the directory structure and the naming conventions of a Rails application, we will do the same for the models.

We are thus adding an app/models/ directory to our application:

rack_demo/
├─ app/
│  ├─ controllers/
│  │  ├─ base_controller.rb
│  │  └─ dogs_controller.rb
│  ├─ models/
│  │  ├─ base.rb
│  │  └─ dog.rb
│  ├─ views/
│  │  ├─ base/
│  │  │  ├─ _head.html.erb
│  │  │  └─ index.html.erb
│  │  └─ dogs/
│  │     ├─ index.html.erb
│  │     ├─ new.html.erb
│  │     └─ show.html.rb
│  └─ router.rb
├─ application.rb
└─ rack_demo.ru

There, base.rb will be an abstract parent class that contains the common logic that all our models should have, and dog.rb will be a concrete implementation we will used in the DogsController.

Let’s have a look at the implementation for base.rb:

require 'pstore'

class Base
  DB_FILE = File.expand_path("../../../db.pstore", __FILE__)

  module ClassMethods

    # Find a record by ID
    #
    def find(id)
      db.transaction(true) do
        db[derive_db_id(self.name, id)]
      end
    end

    # Returns all records for this model
    #
    def all
      db.transaction(true) do
        ids = extract_model_ids(db)
        ids.map { |key| db[key] }
      end
    end

    # Store an instance in the DB
    #
    def save(object)
      db_id = derive_db_id(object.class.name, object.id)
      db.transaction do
        db[db_id] = object
      end
    end

    # Scoped by class, to auto-increment the IDs
    #
    def next_available_id
      last_id = all_ids.map do |key|
        key.sub("#{self.name}_", "").to_i
      end.max.to_i

      last_id + 1
    end

    private

    # Access to the PStore binary file
    #
    def db
      @db ||= PStore.new(DB_FILE)
    end

    # Scoped by class, so that different model classes
    # can have the same numerical IDs
    #
    def derive_db_id(model_name, obj_id)
      "#{model_name}_#{obj_id}"
    end

    # All the IDs for this model
    #
    def all_ids
      db.transaction(true) do |db|
        extract_model_ids(db)
      end
    end

    # Get all the PStore root keys (the DB IDs)
    # scoped for the current class
    #
    def extract_model_ids(store)
      store.roots.select do |key|
        key.start_with?(self.name)
      end
    end
  end
  extend ClassMethods


  def save
    ensure_presence_of_id
    self.class.save(self)
  end

  private

  def ensure_presence_of_id
    self.id ||= self.class.next_available_id
  end
end

There is a lot going on there, so let’s look at it bit by bit, starting with the class methods.

Of course, we begin with requiring the pstore library, which is used in Base::db to instantiate the PStore object. The db.pstore file, which will be created in the root directory of the application, is a binary file: its contents are the marshaled representations of the stored objects.

As you might have read in the documentation for PStore (linked above), all read and write operations must happen inside transactions, and thus all our data access will be wrapped in calls to the PStore#transaction method.

The public interface of Base is simple and only consists of four methods. We have Base::find to retireve an object by ID, and Base::all that returns all stored objects. Both methods are scoped by the class name, so that if we have multiple concrete subclasses of Base, let’s say Dog and Rabbit, we can invoke Dog.all() and obtain only dog objects. Simirarly, the IDs of the objects are only unique per class, and we can persist both a dog and a rabbit with the same ID, e.g. 42, and calling Rabbit.find(42) will correctly return the matching rabbit object, not the dog.
The IDs are namespaced with the private class method Base::derive_db_id.

Next, we have Base::save, that accepts an instance and stores it in the DB, and Base::next_available_id, that we use to auto-increment the IDs of new objects as we persist them.

Among the private class methods, Base::extract_model_ids is the counterpart of the already mentioned Base::derive_db_id. It iterates through the root-level keys in the store and selects the ones that match the current class name (again, to scope the IDs by class).

The only public instance method is Base#save, that assigns an auto-incremented value for the ID and then passes the object to the Base::save class method.

That’s it. With less than 90 lines of code, comments included, we have enough functionality to support simple CRUD operations on our models. Or, well, not entirely. We have not implemented a way to delete resources, but that’s something we deliberately left out when we defined the supported routes for the application. Implementing a method to delete records is left as an exercise. :-)

A Model concrete class

Will all that base logic available, adding models to the application is just a matter of subclassing Base. The class identity and instance variables of the objects will be serialized and stored.

Here is our Dog class.

require_relative './base.rb'

class Dog < Base
  attr_accessor :id, :name

  def initialize(id: nil, name: nil)
    @id = id
    @name = name
  end
end

Quite simple, isn’t it? We can check that everyting is working correctly in a REPL like pry or irb:

$ pry

> require "./app/models/dog"
=> true

> Dog.all
=> []

> dog = Dog.new(name: "Droolmachine")
=> #<Dog:0x007fa6d1428c98 @id=nil, @name="Droolmachine">

> dog.save
=> #<Dog:0x007fa6d1428c98 @id=1, @name="Droolmachine">

> other_dog = Dog.new(name: "Muncher")
=> #<Dog:0x007fa6d1332410 @id=nil, @name="Muncher">

> other_dog.save
=> #<Dog:0x007fa6d1332410 @id=2, @name="Muncher">

> Dog.all
=> [#<Dog:0x007fa6d2b5f880 @id=1, @name="Droolmachine">,
 #<Dog:0x007fa6d2b5f3f8 @id=2, @name="Muncher">]

> "I have #{Dog.all.count} dogs: #{Dog.all.map(&:name).join(' and ')}."
=> "I have 2 dogs: Droolmachine and Muncher."

It looks like everything is working fine. You’ll notice that a db.pstore file has been created in the root directory of the application (delete it to reset the data), and if you exit and restart the Pry session, your dog objects will still be available. Neat.

Using the models in the application

Creating and inspecting records in the REPL is not that useful, so the next step is to update our controller code to replace the OpenStruct objects with proper Dog objects.

It’s now a good idea to have a quick look at how we left the controller when we added the stubs for the view templates, and to compare it with the new implementation, below:

require_relative './base_controller.rb'

class DogsController < BaseController
  # GET /dogs
  #
  def index
    @title = "So many dogs"
    @dogs = Dog.all
    build_response render_template
  end

  # GET /dogs/:id
  #
  def show
    @dog = Dog.find(params[:id])
    @title = "#{@dog.name}'s page"
    build_response render_template
  end

  # GET /dogs/new
  #
  def new
    @title = "More dogs please"
    build_response render_template
  end

  # POST /dogs
  #
  def create
    dog = Dog.new(name: params['dog']['name'])
    dog.save
    redirect_to "dogs/#{dog.id}"
  end
end

There are a few small changes.

In index and show we are no longer building the dog objects on the fly, and we are instead loading them from the PStore DB. As long the values of the instance variables have the same interface, the templates will not require any change. In show we are also using the retrieved object’s name to set the page title.
There is nothing new in the new action (no pun intended), but the create action becomes interesting. There we are instantiating and persisting a new dog, using a name value coming from the request parameters. Once the dog has been persisted we redirect to that dog’s show page. More on that in a second.

Let’s visit http://localhost:9292/dogs and, if everything is working correctly, we should see a list with the two dogs we created in the REPL. Click them, and you’ll see their details page. Try to go to http://localhost:9292/dogs/new, however, and you’ll be presented with an empty page. Let’s fix that next.

Creating new dogs

The create action in the controller is already fine, and all that’s missing are the inputs in the form. We don’t have access to the fancy form helpers provided by Rails, but for such a simple case we can get our hands dirty and manually write the required HTML.

Our new views/dogs/new.html.erb file:

<html>
  <%= render_partial "base/_head.html.erb" %>
  <body>
    <div>
      <h1><%= @title %></h1>
      <form action="/dogs" method="POST">
        <div>
          <label for="dog_name">The dog's name</label>
          <input type="text" name="dog[name]" id="dog_name">
        </div>
        <input type="submit" name="commit" value="create">
      </form>
    </div>
  </body>
</html>

Now reload the page and add some more dogs to the pack. :-)

With this last piece of functionality our simple Rack application is finally complete.

Other persistence options

Earlier in this post I briefly mentioned the possibility of changing the underlying storage without modifying the interface of the Model layer. It would be a good exercise, so let’s do it now by replacing PStore with something similar.

We are lucky, as the Standard Library provides the excellent YAML::Store class, a drop-in replacement for PStore that will save the objects as YAML instead of marshaled binary data.

Since the interface of YAML::Store is the same of PStore, the required changes are minimal:

require 'yaml/store'

class Base
  DB_FILE = File.expand_path("../../../db.yml", __FILE__)

  module ClassMethods
    def db
      @db ||= YAML::Store.new(DB_FILE)
    end
    # ...
  end
  # ..
end

We’ve just replaced:

require 'pstore' with require 'yaml/store',
"../../../db.pstore" with "../../../db.yml" and
PStore.new(DB_FILE) with YAML::Store.new(DB_FILE)

Restart the Rack server, reload the dog index page, and you’ll see that all the dogs have disappeared because we are now looking for a different DB file. The form will still work and, if you recreate the dogs we had previously added, you’ll see that everhing is still working as before.

Something is a bit better, though. YAML files are just plain text, and we can now check the contents of the db.yml file in the root directory of the project:

---
Dog_1: !ruby/object:Dog
  id: 1
  name: Droolmachine
Dog_2: !ruby/object:Dog
  id: 2
  name: Muncher

Wrap up

We started this series of posts with an introduction to Rack and we concluded with a working Rack application built from scratch. Even more than that, we adopted a set of patterns that would make it easy to expand the application with new functionality, and this will hopefully help understand how frameworks like Ruby on Rails work.

For reference, the full sourcecode for the application is available on github.

That’s all Folks!

tags: code ruby rack rails