ORMHate and Code Generation with Golang

ORM is a handy tool to assist engineers in developing data-related functions but some may have doubt. The favor to use ORM is always interesting to discuss among software developers.

Object Relational Mapping (ORM)

As its name says Object Relational Mapping (ORM) automatically maps relational tables (SQL) into programming objects (Entity). It saves time and effort, as well as produces cleaner code.

# without ORM
var sql = "SELECT id, first_name, last_name, phone, birth_date, sex, age FROM persons WHERE id = 10";
var result = context.Persons.FromSqlRaw(sql).ToList();
var name = result[0]["first_name"];

# with ORM
var person = repository.GetPerson(10);
var firstName = person.GetFirstName();

ORMHate

Automatically mapping the data is a tricky problem to solve.

We can’t assume one table will only convert to a single entity object, it is not uncommon for entities object to consist of multiple relational tables.

Another concern is that both sources keep changing, which means entities must be updated when tables change, and vice-versa. The ORM should be able to bridge these dynamics.

The criticism for ORM is a leaky abstraction, the condition where the abstraction failed to hide the complexity behind it. The developer must understand what the database tables look like and also how the mapping process before using the ORM. It can lead to an exhausting learning curve and reckless use of ORM may cause performance issues.

There are good reasons not to use ORM (ORMHate), but on the other hand, ORM also offers its virtues. Here are some suggestions for using the ORM:

• Use a solid, mature, and proven framework/library for ORM. If the library is “broken”, you are in big trouble.
• Use it on the new project. Introducing ORM to legacy projects is a bad idea.
• Use a simple and common data structure. Joining many tables with ORM is disastrous.

Code Generation

Besides ORM, we can avoid monotonous work handling relational data with code generation. SQLc is one tool to generate code from SQL, but in the end, code is about needs and tastes. You may think of generating code for the entity and repo object according to your favor.

Generating code in Go is dead simple. Go already provides text/template standard package for this cause (don’t mistake with html/package package).

tmpl, err := template.ParseFiles("my_code_template.go.tmpl")
if err != nil {
  log.Fatal(err)
}

err := tmpl.Execute(file, table)
if err != nil {
  log.Fatal(err)
}

Put your desired code template to *.go.tmpl files

// Code generated by `tools/entity-gen`. DO NOT EDIT.
package {{.PackageName}}

type (
 {{.StructName}} struct { 
  // ...
 }

As a final touch, call GoImport to format the document and fix the missing import.

Tool folder

You don’t need to create a separate project for code generation, put it into the same project under tools folder.

The standard Golang project layout would look like this:

• cmd : the main function of the project
• internal : internal project source code (similar with src in other programming language)
• external : external project source code (the codes you want to share outside of the project)
• tools : supporting tools for the project (your code generation tool)

Table Metadata

The code generation is based on tables and columns in the database which are fetchable via the query. Here is the query to fetch table metadata in Postgres:

SELECT table_name FROM information_schema.tables WHERE table_schema='public';
SELECT column_name, data_type, is_nullable FROM information_schema.columns WHERE table_name = '<TABLE_NAME>' ORDER BY ordinal_position;

Database Migration Tool

Most ORMs have features to create/alter the database schema automatically in the database. Since we don’t use ORM, we manage the database schema manually. Executing DDL Scripting directly to the database is a bad practice. The recommendation is to do it via database migration tools (GoMigrate).

GoMigrate ensures the database instances (across the environment) have the same schema. The database change is recorded in migration files.

1481574547_create_users_table.up.sql
1481574547_create_users_table.down.sql

Convention over configuration

Convention is the key to making maintainable code generation. When we encounter an unconventional design, we change the database to follow the conventions instead of improving the code generation (Convention over configuration).

Example of the convention for code generation:

1. Tables only have a single primary key called `id` with data type `SERIAL` or `BIGSERIAL`
2. Audit columns (i.e. `created_at`, `updated_at`, and `deleted_at`) are mandatory. Generated entities don’t include these columns
3. Hard delete is not allowed. The select operation doesn’t show soft-deleted rows.

Talk is cheap, show me the code

For reference, find the code here.

type (
  Table struct {
    PackageName string
    TableName   string
    PrimaryKey  *Column
    Columns     []*Column
    
    StructName string
  }
  Column struct {
    ColumnName string
    DataType   string
    IsNullable string
    
    FieldName     string
    FieldType     string
    DefaultValue  string
    IsPrimaryKey  bool
    IsAuditColumn bool
  }
)

func generate(db *sql.DB) error {
 tables, err := getTables(db)
 if err != nil {
  return err
 }

 os.MkdirAll(TargetDir, os.ModePerm)

 tmpl, err := template.ParseFiles(TemplatePath)
 if err != nil {
  return err
 }

 for _, table := range tables {
  targetPath := TargetDir + "/" + table.TableName + ".go"
  file, err := os.Create(targetPath)
  if err != nil {
   return err
  }
  defer file.Close()

  if err := tmpl.Execute(file, table); err != nil {
   return err
  }
 }

 cmdkit.GoImports(TargetDir)

 for _, table := range tables {
  dest := MockDir + "/mock_" + table.TableName + ".go"
  src := TargetDir + "/" + table.TableName + ".go"
  cmdkit.GoMock(dest, src)
 }

 return nil
}

What do you think about ORM? Should we use one or not? Have you try code generation to handle data?