mkdi...

In the article introducing this series, I shared that I wanted to explore full-stack app development. In particular, I wanted to develop a full-stack application that helps people manage their checklists. Users will use a browser to connect to the front-end server, which, in turn, will use the API provided by the back-end server to access and modify the checklists.

This article is the first one describing what I am doing in Go. I will cover project creation and structure, and build automation. We won’t have much of the application implemented by the end of this article, but it should be a good and helpful start. So buckle up and let’s get cracking!

Project structure and setup

First things first. Let’s initialize the module with the following commands on the shell.

mkdir rexlists
cd rexlists
go mod init rexlists

As I shared in the introduction to this series, I plan to produce two binaries. The first will handle all front-end tasks as an HTTP server, serving HTML and related content, while communicating with the back end. The second binary will manage the back-end tasks and responsibilities. The front-end binary will implement the role of a Back end For Front end (BFF), which will be especially useful as the application grows to support clients like web, mobile, and desktop, and connects to various back-end services such as checklist management, payments, and calendar.

We set up a cmd directory to hold the main packages for the front-end and back-end binaries. Each binary has its own subdirectory.

mkdir -p cmd/{rexlists-fe,rexlists-be}
touch cmd/rexlists-fe/main.go
touch cmd/rexlists-be/main.go

And we add a main.go to each of those directories. The first one is cmd/rexlists-fe/main.go:

package main

import "log"

func main() {
log.Println("Running front end")
}

And similarly, cmd/rexlists-be/main.go contains:

package main

import "log"

func main() {
log.Println("Running back end")
}

We could produce the binaries in their respective directories, but we will also create a target directory and add it to our .gitignore.

With this structure in place, we can produce both binaries and run each one of them.

go build -o target/rexlists-fe cmd/rexlists-fe/main.go
./target/rexlists-fe
go build -o target/rexlists-be cmd/rexlists-be/main.go
./target/rexlists-be

There will be more directories for organizing the code in packages1, but I will take care of them as the needs arise.

Build automation

Despite most Go developers’ love for go build, it is really useful to have some kind of automation that simplifies our repetitive build process. We are going to work on the user interface using iterative and incremental development, and it would be nice to have a tool that restarts the front-end server whenever we save changes to its source files. Air, which offers “live reload for Go apps,” is the tool that I have chosen for that purpose. And I am going to install it with go install.

go install github.com/air-verse/air@latest

However, Air is meant only to address very specific needs and only for the front end. We are also going to use Mage for any other automation in the project, because it provides all of the benefits of a Makefile using the Go syntax that we know and love.

We can install Mage in many ways, but I strongly suggest using go install here, since it will also install the packages we need to write the tasks in our magefile.go.

go install github.com/magefile/mage@latest

We create a new directory called magefiles at the root of our project, and we place our magefile.go in it, so our project stays nice and tidy. You can use another name for your magefile.go, but I believe that using that one makes things clearer to the readers of our repository.

mkdir magefiles
touch magefiles/magefile.go

The rules for your magefile

There are a few rules that you have to keep in mind to write your build automation:

• File(s) must start with //go:build mage in its own line. Other build tags are allowed if necessary.
• Task file(s) must belong to the main package. Other packages may be added.
• One exported function per task, i.e., starting with a capital letter.
• The first sentence of the comment above the exported function is the short description of the task.
• The full comment is the long description printed with -h and the task name.
• A comment above the package declaration is printed before the list of tasks.

Using these rules, we can write the seed of the build automation “script.” I am using sh.Run() here, that is a helper provided by Mage to run commands with arguments and return an error, if needed. This function will not display the output of the command unless we run mage with the -v flag.

//go:build mage

// Automated build tasks for development and release to production.
package main

import "github.com/magefile/mage/sh"

// Build front end application
func Build() error {
return sh.Run("go", "build", "-o", "target/rexlists-fe", "cmd/rexlists-fe/main.go")
}

Remember to get this module so it is available during compilation.

go get github.com/magefile/mage/sh

Although this is very simple, we can now build the front end without specifying all the parameters to the build subcommand. Running mage from the project root returns the list of tasks, and mage build produces the front-end executable.

% mage
Automated build tasks for development and release to production.

Targets:
build    front end application
% mage build
% rm -f target/rexlists-*
% mage -v build
Running target: Build
exec: go "build" "-o" "target/rexlists-fe" "cmd/rexlists-fe/main.go"
% ls target
rexlists-fe

Wow! With just three comments and five lines of code, we’ve made solid progress. Mage offers even more capabilities. I’ll walk through a few additional features and finish the first version of the build automation.

Mage best practices

Mage really helps to have a streamlined build process that applies the best practices. Let’s cover some of the more frequently used ones.

Use global constants and variables

Constants and variables are as useful as in any other Go code. We can define some constants for the most common strings. This will reduce the amount of problems caused by typos.

const appName = "rexlists"
const targetDir = "target"

Simplify your tasks

So far, Mage is able to build the binary for the front end. We should add a task for the back end, which, unsurprisingly, is going to be very similar.

If tasks share part of the code or get too complex, you can/should use non-exported functions to avoid repetition or simplify code. In our case, the tasks for building the front end and the back end share most of the code with some small differences in the file names.

// Build front end application
func BuildFrontEnd() error {
fmt.Println("Building front end.")
return buildVariant(appName, "fe")
}

// Build back end application
func BuildBackEnd() error {
fmt.Println("Building back end.")
return buildVariant(appName, "be")
}

func buildVariant(appName, variant string) error {
var bin = fmt.Sprintf("target/%s-%s", appName, variant)
var src = fmt.Sprintf("cmd/%s-%s/main.go", appName, variant)

return sh.Run("go", "build", "-o", bin, src)
}

Identify dependencies

Build tools simplify repetitive tasks by organizing them hierarchically. If you use mg.Deps with a list of the tasks that the current one depends on, they will be run in parallel. You can also use mg.SerialDeps, if you need to ensure execution order.

We can create a task that builds both parts by depending on them.

// Build both applications
func BuildAll() {
mg.Deps(BuildFrontEnd, BuildBackEnd)
}

Use different run methods

We may need to show the output always or only when the user requires it. There are helper functions for that.

sh.Run executes the command, but output is only shown with mage -v sh.RunV executes the command and shows the output (Useful for a linter, for example) sh.Output executes the command and returns the output (Useful to capture output of auxiliary commands, for example, git related)

If we run a linter task2, we want its output displayed.

// Run linters.
func CheckLint() error {
fmt.Println("Run Linters.")
return sh.RunV("golangci-lint", "run")
}

Use namespaces

Namespaces group related tasks. We just have to define them and use them as receivers of the task functions. The names of the tasks will include the namespace they belong to when you print the list.

We can use one namespace for building tasks and another for quality assurance tasks like testing and linting.

// Namespaces
type Build mg.Namespace
type Check mg.Namespace

// Build front end application
func (Build) FrontEnd() error {
//...
}

// Build back end application
func (Build) BackEnd() error {
//...
}

// Builg both applications
func (Build) All() {
//...
}

// Run linters.
func (Check) Lint() error {
//...
}

// Check code quality.
func (Check) All() {
mg.Deps(Check.Lint)
}

Define aliases

Aliases allow for keeping the amount of typing to a minimum.

We can define them for the most frequent tasks.

var Aliases = map[string]interface{}{
"b":  Build.All,
"bf": Build.FrontEnd,
"bb": Build.BackEnd,
"c":  Check.All,
}

Use global variable to define default action

Optionally, we can identify the action run by Mage by default, i.e., using mage with no other arguments.

var Default = Build.All

We can still print the list of available tasks using mage -l.

First version of the build automation

The magefile.go should be similar to the following one.

//go:build mage

// Automated build tasks for development and release to production.
package main

import (
"fmt"

"github.com/magefile/mage/mg"
"github.com/magefile/mage/sh"
)

const appName = "rexlists"
const targetDir = "target"

var Aliases = map[string]interface{}{
"b":  Build.All,
"bf": Build.FrontEnd,
"bb": Build.BackEnd,
"c":  Check.All,
}

// Default target to run when none is specified
// If not set, running mage will list available targets
var Default = Build.All

// Namespaces
type Build mg.Namespace
type Check mg.Namespace

// Build front end application
func (Build) FrontEnd() error {
fmt.Println("Building front end.")
return buildVariant(appName, "fe")

}

// Build back end application
func (Build) BackEnd() error {
fmt.Println("Building back end.")
return buildVariant(appName, "be")
}

func buildVariant(appName, variant string) error {
var bin = fmt.Sprintf("target/%s-%s", appName, variant)
var src = fmt.Sprintf("cmd/%s-%s/main.go", appName, variant)
return sh.Run("go", "build", "-o", bin, src)
}

// Build both applications
func (Build) All() {
mg.Deps(Build.FrontEnd, Build.BackEnd)
}

// Run linters.
func (Check) Lint() error {
fmt.Println("Run Linters.")
return sh.RunV("golangci-lint", "run")
}

// Check code quality.
func (Check) All() {
mg.Deps(Check.Lint)
}

Set up Air

Air will handle live reloading of the front end when we change anything, but we need to provide it with some information for that. We have to create a configuration file using air init and edit the resulting file .air.toml to capture our needs.

root = "."
testdata_dir = "testdata"
tmp_dir = "tmp"

[build]
args_bin = []
bin = "./target/rexlists-fe"
cmd = "go build -o ./target/rexlists-fe cmd/rexlists-fe/main.go"
delay = 1000
exclude_dir = ["assets", "tmp", "vendor", "testdata"]
exclude_file = []
exclude_regex = ["_test.go"]
exclude_unchanged = false
follow_symlink = false
full_bin = ""
include_dir = []
include_ext = ["go", "tpl", "tmpl", "html"]
include_file = []
kill_delay = "0s"
log = "build-errors.log"
poll = false
poll_interval = 0
post_cmd = []
pre_cmd = []
rerun = false
rerun_delay = 500
send_interrupt = false
stop_on_error = false

[color]
app = ""
build = "yellow"
main = "magenta"
runner = "green"
watcher = "cyan"

[log]
main_only = false
silent = false
time = false

[misc]
clean_on_exit = false

[proxy]
app_port = 0
enabled = false
proxy_port = 0

[screen]
clear_on_rebuild = false
keep_scroll = true

Our front end will have an HTTP server and an HTTP client that talks to the back end. The server sends the browser the HTML and assets needed to render and interact with the user interface. The client makes requests to the back end to access and modify the data.

Configuration

Following the Twelve-Factor App methodology, we want to store the configuration in the environment, for both the front end and back end. So, we first create a config directory for the helper functions in that package.

mkdir config

And we put the first function inside of a new file, config.go, in that directory.

// Package config provides configuration functions for the application.
package config

import "os"

func GetFromEnvOrDefault(key, fallback string) string {
if value, ok := os.LookupEnv(key); ok {
return value
}
return fallback
}

Now, let’s use some configuration from the HTTP server.

HTTP server

First, we add a new directory named app at the root of the project to hold the front-end application code. Inside app, we create an app.go file and define a struct for the application. This struct will store the configuration and any needed dependencies.

// Package app provides the front-end application code.
package app

type App struct {
serverAddr string
serverPort uint16
}

This new type will define methods for controlling the application and contain the necessary configuration. The most obvious one would be a method for launching the HTTP server, but we also want to have a constructor function to load or set the configuration.

func New() App {
var port int
var err error
if port, err = strconv.Atoi(config.GetFromEnvOrDefault("FRONTEND_PORT", "4000")); err != nil {
port = 4000
}
return App{
serverAddr: config.GetFromEnvOrDefault("FRONTEND_ADDR", "localhost"),
serverPort: uint16(port),
}
}

func (app *App) Start() error {
log.Printf("Launching RexLists front-end server: http://%s:%d\n", app.serverAddr, app.serverPort)
return http.ListenAndServe(fmt.Sprintf("%s:%d", app.serverAddr, app.serverPort), nil)
}

I have chosen to use a pointer receiver –*App– instead of a value one, because the struct is going to grow and I don’t want to refactor it later.

The method can now be used in the main function (in rexlists-fe/main.go), after creating an instance of the type with the constructor.

import (
"log"

"rexlists/app"
)

func main() {
log.Println("Running front end")
app := app.New()
log.Fatal(app.Start())
}

This should be enough to use the configuration data and run with those values or the defaults. We can try that using the following commands.

mage bf
target/rexlists-fe

The front-end server should be up and running and you can connect to it from the browser using the default URL http://localhost:4000/. However, your welcome message is going to be more disappointing than warm: “404 page not found”. But fear not! That is expected because there is no content being served yet.

You can use a different port, though. Just run the command preceded by the corresponding environment variable and the application will use the designated port, for example 5646.

FRONTEND_PORT=5646 target/rexlists-fe

Summary

In this first article, we have created the structure for our project and automated the building tasks. We have gone beyond the build capabilities offered by Go. Not yet to infinity, but quite further.

As for the front-end HTTP server, we’ve made some initial progress. Mostly around reading the configuration and launching it with those parameters. This is still completely useless as an application, but I hope you agree that we have provided a solid foundation for it.

The full code repository for this project with individual commits for each part of the explanation is available for you to check what I did and where, and code along with me.

In the next article we will work on the front end to make it more useful.

Stay curious. Hack your code. See you next time!

Footnotes

---

2

This assumes that you have golangci-lint installed in your system.