This blog consists of static content generated by Hugo from source code in a GitHub repository and served by Caddy. Hugo and Caddy are wonderful pieces of Go-based software, but together, they cannot automatically update this blog when I make a change.

Manually updating the blog is rather cumbersome. After pushing an update to GitHub, I log into this server, pull down the updates, run Hugo to generate the static content, and verify the new content in my web browser. I’d much rather automate this process so all I must do is push changes and refresh my browser. Since I just wrote a book where Chapter 10 covers writing Caddy extensions, I have no excuse not to automate this process.

What I need is a way for GitHub to inform my web server about new commits, so my server can retrieve the changes and rebuild the website. I know that GitHub allows you to define a webhook whereby GitHub posts a payload to a given endpoint when a repo event occurs. Although validation isn’t required, I can share a secret with GitHub that it will use to sign each request. I can then use the shared secret to confirm the validity of the request by validating the request’s signature. If the request is legit, I want the server to update the website. If not, Caddy should respond with an error.

Since I’m dealing with an HTTP request, this functionality should reside in a handler. Before I write one, I’ll see what’s available. A quick search leads me to caddy-exec. I can use this module to execute the command necessary to rebuild the website. The only downside is caddy-exec does not handle request validation. Security is a requirement, so it looks like I need middleware that can handle the validation before passing the request onto the caddy-exec handler.

My search continues, but I fail to find anything that fit my use case. One module looks promising, but it wants to play the role of both the validator and the executor. I prefer modules that do one thing well rather than try to do too many things all at once. I’ll use this opportunity to get a bit of practice writing Caddy modules.

First, I need a name. Since this is a Caddy module that validates GitHub webhook requests, I settle on caddy-validate-github. Easy enough. After creating the repo and setting up my dev environment, let’s get to coding.

Creating the Middleware

The only requirement is the shared secret. I could define a new string type and create various methods on it, but I’ll use a struct for my middleware since it gives me more flexibility. In addition to the shared secret, I also want to embed a logger for debugging purposes, and a field to store the shared secret as a byte slice (I’ll explain why further in this post).

type Middleware struct {
    Secret string `json:"secret,omitempty"`

    logger *zap.SugaredLogger
    secret []byte
}

To keep it simple, I call the struct Middleware and store the shared secret in a field named Secret. Caddy stores its configuration, and the configuration for all modules in JSON format. Therefore, I include a struct tag on the Secret field to explicitly define its key in the configuration file. I’ll show you how to add Caddyfile support a bit later in this post.

The logger field stores a pointer to a sugared zap logger and the secret field stores the byte representation of the shared secret.

Interfaces

I want this middleware to implement a few Caddy interfaces. I’m in the habit of defining interface guards up front and leveraging my editor’s ability to generate the methods for me. Although I enjoy typing, a developer’s hard work added this functionality to my editor, and I’d feel like I’m letting him/her down if I didn’t use it.

var (
    _ caddy.Module                = (*Middleware)(nil)
    _ caddy.Provisioner           = (*Middleware)(nil)
    _ caddy.Validator             = (*Middleware)(nil)
    _ caddyfile.Unmarshaler       = (*Middleware)(nil)
    _ caddyhttp.MiddlewareHandler = (*Middleware)(nil)
)

Since I want the module to register itself with Caddy, it will implement the caddy.Module interface. I want to embed a logger, so it needs to implement the caddy.Provisioner interface. The shared secret is a requirement, which I can enforce if the module implements the caddy.Validator interface. I like the convenience of using a Caddyfile, which dictates the module should implement the caddyfile.Unmarshaler interface. Lastly, I’m writing middleware, so it needs to implement the caddyhttp.MiddlewareHandler interface.

The Module Interface

The caddy.Module interface is quite simple, and you’ll see this pattern almost verbatim in all Caddy modules. It defines a method named CaddyModule() that returns a caddy.ModuleInfo struct. My module’s implementation looks like this.

// CaddyModule returns the Caddy module information.
func (Middleware) CaddyModule() caddy.ModuleInfo {
    return caddy.ModuleInfo{
        ID:  "http.handlers.validate_github",
        New: func() caddy.Module { return new(Middleware) },
    }
}

The important parts here are the ID field and the New field. The ID field defines the module’s namespace. Since this module is middleware, I give it a name under the http.handlers namespace. If you create a module and give it the same http.handlers.validate_github namespace, and then attempted to register your module and my module with the same Caddy instance, a panic will ensue. Try to be unique, at least amongst your install base. I’m digressing.

The New field defines a function that returns a caddy.Module.

The Provisioner Interface

Since I want to embed a logger, the middleware must implement the caddy.Provisioner interface before Caddy will provision one.

// Provision implements the caddy.Provisioner interface.
func (m *Middleware) Provision(ctx caddy.Context) error {
    if m.logger == nil {
        m.logger = ctx.Logger(m).Sugar()
    }

    return nil
}

Here, I retrieve a pointer to the logger from the caddy.Context and since I’m less concerned about performance and more about flexibility, I assign a sugared logger to the middleware. Typically, you don’t need to check if the logger is nil before assigning one. I’m writing tests, and I want the flexibility to inject my own logger, specifically a zaptest logger. If I didn’t check for nil here, provisioning would overwrite my injected logger. There may be other ways to inject a logger into the caddy.Context in my tests, but this approach is simple, which I prefer.

The Validator Interface

The caddy.Validator interface defines a method that returns an error. I’ll use this method to ensure I add a shared secret to the Caddyfile or that there isn’t a bug in my code (e.g., neglecting to make a method that modifies Middleware use a pointer receiver).

func (m *Middleware) Validate() error {
    if m.Secret == "" {
        return fmt.Errorf("empty secret")
    }

    m.secret = []byte(m.Secret)

    return nil
}

If Caddy didn’t populate the middleware’s Secret method, I want to stop the show and tell Caddy to complain, loudly. Therefore, I check if the Secret field is populated and return an error if not. I could enforce other requirements, like a minimum length. But for now, this suits my needs.

If everything looks good, I populate the secret field by casting the Secret field’s string to a byte slice (don’t forget the pointer receiver!). This secret field’s value will be used to derive the HMAC hash for each request, so casting it now saves the slightest bit of overhead with each request and marginally simplifies the code later on.

The Caddyfile Unmarshaler Interface

Adding Caddyfile support to a module seems to be an afterthought for most third-party modules I’ve reviewed. It isn’t as intimidating as it may seem, especially not for this middleware since it only requires a shared secret.

The caddyfile.Unmarshaler interface describes a method that accepts a caddyfile.Dispenser and returns an error.

// UnmarshalCaddyfile implements caddyfile.Unmarshaler. Syntax:
//
//     validate_github <secret>
//
func (m *Middleware) UnmarshalCaddyfile(d *caddyfile.Dispenser) error {
    for d.Next() {
        if !d.Args(&m.Secret) {
            return d.ArgErr()
        }
        if d.NextArg() {
            return d.ArgErr()
        }
    }

    return nil
}

The initial call to d.Next() consumes the validate_github argument from the Caddyfile. The next call to the d.Args() helper consumes the shared secret and assigns it to the middleware’s Secret field. Lastly, I’m checking if there’s any additional arguments and returning an error, if so.

Despite the temptation, this isn’t the appropriate place to validate the middleware. That’s the responsibility of Validate().

The Middleware Handler Interface

The caddyhttp.MiddlewareHandler describes an object that Caddy can use as middleware or as a handler. The object has a ServetHTTP method that accepts an http.ResponseWriter, an *http.Request, and a caddyhttp.Handler. The first portion of the implementation looks like this.

// ServeHTTP implements the caddy.Handler interface.
func (m Middleware) ServeHTTP(w http.ResponseWriter, r *http.Request,
    next caddyhttp.Handler) error {
    b, err := ioutil.ReadAll(r.Body)
    if err != nil {
        m.logger.Errorf("reading request body: %v", err)
        http.Error(w, "invalid signature", http.StatusForbidden)
        return nil
    }
    _ = r.Body.Close()

    if len(b) == 0 {
        m.logger.Debug("cannot validate an empty request body")
        http.Error(w, "invalid signature", http.StatusForbidden)
        return nil
    }

First, I need to read in the entire request body even though I don’t plan on consuming any of it. The request body plays a role in calculating the HMAC and validating GitHub’s request. If I don’t receive a request body, there’s nothing to validate, so the client receives a 403-status code.

But if I read at least something from the request body, I next need to retrieve the signature from the request headers.

    s := strings.TrimPrefix(r.Header.Get("X-Hub-Signature-256"), "sha256=")
    if s == "" {
        m.logger.Debug("missing X-Hub-Signature-256 header in request")
        http.Error(w, "invalid signature", http.StatusForbidden)
        return nil
    }
    sig, err := hex.DecodeString(s)
    if err != nil {
        m.logger.Debugf("error hex-decoding signature '%s': %v", s, err)
        http.Error(w, "invalid signature", http.StatusForbidden)
        return nil
    }

GitHub includes a X-Hub-Signature-256 header in its request whose value is derived from the shared secret and the request body. It’s prefixed with the sha256= string followed by the hex-encoded HMAC hash. I want to compare the original hash, so I hex-decode the value after stripping the prefix.

Now, I can calculate the HMAC hash from the shared secret and the payload, and compare my result with GitHub’s signature. If they’re equal, the request is valid.

    mac := hmac.New(sha256.New, m.secret)
    mac.Write(b)
    if sum := mac.Sum(nil); !hmac.Equal(sum, sig) { // constant time comparison
        m.logger.Debugf("signature: expected '%x'; received '%s'", sum, s)
        http.Error(w, "invalid signature", http.StatusForbidden)
        return nil
    }

    m.logger.Debugf("successful webhook invocation from %s", r.RemoteAddr)
    r.Body = ioutil.NopCloser(bytes.NewBuffer(b))

    return next.ServeHTTP(w, r)
}

I create a HMAC hash using the shared secret and write the request body to it. Then, I calculate the final hash and use hmac.Equal() to compare them. If false, the request is invalid, and I return a 403-status code.

If the request signature is valid, I re-populate the request body with an io.ReadCloser in case downstream middleware or handler needs it.

Constant Time String Comparisons

Simple string comparisons, like string1 == string2 are subject to timing attacks. It’s inappropriate to compare the hex-encoded request signature with one I calculate since this comparison can leak details an attacker could use to derive my shared secret. Granted, this blog isn’t a juicy target, but I’ve watched enough Black Hat talks and dabbled in a few side channel attacks to take this stuff seriously.

If you’re interested in learning more about timing attacks, this video is one place to start.

You can perform constant time comparisons in Go using hmac.Equal() if you’re comparing HMAC hashes, as I’m doing here, or something like subtle.ConstantTimeCompare() when comparing strings (casted to byte slices).

Registering the Module

My module is almost ready for use in Caddy. Lastly, it needs to register itself with Caddy and tell Caddy about its Caddyfile directive.

func init() {
    caddy.RegisterModule(Middleware{})
    httpcaddyfile.RegisterHandlerDirective("validate_github", parseCaddyfileHandler)
}

The caddy.RegisterModule() function accepts an instance of Middleware. At this point, I could use JSON to configure the middleware and use it with Caddy. However, I added Caddyfile support. I should let Caddy know. The last detail in that regard is the registration of a new Caddyfile directive. I settled on naming it validate_github.

The second argument to httpcaddyfile.RegisterHandlerDirective() is a function that returns a caddyhttp.Middlewarehandler. I called this function parseCaddyfileHandler(), which looks like this.

func parseCaddyfileHandler(h httpcaddyfile.Helper) (caddyhttp.MiddlewareHandler, 
	error) {
	m := new(Middleware)
	if err := m.UnmarshalCaddyfile(h.Dispenser); err != nil {
		return nil, err
	}

	return m, nil
}

Since the middleware implements the caddyfile.Unmarshaler interface, I can leverage that here by instantiating a new Middleware object, passing the caddyfile.Dispenser to its UnmarshalCaddyfile() method, and returning the now-populated object.

With this code in place, I can compile it into Caddy by importing it like so.

import (
	// other Caddy-related imports here ...
    _ "github.com/awoodbeck/caddy-validate-github"
)

Then, I can add a route to my Caddyfile and use this middleware to validate requests before they’re passed off to caddy-exec.

route /update {
    validate_github KcuP9N0iEqYHFBRUda6oHLP4UUub6EMz
    exec * /path/to/bin/update-blog.sh
}

Now, I have a defined route that validates GitHub’s webhook request and passes it onto caddy-exec, which triggers the update of my blog.

You can find the full source code for this module on GitHub.

Creating the Middleware#

Interfaces#

The Module Interface#

The Provisioner Interface#

The Validator Interface#

The Caddyfile Unmarshaler Interface#

The Middleware Handler Interface#

Constant Time String Comparisons#

Registering the Module#