Composable Go Services Using Libchan

With the increasing popularity of container technologies and microservices, a number of challenges have arisen around service discovery and scale. The separation principles of microservices, when applied to a fresh application at low scale, would be considered by many to be overengineering at its finest. Solving these challenges at an early, prototyping stage could mean a costly investment at a fragile stage of a project.

In addition to solving this either by investing heavily in microservice architecture upfront or following a monolith approach, the flexibility of Go provides an alternative. Libchan provides a generic wrapper around a number of transports including a Go channel, a Unix socket, and HTTP. This means that by connecting components of an application using libchan instead of using standard Go channels allows the transport to be switched without altering any business logic of the application.

In this article, I’ll go through some of the steps needed to use libchan to power a generic interface between application components. By designing your application in this manner, you’ll be able to switch out the transport-connecting components as part of an infrastructure design change or as a response to monitoring. The Docker network plugins can assist in simplifying the service discovery of connected components. We’ll talk about this later in this article, but also keep an eye on the Codeship blog for future articles about using the Docker plugins.

To demonstrate this concept, I’ve built an example application: Banano. This can be run as a standalone, monolithic application or be deployed using a client-server microservices model, with very little difference in the code being run in each scenario. I’ll be referring to code examples from this project throughout the article; if you want to try out this concept, you can do so using Banano. You can use Docker Compose to start the services. See the project readme for more.

It is important to understand that while libchan claims to be “like Go channels over the network,” there are several differences to how libchan behaves when compared to standard Go channels. As such it’s important to consider it to be an independent channel implementation but still useful.

Libchan provides a Go channel style interface. A message is dispatched via a sender, and it is received via a paired receiver. The details on how the message is transported or where the receiver is in relation to the sender are obscured from the dispatcher component. Messages are dispatched asynchronously, and in order to receive a response, a good practice is to send a responder channel as part of the request. Libchan handles all the routing to ensure that the linked sender and receiver remain connected across the transport, even if the receiver is remote.

Using libchan in an application involves separating out component interfaces using a libchan sender and receiver pair. All interaction between these components is done by passing messages via the sender and receiver pair. This will guarantee consistent interaction between local and remotely configured instances.

The only requirement for linking components across a transport is access to the sender and receiver, paired with a sender and receiver on the other end of the link. Beyond this, the only complexity is around the marshalling and unmarshalling of data and serving sender and receiver channels to those components depending on the required configuration.

For a local configuration, this is as simple as creating two libchan pairs and passing them into each component. For a remote connection, this is more work involved since the intermittent nature of the connection has to be taken into account. In this case, the link must be built from a base connection, up to a libchan compatible transport, and finally to a sender and receiver pair. Since in the client-server model the server must support multiple clients and the connection can be broken and reestablished, the sender and receiver pair must be considered throwaway. This must hold true for the local instance in order to maintain cohesion.

The simplest implementation of an application using libchan involves creating an uplink pipe and a downlink pipe.

In this scenario, the server side receiver is termed the remote receiver and is triggered by an incoming message via the configured transport. The server side sender is termed the remote sender, dispatched as part of the initial message.

Pipes are simple to create:

receiver, remoteSender := libchan.Pipe()
remoteReceiver, sender := libchan.Pipe()

The sender is used to dispatch messages from the client, and the receiver to accept the responses. For a local instance, we can rely on the stability of the transport and can keep the same pipes for the entirety of the application’s life. All we need to do now is pass the sender, receiver, and the remoteSender to our client, and pass the remoteReceiver our server.

In the Bonano example, the repository is our client and the adapter is the server side. The repository dispatches messages via the sender, each message containing a remoteSender. This allows the server side to receive the message and return a response using the remoteSender provided in the message itself.

Lets take a look at the initialization:

senderFunc := func() (libchan.Sender, error) {
    return sender, nil
}
repo := NewThingeyRepository(senderFunc, receiver, remoteSender)
adapter := NewThingeyAdapter()
go func() {
    for {
        adapter.Listen(remoteReceiver)
    }
}()

In this case, we’re using a senderFunc instead of a sender which allows the repository to handle cases where the transport is intermittent. Similarly, we’re passing the remoteReceiver into a handler function (Listen) on the adapter. This ensures that the repository and adapter are consistently linked across whatever transport is being used.

On the client side, our repository forms a request and dispatches it via the sender. A response is then received via the receiver and handled.

req := &ThingeyCreateRequest{thingey}
data, err := json.Marshal(req)
if err != nil {
  return nil, err
}
request := &Request{
  Payload: data,
  Type: "ThingeyCreateRequest",
  ResponseChan: repo.remoteSender,
}
sender, err := repo.senderFunc()
if err != nil {
  return nil, err
}
if err := sender.Send(req); err != nil {
  return nil, err
}
response := &Response{}
if err := repo.receiver.Receive(response); err != nil {
  return nil, err
}
if response.Type == "ThingeyCreateResponse" {
  // handle response
}

On the server side, our adapter receives a message and extracts the payload and the response channel. After handling the message appropriately, a response is formed and dispatched down the response channel.

request := &Request{}
response := &Response{}
err := receiver.Receive(request)
if err != nil {
  return err
}
if request.Type == “ThingeyCreateRequest” {
  payload := &ThingeyCreateRequest{}
  if err = json.Unmarshal(request.Payload, payload); err != nil {
    return err
  }
  // handle request
  response.Payload, err = json.Marshal(&ThingeyCreateResponse{})
  response.Type = "ThingeyCreateResponse"
  return request.ResponseChan.Send(response)
}

Much of the logic here is handling errors and marshalling/unmarshalling content. The only libchan related pieces are the receiving and sending actions. Running the local example in Banano shows an item being stored, retrieved, and deleted using an in-memory map:

10:43:13-bfosberry~/.go/src/github.com/bfosberry/banano (master)$ go run cmd/main.go
2015/08/31 08:02:14 Starting..
2015/08/31 08:02:14 Created d1
2015/08/31 08:02:14 Got d1
2015/08/31 08:02:14 Listed 1 items
2015/08/31 08:02:14 Deleted d1
2015/08/31 08:02:14 Listed 0 items

With a remote configuration, it gets a little more complicated due to the transport initialization. However the entirety of the adapter and repository code is exactly the same. All that changes is how sender and receivers are created.

On the client side, the receiver and remoteSender are still initialized as a Pipe, since libchan will maintain the connection between those endpoints for us. In this example, we’re using spdy to wrap a tcp endpoint.

client, err = net.Dial("tcp", remoteURL)
if err != nil {
  log.Fatal(err)
}
transport, err := spdy.NewClientTransport(client)
if err != nil {
  log.Fatal(err)
}

Our transport exposes a senderFunc for us via NewSendChannel, so we can initialize the repository with a reference to this function. This ensures that intermittent network issues do not orphan libchan pairs.

repo := NewThingeyRepository(transport.NewSendChannel, receiver, remoteSender)

On the server side, it gets a little more complicated. We initialize a transport listener in a similar way, however this component requires some looping around accepting connections and receiving channels. For the entire code listing for this section, see the Banano project on GitHub.

listener, err := net.Listen("tcp", fmt.Sprintf("localhost:%s", port))
if err != nil {
  log.Fatal(err)
}
tl, err := spdy.NewTransportListener(listener, spdy.NoAuthenticator)
if err != nil {
  log.Fatal(err)
}
adapter := nano.NewThingeyAdapter()

After creating a transport listener, we accept a session and handle it:

for {
  t, err := tl.AcceptTransport()
  if err != nil {
    log.Print(err)
    break
  }
  go func() {
    // handle sesssion
  }()
}

Handling the session simply involves waiting for any receivers to be sent:

for {
  receiver, err := t.WaitReceiveChannel()
  if err != nil {
    log.Print(err)
    break
  }
  go func(){
    // handle receiver
  }

In our case, handling the receiver is simply a matter of calling adapter. Listen with the receiver the transport provided.

Running a remote server and client yields exactly the same result as a local instance:

08:46:46-bfosberry~/.go/src/github.com/bfosberry/banano (master)$ go run client/main.go
2015/08/31 08:46:50 Starting..
2015/08/31 08:46:50 Created d1
2015/08/31 08:46:50 Got d1
2015/08/31 08:46:50 Listed 1 items
2015/08/31 08:46:50 Deleted d1
2015/08/31 08:46:50 Listed 0 items

An implementation like this may not be ideal for all situations; it does add complexity around initialization of application components and also around concurrency. Much of the initialization logic as well as any marshalling can be simplified and abstracted. In cases where Go channels are already being used to isolate and scale components, adding in libchan would increase complexity by a minimal amount.

The ideal use case for libchan would be a monolithic application being split up into microservices. Due to the native Go nature of libchan, minimal support code is needed to distribute components, and the margin of error is decreased when compared to a standard services api. The key factor here is the simplicity involved in setting up a libchan connected application. There are many excellent reasons for setting up an architected set of microservices, however complexity and effort are barriers to this. By using libchan as a channel fabric, this goal can be achieved with very little effort.

Another significant challenge when it comes to microservices is the process of service discovery. Since one of the big benefits of using libchan is simplicity, pairing it with a simple service discovery method is ideal. With the recent release of Docker plugins, we can make use of some nonstandard container networks. Be sure to check out Calico and Weave. Each of these, in different ways, allows you to create finely controlled and isolated networks linking containers. Both allow containers to be routable by IP across hosts and provide simple DNS and proxy components.

Starting with a growing, monolithic application, with the goal of splitting it into a set of microservices, a good, iterative process is:

1. Identify logical boundaries within the application.

2. Replace those logical boundaries with libchan operating over a local Go channel.

3. Add support for those downstream components to be hosted as standalone singletons over an appropriate libchan transport.

4. Allow the upstream components to switch between local and remote instances of those components. This can be simplified by using Docker network plugins to reference containers by IP as a method of service discovery.

5. Extend downstream components to operate at scale, implement proxy and DNS network components as needed.

This process can continually be applied to the largest services in the architecture. Monitoring could also be used to dynamically switch components between being local during periods of little or no traffic, and remote at scale during burst periods.

Libchan provides a simplified interface allowing Go services and components to natively interact across a number of different transports. This can be a powerful way of standardizing application interaction and ensures that components respond and behave in a repeatable manner when running locally or as a remote service. This generic layer can ease the transition from a monolithic application into a set of microservices. The available Docker network plugins complement this by providing features and tooling that support simplified network discovery.

There are a number of caveats with this:

• All logic in the server side handler should return errors down the response channel; otherwise the client side will hang.

• The client side receiver should time out appropriately.

• Not all transports behave equally; all handlers should serve the lowest common denominator.

• The libchan project does not seem to be supported at the moment. Apparently Solomon Hykes has something more important to do.