CodeGuard, like many rapidly growing software as a service businesses with a maturing product, is breaking apart our monolithic software architecture into smaller, modular services, or a Service-Oriented Architecture (SOA). While monolithic architecture has the advantage of avoiding premature architectural decisions when at the experimental stage of a product’s life, it is difficult to scale after this stage and lacks modularity by definition. As monolithic applications grow, it becomes increasingly difficult to reason about the impact of changes and predict or avoid side-effects. For these reasons, new features and products at CodeGuard are built using an SOA approach with modularity in mind. In addition, functionality of current monolithic applications is broken out into separate modular services when possible.
SOA by Command-line Interface
For many developers, the term SOA, or Service-Oriented Architecture, is synonymous with web services. This perception is not only a misconception, but also misses the core ideas of SOA and focuses instead on the method of implementation. SOA, as the term suggests, is instead about software architecture focusing on loosely coupled units that are self-contained and expose well-defined interfaces for the services they implement. While web services are one way to implement SOA, they are not always the best choice, especially when breaking apart monolithic software. When our engineering team decided to build a backup verification service we implemented it as a command-line interface, or CLI.
Reasons for choosing this approach over a web service include:
I/O cost: A web service is typically deployed on separate infrastructure from the client of that service. While this can be an advantage, it can also incur overhead if the service requires access to data such as a backup. For backup verification, this data may already be available on local disk and running a service locally via a command avoids the cost of retrieving that data twice.
Everything can shell out: Building a service as a CLI is language-agnostic since every language and framework that runs on linux can shell out. In this respect, it is no more limiting than a web service and possibly less so as some technology stacks lack robust networking or good support for relatively newer technologies like JSON.
No data model duplication: In addition to avoiding I/O overhead, a CLI can help avoid the temptation of duplicating data models or relationships from the client application.
Well-defined, documented API: Choosing to build a service as a CLI instead of a web service does not mean compromising on the interface for the service. A CLI can explicitly require options and combinations of options, returning an error if the requirements are not met. Documentation tools include help output and man pages and can often be auto-generated from code.
Easy deployment: A web service has to be deployed, the clients of the service have to be configured to know where to send requests or use service discovery, and then of course the service must be available at all times that it is needed. A CLI, in contrast, needs only to be deployed (in many cases just a binary) to the servers that call it.
CLI -> Web Service: A CLI is just another form of interface and a CLI, if it is built with modularity and separation of the interface from the implementation, can become a web service without rewriting the functional code. Instead of command-line options, the options are specified in a network request, usually an HTTP request containing JSON.
Backup Verification Service
One of the services that the CodeGuard engineering team has built as CLI is our backup verification tool. This tool is used to independently verify that the file backups of our customers’ websites are complete and correct. It does not share any code with our backup system. Given the location of a backup directory structure, it compares the contents to the contents of the remote server and logs any differences it finds, including missing files, files of the wrong size, additional files, and files with differing metadata. This service is built in Go (www.golang.org), which compiles to easily-deployable static binaries with no dependencies and is also simple to cross-compile.
A simplified version of the CLI is included below:
To separate the interface from the implementation of the service, we use the excellent go-flags package. Using the backup verification service as a CLI, the backup system can simply shell out to start a verification. Logs of differences found or errors are written to the standard error output and can be retrieved by the client service. The interface clearly documents the parameters required to run a verification and will exit with an error if the required parameters are not included as command-line flags.
At CodeGuard, we also build web services as part of our SOA, but they are certainly not the only way to build a service-oriented architecture. Often they are not even the best choice as the overhead and complexity incurred can be unacceptable or lead to new problems. As our architecture evolves we may someday need to run our backup verification service as a web service. When that day comes we will need only to rewrite the implementation of the wrapping interface. The encapsulated functionality may not change at all. For now, a CLI gives us all the major benefits of SOA without unnecessary architectural complexity or I/O overhead.
-Randall