Skipper: Tooling for DevOps

What Developers Want

We identified that developers needed to know if there was an issue with their site in production. They are usually the most knowledgeable about the architecture, idiosyncrasies of the application, and know where the performance bottlenecks are.

To this end, we built a command line-client (CLI) called skpr which provides a number of features to allow developers to know what is going on.

An Overview of All Environments

On simple projects, there might only be dev, staging, and production sites. On larger sites there might be a lot more. The skpr list command provides an overview of what environments exist, what version is deployed, the current resources allocated and the current CPU activity. 

Log Streams

Skpr provides quick way to stream the application logs from across multiple application instances. This allows the developer to quickly see if there are any errors that would provide information on what could be causing issues.

Log History

In addition to providing the current log streams, developers can view a history of logs by providing some parameters. For example, show me the logs between 3:40 pm and 3:50 pm yesterday afternoon.

Metrics

Skpr can show what the current CPU and memory usage is, as well as how many application instances are currently active. This provides useful insights for the developers to see how their application is performing in production.

Deployments

Our deployment pipeline involves creating Docker containers and automatically tagging them using git describe --tags. This allows us to track exactly what version of the code is in each environment. skpr provides a ‘list’ command to let you see which versions have been deployed. You can also deploy directly from the CLI specifying a Docker tag using the ‘deploy’ command.

Syncing Files

Often developers will want to copy files to and from an environment. skpr provides and ‘rsync’ command which lets you download files for local development, or upload files to the hosted environments.

Shell Access

Of course, despite all these commands, sometimes a developer will need to shell into an environment to view a file, or run a drush command. skpr provides this too, via an authenticated connection, without the developer needing to know server addresses etc.

Cron Jobs

Although we supported recurring tasks, recently we added support for scheduled tasks using cron syntax. This allows you to specify which tasks get run by specifying them in the .skpr.yml file in your project repo. This is perfect for running drush cron or drush search-api-index at intervals you want, without having to bother sysops. In addition, skpr has a command skpr cron which shows you a history of which cron tasks were run, and the results.

What SysOps Want

What SysOps don’t want is to be pinged every 5 minutes to be asked for logs, or information about the production environments.

Most of the above tools provide developers direct access to this information, and the need to involve a SysOp is greatly reduced.

SysOps are able to focus on the environments, in a structured and declarative way, then get out of the way.

.skpr.yml

The skpr CLI works with a .skpr.yml file which defines the environments. Things that can be configured per environment include:

• CPU share
• Minimum and Maximum memory limits
• Minimum and Maximum application instances in autoscaling rules
• Cron tasks

Summary

Providing tools like skpr is bringing the worlds of developers and SysOps together. Developers have access and insights into the production environments, and the ability to deploy. SysOps can provide managed infrastructure with a structured framework. Everyone wins.