From the launch of the Lagom framework by Lightbend, to Mantl 1.0 from Cisco, across the sector, the first generation of microservice-centric offerings from the enterprise is coming of age. The latest release to catch our eye at Voxxed is VAMP – the self-proclaimed Very Awesome Microservices Platform built by Magnetic.io, which makes some pretty bold claims in its branding. In this interview, company co-founder Olaf Molenveld explains what the team are trying to achieve with VAMP, and why you might want to consider investing some time to take its canary-testing, release and autoscaling features for a spin. (If you’d like like a little clarification on what canary testing actually is, check out the visual demo filmed by the NetBeans crew at the bottom of the page.)
Voxxed: What big problems are you trying to solve with VAMP?
Molenveld: High Level goals: VAMP is helping organisations to deliver higher quality software in less time. It does this by providing canary-testing and releasing and autoscaling features in an easy-to-use open source solution. These features are specifically relevant for organisations that adopt containers and microservices.
The software’s features enable DevOps and business teams to solve the following business and IT problems:
- Performance issues and instability: a gradual, controlled and measured release will identify potential performance issues before they become real problems.
- Downtime when upgrading: VAMP enables a gradual cross-over from current to new versions of your online services, without any downtime for users.
- Improve Time-to-market: automated canary-testing and -releasing is the holy grail for Continuous Delivery and helps to deliver new functionalities easier, more often and with higher quality and trust.
- Validating business cases asap: A/B testing of entire applications, services or architectures on limited segments of visitors, compared to only A/B testing visuals or content.
- Removing (cloud) vendor lock-in: Our solution is container-agnostic and can integrate with all major container-schedulers and container-clouds. This delivers freedom of choice when selecting cloud or container providers.
More technically, when adopting container technology, VAMP solves the following challenges:
- Container-cloud solutions like Docker (Swarm), Mesos/Marathon and Kubernetes provide the deployment, running and scaling of containers on clusters. But they don’t provide features to automate and manage when and how to deploy and scale these containers and how to load-balance and route traffic to these containers. This is essential for a Continuous Delivery loop with zero-downtime releases/upgrades, on-demand scaling, and Canary-testing.
- Continuous Delivery (CD) platforms need an “experiment system” to avoid possible issues being pushed to full-scale production. VAMP provided the features to create such an “experiment system”.
By delivering these features in an easy to use package we believe VAMP can extend a Continuous Delivery pipeline into a data-driven Continuous Improvement loop.
How do you stand out from other solutions on the market?
VAMP has a specialised focus on canary-testing & releasing features and workflows. It’s cloud and container vendor agnostic, open (source) and modular which integrates with all major container solutions and clouds and can be easily expanded to new solutions that will appear in this space. The open and modular approach makes VAMP easy to integrate in existing IT landscapes and container-schedulers.
It also provides easy-to-use SLA-based auto-scaling and optimisation workflows, and offers a user-friendly approach with graphical User Interface and dashboard, REST API’s. Developers experience a quick learning curve, and real world value can be achieved very quickly.
Can you give us a technical deep dive?
The VAMP framework is open source Apache 2.0 licensed, and consists of two core components: VAMP and Vamp Gateway Agent (VGA). It’s a JVM-application and is written in Scala and Akka. It is the API endpoint, business logic and service coordinator. It communicates with the configured container manager (Docker, Marathon etc.) and synchronizes this with Vamp Gateway Agent via ZooKeeper, etcd and Consul. VAMP uses Elasticsearch for artifact persistence and to store Vamp events (e.g. changes in deployments).
VampGatewayAgent (VGA) is a lightweight Go-application that works in tandem with HAProxy. VGA reads the HAProxy configuration from ZooKeeper, etcd and Consul and reloads the HAProxy on each configuration change with as close to zero client request interruptions as possible. It reads the logs from HAProxy over socket and pushes them to Logstash over UDP. It handles and recovers from ZooKeeper, etcd, Consul and Logstash outages without interrupting the HAProxy process and client requests.
We also provide an open source graphical UI which is packaged with VAMP, and a CLI is available for download.
VAMP was designed from the ground up to run in high-availability and high-performance setups, and to be open and modular. It supports multiple container-orchestration platforms and will expand the number of supported container-orchestration platforms in the future. When installing VAMP, you choose your container-orchestration platform and configure the appropriate driver. Think of how ORM’s work: a single DSL/language with support for multiple databases through a driver system. Currently we support Docker and Mesos/Marathon, and DCOS and Kubernetes drivers are in the works. We’re also very much open for collaborations in that department.
The software uses Elasticsearch (ES) as it’s main persistence store (e.g. for artifacts and events). Vamp also depends on a distributed key-value (KV) store for communication between Vamp and Vamp Gateway Agents (VGA). Currently ZooKeeper, etcd and Consul are supported. Typically there should be one Vamp instance and one or more VGA instances. There is no direct connection between Vamp and VGA instances – all communication is done by managing specific Key/Value in the store. When VAMP needs to update HAProxy configurations (e.g. a new service has been deployed), VAMP generates the configuration and stores it in the KV store. VGA’s read the specific values and reload the HAProxy instances. There should be one dedicated HAProxy for each VGA. Vamp also supports custom HAProxy configurations – base configuration should be used as a template and HAProxy frontends and backends are then appended by VGA.
Since VGA (and HAProxy) can be a single point of failure when only running one instance of it, it is recommended for high availability setups to have more than one VGA instance. VGA instance can be added or removed any time – once VGA starts running it will pick up the HAProxy configuration from the configured KV store and it will reload the HAProxy instance. This also means that VAMP (not VGA), can be restarted, stopped etc. without main consequences on running services – there will be no HAProxy configuration update, but once VAMP is up, it will sync HAProxy configurations (e.g. if Marathon restarted some service, so hosts/ports are changed).
Why did you decide to go vendor neutral?
We believe that our focus on higher-level canary-test and release features delivers functionalities that are relevant and useful for all container-schedulers and -clouds. We also believe that one of the main drivers of adopting container-technology is its abstraction, and thus we think that enabling vendor-independence is a very important selling point for container-related tools like VAMP.
Finally, the container race has just started and is not finished yet. We believe in agile architecture, which means that VAMP should be able to work with and deliver value for all current and upcoming container-technologies, in essence and design be future-proof.
What’s the roadmap ahead?
We are currently wrapping up Alpha phase and will be moving into Beta in a few weeks time. This means we will increase our efforts on making VAMP’s performance and stability even better, while keeping the API stable, and at the same time adding useful features and improving the open source VAMP framework.
We are ramping up trial and launching-customers and see a quick increase in interest in the canary and autoscaling features. We will be focusing on making the adoption of VAMP as easy as possible with installation-wizards and lots of valuable content like tutorials and best-practices for common canary patterns.
We are working on new integrations for several container scheduling platforms and clouds, and we will be adding (commercial) enterprise features like audit-trailing, SSL support and LDAP/AD integration.
Finally, we are building additional modules on top of VAMP open source, that will also be commercially licensed. The first will be a canary-analytics engine with an integrated dashboard and graphical UI. The canary-analytics engine will constantly monitor and analyse all relevant technical and business metrics that flow through the system, is able to compare new versions to currently running versions, and indicate how these new versions compare to the old ones with statistical significance.
To deliver this we are currently expanding our (Amsterdam-based) team, so if you’re interested in helping out or collaborating on the open source version, we’d be more than happy to have a chat!