By Ron Fabio
Why Docker Is Not the Best Container Solution for JVM Applications
Virtualized containers and container solutions – first among them Docker – are a smash hit. Containers provide a mechanism for isolating applications and controlling their resource usage that is more performant and less wasteful than virtual machines (although less secure), and Docker (and other similar solutions) provides a portable, mechanism to package and deploy applications into containers. There is also no lack of advice on how to package and deploy JVM applications with Docker, but we believe Docker and the JVM don’t make a good match.
Docker provides packaging and portability through container images, the management of which is arguably Docker’s main feature. But JVMapplications are already portable (i.e. if properly packaged and deployed – we’ll get back to that), and a JAR file makes for much a smaller and convenient package than an image. Docker images for JVM applications require that the JVM itself be included in the image, even though JVMs are completely standard, need no special arrangement for portability, and are likely to already by installed in any machine used by JVM shops (or could easily be installed in any cloud VM without introducing any portability issues among different distributions). Isolation and resource restriction are what containers are all really about, and those are provided by the OS (sometimes wrapped by tools for easier use) and don’t require Docker’s additional features.
Docker is a terrific product, but as a general-purpose solution, is simply too blunt an instrument for the deployment and management of JVMapplication containers. On the one hand, most of what it does is not required by the JVM, and on the other, it can’t leverage the JVM’s specific strengths. Image management, logging and monitoring present challenges to general-purpose container solutions, but are non-issues for the JVM.
Capsule is a simple, lightweight and powerful open-source packaging and deployment solution for JVM applications. One way of thinking about a capsule is as a fat JAR on steroids (that also allows native libraries, never interferes with your dependencies) and a declarative startup script rolled into one; another, is to see it is as the deploy-time counterpart to your build tool. Just as a build tool manages your build, Capsule manages everything from the build to the launch of your application. Capsule, then, takes care of the problem of portability by handling the application’s packaging, dependencies and launch configuration. It doesn’t require anything at all to be installed on the host other than a JVM.For a more thorough discussion of Capsule and its design see our previous post. If all you need is portable, convenient packaging that means you don’t have to worry about dependencies and configuration, then Capsule does all that without requiring any new tools. But if you also need the isolation and resource-restriction offered by containers, for that there’s Shield.
Capsule Shield is one of Capsule’s available caplets. Caplets are classes that hook into the capsule and modify its launch behavior. A capsule contains metadata about your application and the means to execute it as a plain JVM program. Caplets can use the metadata in the capsule to launch the application in some more sophisticated ways, or customize its packaging. For example, there are caplets that allow a capsule to declare its dependencies rather than embed them in the JAR and then downloads those dependencies when the application is first launched. Another launches your capsule as a daemon, and yet another can turn any capsule into a native binary package. Caplets can be either embedded in the capsule, or used at the command line as wrappers that control the behavior of a capsule.
Capsule Shield is such a (usually wrapper) caplet that launches any capsule inside a container rather than directly in the host OS. At the moment, Shield is Linux-only, but future versions will support other operating systems. Shield will also be (optionally) fully compatible with runC and the Open Container Initiative, and will soon support linking to Docker containers, so the two will work perfectly together.
How Capsule Shield is Tailored for the JVM
Capsule, and Shield in particular, is a solution for the JVM, by the JVM. Capsule and Shield separate Docker’s features into two orthogonal concerns. Capsule takes care of packaging and portability, while Shield adds isolation and resource restriction with the following features:
- Footprint: Docker images are big, and managing their archival and evolution can become a serious hassle (they could be made smaller, but that requires extra hassle to factor out common dependencies into different layers). On the other hand JVM applications need nothing more than a JVM and a kernel, so Shield takes only the application’s capsule (a single JAR file), which contains or lists all its dependencies, and launches it in a container. Unlike Docker images – that take a long time to build, are big, require the inclusion of the JVM itself as well as a repository – capsules are packaged in a matter of seconds with any build tool (there are community-contributed plugins for Maven, Gradle and Leiningen, but you can package capsules with other build tools, too, with no need for special plugins). And if you’d like to place your capsules in a repository, you’re welcome to use industry-standard Maven repositories. If you’re application’s dependencies are too large, simply declare – rather than embed – them in the capsule, and let the Maven caplet handle the caching and sharing of dependencies for you.
- Security: Container security is a debated issue, with the consensus being that containers are not (yet) secure enough. However, unlike Docker, Shield uses unprivileged containers by default (which work well for JVM applications), and will map the container’s
rootto a regular user on the host, so that no harm will occur to the host if the application somehow manages to escape the container. Secrets can be passed down from the host through environment variables or Java system properties.
- Linking: Java already supports customized domain-name resolution, so Docker’s solution of modifying the container’s hosts file is unnecessary (and, indeed, the container doesn’t even need its own hosts file). Shield leverages this feature to allow easy linking of containers to other running containers.
When Shield is used to launch a capsule, it will create a container as well as a simple JVM process on the host that will reflect the application’s monitoring, management and logging functions, so that it could be monitored/managed and its logging configured as if it were a simple process even though it is running inside a container. This lets Shield do the following:
- Logging: Applications in general can use a variety of logging mechanisms, tools and processes, and the logs they produce need to be somehow delivered outside of the container. Docker has recently introduced logging drivers, those are rather crude, general-purpose tools.JVM applications, on the other hand, make use of a well-defined set of logging APIs (e.g. Log4J, SLF4J, Commons Logging, j.u.l) and Shield will automatically forward them all to the controlling capsule process in the host (as Log4J 2 logs), making log management for JVMapplications running in containers (including configuration of logging levels) just as straightforward as if they were running in the host.
- Monitoring and Management: The JVM has its own rich monitoring and management API, JMX, which is used by the platform itself, libraries and applications. Shield lets you manage and monitor your containerized application by reflecting all its monitoring info and management commands into the controlling capsule process, which is a simple Java process running in the host. Just connect to it with any JVMmonitoring tool (like VisualVM).
While Docker containers require Docker to be installed on the host, Shield requires a JVM. In fact, it is possible to embed Shield (like any caplet) directly in the capsule, so the host doesn’t even need the
capsule-shield JAR to be available.
Getting Started with Capsule Shield
Capsule Shield currently uses LXC to create its containers, but that is an implementation detail that may change in the future (in any event, we plan on Shield supporting Solaris/Illumos zones and BSD jails). It does however require that you have a working installation of the LXC tools 1and in the following example, a working Gradle setup is assumed as well.
We’re going to use an unprivileged Linux container running in a new user namespace. In order to create a user namespace, your regular user needs to have been assigned beforehand a range of subordinate user- and group-
ids. A quick way of doing it is the following:
sudo usermod -v 100000-165536 -w 100000-165536
capsule-shield JAR can already be found in your filesystem (else you can
git clone and
gradle install it), then it is just enough to wrap your existing capsule as follows:
java -jar capsule-shield-0.2.0.jar my-capsule.jar my-capsule-arg1 ...
In addition, the Capsule manifest and/or system properties can include some Shield-specific configuration options that are listed in the docs. Defaults should be just fine for most cases.
Shield can also be run as an embedded caplet. In fact, a
quasar-stocks includes additional tasks to build several types of capsules, some of which embed
Let’s run it in a container:
git clone https://github.com/circlespainter/quasar-stocks cd quasar-stocks gradle thinCapsule java -Dcapsule.id=quasarstocks1 -Dcapsule.ip=10.0.3.100 -jar capsule-shield-0.2.0.jar build/libs/quasar-stocks-thin.jar
You’ll see the regular Jetty and application logging on the host, configured through a regular Log4J file, except this time it’s actually being redirected from the unprivileged Linux container. You can easily inspect the application’s MBeans by opening JVisualVM (with JMX plugins installed) on the host and connecting to the local capsule process.
Opening Firefox at http://10.0.3.100:8080/ will let us access the
quasar-stocks webapp running in the container.
Let’s open a new shell and fire up another instance of
quasar-stocks in a new container with a different static IP and a name link to the previous one as
java -Dcapsule.id=quasarstocks2 -Dcapsule.ip=10.0.3.101 -Dcapsule.link.qs1=quasarstocks1 -jar capsule-shield-0.2.jar build/libs/quasar-stocks-thin.jar
Open Firefox at http://10.0.3.101:8080/ws/resolve/qs1 and you’ll see its IP address and reachability status.
Making Things Even Easier
Your organization’s ops people have probably set up your production machines in some uniform way, or at least in a few well-known configurations. They can then package your capsules along with the Shield caplet so that the entire container configuration is entirely contained in the capsule – or even add their custom capsule that automatically selects among a few configurations – so that all that’s required to run a capsule in a fully-configured container is the same as all that’s required to run any capsule, no matter how complex the application it packages:
java -jar your-app.jar
Under the Covers of Capsule Shield: Linux Containers
This Shield release uses LXC as a “container backend”. LXC is a collection of tools implementing lightweight containers using the Linux kernel’s built-in containment features, namely namespaces and cgroups2.
In particular user namespaces allow regular users to have full,
root-like capabilities inside each namespace they create and to perform administrative actions on associated resources like user-owned files and a set of additional
guids that can be mentioned in user mappings. This makes it possible to run a full Linux container as a regular user without extended capabilities and it further limits the reach of malicious interactions with the host environment.
Because of their minimal footprint and strong security, Linux containers – especially the unprivileged kind – are a very attractive option to run applications and services in a practical, secure and efficient sandbox.
verbose capsule logging level (add
-Dcapsule.log=verbose to the command line) will expose the LXC container setup actions performed by the Shield caplet before running your application:
- Checking if the container associated to the application ID is up-to-date and destroying it if it’s not.
- Creating the LXC configuration file.
- Creating the minimal root disk of the container.
The last step includes
chowning the root disk to the container’s
root user, which is slightly tricky because a regular Unix user cannot transfer ownership to other users (and we want the container’s
root to be mapped to an unprivileged
subuid, rather than to our host user, for extra security). So we’ll create a temporary user namespace (in it we’ll have full capabilities), map ourselves to
root, map to a temporary
gidwhat will be the container’s
root user in the container’s user namespace (that is, the first subuid) and finally
chown -R the root disk to it while inside the temporary user namespace 3 4.
Try Shield Today!
Shield is a young caplet (although Capsule is certainly production-ready), and so hiccups and bugs are to be expected, but it is now ready for you to try it. Let us know what you think and how we can improve Capsule and Shield on the Capsule forum/mailing-list.
BONUS: Capsule OSv
A different solution for lightweight application isolation and resource-restriction is OSv. OSv is a microkernel: a single-process POSIX kernel that links with the JVM to form a single bootable image (if you will, it is a kernel-as-a-library). If containers remove the need for a VM by implementing virtualization at the OS level, OSv removes the need for a full-blown OS, and basically runs the JVM directly on any hypervisor. It has small deployable image sizes and boots in under a second.
capsule-osv JAR can already be found in your filesystem (else you can
git clone and
gradle install it), then it is just enough to wrap your existing capsule with
capsule-osv as follows:
java -Dcapsule.log=verbose -jar ~/.m2/repository/co/paralleluniverse/capsule-osv/0.1.0/capsule-osv-0.1.0.jar my-capsule.jar my-capsule-arg1 ...
The first time you run the capsule Capstan will download the OSv root disk image.
In addition, the Capsule manifest and/or system properties can include some OSv-specific configuration options that are listed in the docs and that strictly correspond to Capstan options. The most common things you’ll probably want to do is to choose the networking type and to choose a specific hypervisor.
- An Ubuntu-specific LXC installation guide is available and has some hints about packages that could be needed in other distros. ↩
- There are, as of today, 6 types of namespaces that virtualize different kinds of kernel resources; here’s an excellent series about Linux namespaces. ↩
- This is allowed because the filesystem resources making up the container’s root disk are owned by the user creating the user namespace, and are thus associated with the latter. ↩
- The LXC distribution includes
lxc-usernsexec, a command-line tool that allows to execute other commands in a temporarily created user namespace with mappings. Missing that, the only other option would be calling into native OS libraries, although this is made much easier today by jnr-ff and even more so in future Java releases as part of Project Panama. ↩
- Unfortunately an open OSv issue currently prevents from running Java applications using agents. ↩