Container infrastructure trends
Optimizing for production workloads

Daniel J Walsh

Consulting Engineer

Twitter: @rhatdan




Would it be as successful if we all called it Adobe PDF?

If the only way to view was Adobe Reader?

If the only way to create was Acrobat?


Would it be as successful if we all called it Red Hat Linux?

Standard Format

Competition makes us stronger

Ubuntu, SUSE, Debian, Fedora, Centos, Alpine, Amazon Linux, Android ...

These distributions help make Red Hat Enterprise Linux better


How do we make Dockah Containers into just Containers?

We need to make containers as generic as PDF

Containers need to be OPEN


What exactly is a container?

TPFKD == The Project Formally Known as docker

Pet Peeve, Do we need huge
Container Runtime Daemons?


What Does OpenShift/Kubernetes need to run a container?

Standard container image format

Pull (push?) from a container registry

Explode image local storage

Execute container image

Container Management API (Optionally)

Standard container image format

Open Container Initiative (OCI)

OCI Image Format

Release candidate 6 is now available

Allows us to store standard images at container registries

Most important features in containers

Pull (push?) from a container registry

How do I get an image from a container registry to my host?


Greek for remote viewing

Used by atomic CLI
View container image json data on registry

Added ability to pull and push images from registry

Worked with CoreOS splitting out go library from CLI.

Explode image local storage

Where do I store/explode container image after pulling?

atomic mount fedora /mnt

Allow other tools to work with storage besides container runtime daemons

Originally used TPFKD graphdriver code independent library/CLI

Execute container

Open Container Initiative (OCI)

OCI Runtime Specification

runc - default implementation

As of TPFKD 1.11 - runc is default back end

runv and clear containers alternatives

Alternatives to adobe reader

Now let's

Simple Image Signing Goals

Allow users to PGP sign images in look-aside cache

Multiple people/companies can sign same image

Signatures can be stored in Atomic/OpenShift Registry

Signatures can be stored in any web server

Allow user to "sign" content from

Build policy/rules engine to control which
images/registries are trusted

atomic CLI, skopeo & TPFKD
prevented from pulling untrusted containers

Signing Images

Managing Trust

System Containers

On atomic host, software shipped as container image

Kubernetes requires etcd, flanneld

etcd, flanneld start before TPFKD daemon to setup network

These containers can be run with read/only images

Upstream TPFKD has problems with container priority

systemd doesn't

System Containers

atomic command installs system containers

Uses skopeo to pull image from registry

Uses ostree to store image layers on disk

Creates systemd unit file and uses runc to run container

Optionally use runc to wrap processes in a container

Run as a chroot?

Use system containers to install content on host?

		  atomic install --system etcd
		  systemctl enable etcd
		  systemctl start etcd
		  atomic install --system flannel
		  systemctl enable flannel
		  systemctl start flannel

Systemd makes sure etcd starts before flannel

Systemd makes sure both start before the container runtime

upstream TPFKD
runs as a
System Container

		  atomic install --system system-TPFKD
		  systemctl enable system-TPFKD
		  systemctl start system-TPFKD

Standalone Containers

Packaging standard RHEL content as containers

Daemons run in container standard ports/volumes prepackaged for standard use cases

Package apps as OCI Image Format rather then RPM

Mariadb, PostgreSQL, Apache ...

PHP, Python, Ruby, glibc ...

Potential release cadence different the the Host OS
(Software Collections)

Improved Storage

Read Only Container images

Read/Only images offer better security

In production, most images should be immutable

Get rid of COW file systems when not necessary

Improved Storage

Lets talk about container image size.

Why do we care?

Shared file systems.
Run same images on multiple servers

Instantaneous Updates, containers running same code

Container Image
Development Tools

Why do we care how someone builds an OCI Image?

We will still have TPFKD build with Dockerfile

Container Image
Development Tools

With container storage.

Alternatives to Dockerfile


OpenShift S2I

Build container images without container runtime daemon



CID=$(buildah from fedora)

buildah mount $CID /mnt

buildah run $CID dnf install httpd

make install DESTDIR=/mnt

dnf install --installroot=/mnt httpd

buildah commit $CID --entrypoint=/usr/bin/

Working to get Ansible-container and OpenShift to use buildah



buildah build-using-dockerfile -f Dockerfile .

buildah bud -f Dockerfile .

Management API

kubernetes wants to support more then one container runtime

Introducing the CRI-O effort

Implements Kubelet Container Runtime Interface
Kubernetes server interface launching/running pods

Fully open with contributors from Red Hat, SuSe, Intel,, IBM, and many others

Intel added support for CRI-O running KVM Based containers

cri-o package now available in Fedora 26 and Rawhide

First CRI based daemon to pass full kubernetes CRI test suite as of last Friday

CRI-O Next Steps

Looking to move it out of incubator to kubernetes project or CNCF

Pass OpenShift Test Suite

Hope to move OpenShift & Kubernetes to CRI-O by default

kpod - management tools for administrating CRI-O
storage and pods

OpenShift tells Kubernetes to execute pod

Kubernetes communicates with cri-o

cri-o pull image using container/image

cri-o stores image on disk using container/storage

cri-o runs container/pod using runc

Standards based container runtime alternative to TPFKD/rkt


Breaking up container runtime daemons into core functionality:

Pulling/Pushing images from a container registry

Storing these images on to container storage

Running containers using standard specification

Innovate new and interesting ways of using