Moving Your Application into an Alternate Production Infrastructure

It's Showtime | Production


Applications have lifecycles. At Digital Garage, we are focused on making the development, test and staging part of that lifecycle as easy as possible. We understand that you will likely deploy your production application on infrastructure that you choose specifically for that purpose. This is why we have chosen to base our service on open, broadly adopted standards and technologies.

We would like to make it easy for you to deploy the production version of the applications you build and test on our platform into a production environment of your choice. To that end, I have created this short "how-to" article focused on porting the Linux Containers (Docker Containers) that you build as a natural product of working on Digital Garage from our internal container registry to an external registry such as Docker Hub.

Overview

Virtually everything that the Digital Garage platform does will result in a linux container. Builds, Deployments, Pods are all linux containers running on the Docker Engine and orchestrated (deployed and managed) by Google Kubernetes. This is a very powerful way to build and deploy your application.
The practical implications are that your application is immediately supportive of linux containers and thus deployable to a broad range of infrastructure and cloud provider without modification. In addition, the Digital Garage utilizes the proven and scalable Kubernetes orchestration engine to manage these containers. This means that you are by default building deployment and management templates that can again be used on a broad range of infrastructure and cloud providers without modification. With all of this flexibility, it stands to reason that some developers would be interested in pushing the resultant container image, after it is built, into a separate docker registry that they run outside of Digital Garage.

Note: Because Digital Garage uses standard Kubernetes templates, Docker images and Red Hat Source-to-Image builders, you are always free to port your applications at the source code level. This article is intended to give you an alternative to rebuilding and porting your application from source.

Prerequisites

  • A Github account. If you do not already have a Github account, you can follow this link to sign up for free.
  • A Digital Garage account. If you do not already have a Digital Garage account, you can sign up for free at www.thedigitalgarage.io.
  • Install the Digital Garage CLI. If you have not installed the CLI, you can find instructions here: Installing the Digital Garage CLI
  • A Docker Hub account. If you do not already have a Docker Hub account, you can sign up for free here: Docker Hub

Configuring a private Docker Hub repository.

In this example we will use Docker Hub as an external registry. With Docker Hub, you have the option of configuring your repository as either public or private. If you decide to use a private Docker Hub repository, you must configure a secret in your Digital Garage project that will be used by Deployer and Builder Images to log-in to the private repository.

Note: If you will not be using a private Docker Hub repository you can skip this section.

  • Create a secret to store Docker Hub credentials.

We will supply .docker/config.json file with valid Docker Registry credentials in order to push the output image into a private Docker Registry or pull the builder image from the private Docker Registry that requires authentication. The .docker/config.json file is found in your home directory by default and has the following info (it may be a json file instead of yaml) :

auths:
  https://index.docker.io/v1/:
    auth: "YWRfbGzhcGU6R2labnRib21ifTE="
    email: "user@example.com"

Note: If you have connected to Docker Hub before, you will already have this in your home directory. If you are using a different internal registry you will have the respective registry config here (eg: myregistry.mycompany.io instead of index.docker.io)

  • Add a secret to your project that pulls the auth details from the docker config. In this example, we will name the secret dockerhub. The output of the command shows secret/dockerhub is created.
$ oc secrets new dockerhub ~/.docker/config.json secret/dockerhub
  • Verify the list of secrets by running:
$ oc get secrets
NAME                       TYPE                                  DATA      AGE
builder-dockercfg-3073d    kubernetes.io/dockercfg               1         11m
builder-token-gmhiw        kubernetes.io/service-account-token   3         11m
builder-token-n7ajm        kubernetes.io/service-account-token   3         11m
default-dockercfg-35ywk    kubernetes.io/dockercfg               1         11m
default-token-3zq09        kubernetes.io/service-account-token   3         11m
default-token-6sgod        kubernetes.io/service-account-token   3         11m
deployer-dockercfg-gm2fy   kubernetes.io/dockercfg               1         11m
deployer-token-3and7       kubernetes.io/service-account-token   3         11m
deployer-token-fgblq       kubernetes.io/service-account-token   3         11m
dockerhub                  Opaque                                1         2m

This lists the secret, "dockerhub" that we just created.

Note: Alternately, if you have not logged into DockerHub from your workstation, you can use to create the secret by passing the credentials directly.

$ oc secrets new-dockercfg dockerhub --docker-server=DOCKER_REGISTRY_SERVER --docker-username=DOCKER_USER --docker-password=DOCKER_PASSWORD --docker-email=DOCKER_EMAIL
  • Next, we will add the secret we just created to the Builder service account. Each build in your project runs with using the builder service account. To get a list of service accounts in your project run and it shows the builder service account as one of the service accounts that are automatically created for you when you created a new project.

Note: 'sa' is short form for service account.

$ oc get sa
NAME       SECRETS   AGE
builder    2         34m
default    2         34m
deployer   2         34m
  • We will now edit the builder service account to add the secret.
$ oc edit sa builder

Note: The command above will produce a yaml file for editing. You can edit the file in json by adding -o json to the command:

$ oc edit sa builder -o json
  • Edit the file to add the dockerhub secret to the secrets list as shown below.

Note: With a yaml file, be careful about the indentation.

# Please edit the object below. Lines beginning with a '#' will be ignored,
# and an empty file will abort the edit. If an error occurs while saving this file will be
# reopened with the relevant failures.
#
apiVersion: v1
imagePullSecrets:
- name: builder-dockercfg-3073d
kind: ServiceAccount
metadata:
  creationTimestamp: 2016-08-18T23:34:00Z
  name: builder
  namespace: ext-image-push
  resourceVersion: "636656"
  selfLink: /api/v1/namespaces/ext-image-push/serviceaccounts/builder
  uid: 3db20d64-659c-11e6-9178-fa163e69a197
secrets:
- name: builder-token-gmhiw
- name: builder-dockercfg-3073d
- name: dockerhub

That's it! Now your builder can use the newly created dockerhub secret.

  • Now we will create an example build in our project that can be pushed to Docker Hub. I am using a Dockerfile in my personal Github repository as an example. This repository happens to contain a Source-to-Image builder for the static site generator, Jekyll. Take a moment to look through the repository, if you like. This is a great example of an open, custom Builder Image that we use at Digital Garage.
$ oc new-build https://github.com/johnmccawley/s2i-jekyll-nginx-18-centos7

--> Found Docker image f6350b7 (3 hours old) from Docker Hub for "centos/nginx-18-centos7"

    Nginx 1.8
    ---------
    Platform for running nginx or building nginx-based application

    Tags: builder, nginx, rh-nginx18

    * An image stream will be created as "nginx-18-centos7:latest" that will track the source image
    * A Docker build using source code from https://github.com/johnmccawley/s2i-jekyll-nginx-18-centos7 will be created
      * The resulting image will be pushed to image stream "s2i-jekyll-nginx-18-centos7:latest"
      * Every time "nginx-18-centos7:latest" changes a new build will be triggered

--> Creating resources with label build=s2i-jekyll-nginx-18-centos7 ...
    imagestream "nginx-18-centos7" created
    imagestream "s2i-jekyll-nginx-18-centos7" created
    buildconfig "s2i-jekyll-nginx-18-centos7" created
--> Success
    Build configuration "s2i-jekyll-nginx-18-centos7" created and build triggered.
    Run 'oc logs -f bc/s2i-jekyll-nginx-18-centos7' to stream the build progress.

Kubernetes creates ImageStreams for "nginx-18-centos7" and "s2i-jekyll-nginx-18-centos7" within this project, downloads the the nginx-18 container image from Docker Hub and triggers a build on the Jekyll S2I image.

  • Next, we will list the build configurations to see that we have a build config for the Jekyll Image. The following command lists all the buildconfigs in your project.

Note: 'bc' is the short form for buildconfig.

$ oc get bc

NAME                          TYPE      FROM      LATEST
s2i-jekyll-nginx-18-centos7   Docker    Git       1

In building the Jekyll image a few minutes ago, Kubernetes created an image stream for our "s2i-jekyll-nginx-18-centos7:latest" and automatically pushed the resulting image from our docker build to the Digital Garage internal registry. This is fine for most cases because this image is likely to be used locally for development. However, for this example, we would like to push the resulting image to Docker Hub so we can access it for deployment to other Infrastructure.

  • Edit the BuildConfig to push to the docker registry of your choice (in this case I am pushing to DockerHub).
$ oc edit bc s2i-jekyll-nginx-18-centos7

In the buildconfig find the following section of yaml.

spec:
  output:
    to:
      kind: ImageStreamTag
      name: s2i-jekyll-nginx-18-centos7:latest

You can see that this is telling the builder to push the output to the ImageStream s2i-jekyll-nginx-18-centos7 with ImageStreamTag of s2i-jekyll-nginx-18-centos7:latest

  • Change this to point to your Docker Registry as follows:

Note: You can use the Docker registry of your choice (in which case you will change the docker.io to let us say myregistry.mycompany.io)

spec:
  output:
    to:
      kind: DockerImage
      name: docker.io/my-dockerhub-account/my-s2i-jekyll-nginx-18-centos7:latest
    pushSecret:
      name: dockerhub

Note: We added the pushSecret here to let the builder use that particular secret while pushing into this registry.

  • Now that we are done with the changes to the build configuration, we are ready to re-run the build. As before, the following command will invoke a dockerbuild within Digital Garage. This time however, Kubernetes will push the resultant image to external docker registry, Docker Hub.
$ oc start-build s2i-jekyll-nginx-18-centos7 s2i-jekyll-nginx-18-centos7-2

This started a new build with name s2i-jekyll-nginx-18-centos7-2 and spins up a pod with name time-2-build. The command oc get pods will show you the list of running pods.

Now you can pull your image from Docker Hub and deploy it to your choice of infrastrucrture.

$ docker pull my-dockerhub-account/my-s2i-jekyll-nginx-18-centos7:latest
Trying to pull repository docker.io/my-dockerhub-account/my-s2i-jekyll-nginx-18-centos7 ... latest: Pulling from docker.iomy-dockerhub-account/my-s2i-jekyll-nginx-18-centos7
8ddc19f16526: Pull complete
914f5c514f33: Pull complete
Digest: sha256:644a16ee433086d85797cebb29c0f9ff880d952ef57d55460cc2ff1b0eec742f
Status: Downloaded newer image for docker.iomy-dockerhub-account/my-s2i-jekyll-nginx-18-centos7:latest

Summary

We have seen an example of creating an application image using the build process in Digital Garage and configuring the build to push the resultant image to an external Docker Registry. Enjoy!!