Jekyll2023-04-29T10:05:00-03:00http://mohashi.io/feed.xmlDevModeExchange of developer insights - by Marcelo OhashiMarcelo OhashiHow to build cloud-native pipelines2020-05-26T00:00:00-03:002020-05-26T00:00:00-03:00http://mohashi.io/blog/kubernetes/knative/how-to-build-cloud-native-pipeline<p>In this article I am going to show how to build a <em>Cloud-Native</em> pipeline using <em>Openshift Pipelines</em>. Pipelines have been around for a long time already. How can we still improve it? Do we need to? During the last few years we have been creating a massive amount of them by using technologies that are still “not quite there” in terms of what the Dev and Ops communities are looking for. Tools like <em>Jenkins</em>, <em>Gitlab CI</em>, and others have done a good job (to be fair), but they seem to be more like a general purpose tool for cloud technologies than something built for it. The way we have been integrating these tools to Kubernetes was either to provision hosts dedicated to run our pipeline jobs or to create what we may call a build pod to be provisioned inside the same Kubernetes infrastructure to execute our pipeline into it by using scripts and plugins that interacted with Kubernetes API in a very limited way, for example, using <em>Jenkins Pipeline DSL</em>. And in many cases we were still doing what we did with that old (and dusty) builder machine that had all dev and ops stuff installed there to be able to execute those automated scripts. Everything running in the same VM or image.</p>
<p>On the other side, we are seeing the community evolving their platforms, products, and tools leveraging a more Kubernetes native approach using <em>Custom Resources</em> and <em>Operators</em>. And this new wave comes with a lot of advantages, such as, perform a lot more “advanced” use of the Kubernetes API and reuse a whole ecosystem that was built around the Kubernetes API and provision the image that we want in a so granular approach that will make things much less complex to maintain. By the end of this article, you will understand the concepts and how to build your pipelines in a Cloud-Native way.</p>
<p class="notice"><strong>Note</strong>
There is nothing really wrong with your <em>Jenkins</em> or other tools that can help you deliver pipelines. If you have one, you are in a good shape. The discusion here is that these tools can be really hungry for resources once it get’s a lot of load, they are hard to maintain, and are difficult to scale. So, the approach here is to have a controller which has a very small footprint that uses Kubernetes enviornment to scale as desired once it is required and have as many containers as desired to complete the smallest step.</p>
<p>The concept of what is a pipeline can be very opinionated, so the pipeline I considered to build in this article is one that follows some best practices, but also is not too complex that would make it go into a direction that can not be reused in other scenarios. So, we are going to create a pipeline that has a good foundation and can also be extended in other much more complex scenarios.</p>
<p><a href="https://www.openshift.com/learn/topics/pipelines" target="_blank">Openshift Pipelines</a> is a CI/CD solution based on Tekton and is tigthly integrated to Red Hat Openshift. The web console has some features that helps a team in many different such as dashboards that, among other things, create an easy-to-follow way to observe what is happening in your pipelines.</p>
<p><a href="https://tekton.dev/" target="_blank">Tekton</a> is an opensource project which provides a Kubernetes native CI/CD (Continuous Integration and Delivery/Deployment) solution. It allows developers to build, test, and deploy their applications into any Kubernetes cluster. This project has a vibrant community that includes companies like: Google, Red Hat, and many others. The current project state is in beta with a lot of features to come.</p>
<h2 id="tekton-distilled">Tekton distilled</h2>
<p>Tekton is built upon four main components</p>
<ul>
<li><em>Pipeline CRDs</em>: It’s comprised of all the following <code class="language-plaintext highlighter-rouge">CRDs.</code> These <code class="language-plaintext highlighter-rouge">CRDs</code> composed of several other attributes that are used to implement a pipeline:
<ul>
<li><code class="language-plaintext highlighter-rouge">Task</code>: is used to implement atomic activities to perform as a single unit with a set of <em>steps</em>. For example, to build an application you may have to prepare the environment before executing the source code compilation. So, each activity that is part of a single <em>task</em> is called a <em>step</em>. Every <em>step</em> is executed in the same order that it is defined in the task. Each <em>step</em> can have it’s own image, so inside Kubernetes these <em>steps</em> become a container inside the same pod. A <em>task</em> can also define some external requirements such as storage, inputs and outputs.</li>
<li><code class="language-plaintext highlighter-rouge">TaskRun</code>: is the runtime definition of a <em>task</em>. The <em>TaskRun</em> allows you to provide the actual persistent storage for the <em>task</em>, what are its inputs and outputs, if there is any. When we create one in the cluster, the operator starts a pod definition for the referred <em>task</em> and execute it by attaching all external attributes defined in the <em>TaskRun</em>. It’s possible to run tasks individually.</li>
<li><code class="language-plaintext highlighter-rouge">Pipeline</code>: is a wrapper for several <em>tasks</em> that can be used to deploy an aplication, configure something in the cluster, or whatever is necessary. Each <em>task</em> defines an optional predecessor to create the chain. There are some additional attributes that can be used to make it more resilient to errors and conditions of when the <em>task</em> needs to run. Take a look at <a href="https://tekton.dev/docs/pipelines/pipelines/" target="_blank">Tekton doc</a></li>
<li><code class="language-plaintext highlighter-rouge">PipelineRun</code>: has a similar concept to what a <em>TaskRun</em> is for a <em>Task</em> but in this case for a <em>Pipeline</em>. It is required in order to run a <em>Pipeline</em>. So, it defines what would be the real external resources that can be used in a pipeline.</li>
<li><code class="language-plaintext highlighter-rouge">PipelineResource</code>: A <em>PipelineResource</em> is a way to reuse some internal features that Tecton provides without having to create a task for it. Tekton still have only a few of them and in many situations they have counter-parties as <em>tasks</em>. The can be used in a Task or a Pipeline as inputs and outputs. Some examples are:
<ul>
<li><em>Git Resource</em>: This one has the capability of checkout the source code from a git repo. With minor issues see <a href="#Conclusion">Conclusion</a>;</li>
<li><em>Image Resource</em>: This one has the capability of push an image to a repository;</li>
<li><em>Storate Resource</em>: This one has the capability of creating storages in general, but mostly inside GCS;</li>
<li><em>Cloud Event Resource</em>: This one has the capability of sending events to whatever is being listening to. Just like any other webhook that we configure to notigy other applications;</li>
<li><em>CLuster Resource</em>: This one creates resources in an external cluster;</li>
</ul>
</li>
</ul>
</li>
<li><em>Controllers</em>: are the main orchestrators that are responsible to monitor and react from newly created CRs to execute tasks and update Kubernetes API. Here follows more details regarding what exactly each one does:
<ul>
<li><code class="language-plaintext highlighter-rouge">tekton-pipelines-controller</code>: is a Tekton - K8S Controller</li>
<li><code class="language-plaintext highlighter-rouge">tekton-pipelines-webhook</code>: is a K8S Dynamic Admission Webhook to validate/mutate Tekton CRs</li>
<li><code class="language-plaintext highlighter-rouge">tekton-triggers-controller</code>: is a Generic/GitHub WebHook - K8S Controller. It’s the controller that manages tektoncd/triggers structures and create services for each EventListener (that can be used as GitHub webhook if published, using ingress, openshift routes, or something else)</li>
<li><code class="language-plaintext highlighter-rouge">tekton-triggers-webhook</code>: is a K8S Dynamic Admission Webhook to validate/mutate Tekton Triggers CRs</li>
</ul>
</li>
<li><em>Event Listerners</em>: is used to expose and endpoint to enable integrations with git repositories and receive webhooks requests. It is very flexible with mapping JSON fields, and binding and creating Tekton’s CRs. Using <code class="language-plaintext highlighter-rouge">CRC</code> it is commonly exposed via openshift routes.</li>
<li><em>CLI Tool</em>: is the famous <code class="language-plaintext highlighter-rouge">tkn</code> command line that we use to see logs, list, create, describe all kinds of Tekton’s CRs that we have created throughout the creation of a pipeline. Besides, it is capable of retrieve all logs from all tasks belonging to the same pipeline in a single-shot.</li>
</ul>
<h2 id="architecture-overview">Architecture overview</h2>
<p>The following architecture explains how each components interact between them:</p>
<!-- markdownlint-disable MD033 -->
<figure>
<a href="/assets/images/how-to-build-cloud-native-pipeline/tekton-architecture.png"><img src="/assets/images/how-to-build-cloud-native-pipeline/tekton-architecture.png" /></a>
</figure>
<!-- markdownlint-enable MD033 -->
<p>The first step is to create our definitions or CRs that have what every <em>Step</em>, <em>Task</em>, and the <em>Pipeline</em> is composed of everything that is necessary to get that application component to our environment. After that we have what we call runtime PipelineRuns and TaskRuns components that define what are the actual git repository or where the pipeline should store their components in the <strong>runtime</strong>.</p>
<p>So, once we have all these components properly configured in the cluster we need something that creates the actuall pods and our containers to ultimately start executing things and that’s when the <em>Controllers</em> come into the party. The <em>Controllers</em> tries to guarantee that for each <em>PipelineRun</em> there is a set of pods created for each task and also making sure that the desired chain order is being followed.</p>
<p>And finally, we have what we call the event listerner pod that is responsible to receive any payload that comes from another application. It exposes a service to the world by using a Kubernetes ingress or an Openshift route. There are some CRs responsible to extract, map, and create the templates necessary to create the CRs once it receives the call from any of the git repositories of other applications.</p>
<h2 id="pipeline-overview">Pipeline overview</h2>
<p>This pipeline was designed to perform builds using a Quarkus application, but that does not prevent you from building following the same idea any other languages.</p>
<p>The pipeline that we are going to build has the following steps:</p>
<ul>
<li><em style="text-decoration: underline">Git-Clone</em> - This task is used to clone a repository.</li>
<li><em style="text-decoration: underline">Build</em> - This task is used to build the source code.
In these following links you can find tasks for other languages.
<ul>
<li><a href="https://github.com/openshift/pipelines-catalog" target="_blank">Openshift Catalog</a></li>
<li><a href="https://github.com/tektoncd/catalog" target="_blank">Tekton Catalog</a></li>
<li><a href="https://github.com/open-toolchain/tekton-catalog" target="_blank">Open Toolchain Catalog</a></li>
</ul>
</li>
<li><em style="text-decoration: underline">Tests</em> - This task is used to run the unit tests.</li>
<li><em style="text-decoration: underline">Static-Code-Analysis</em> - This task is used to perform the code analysis using <a href="https://docs.sonarqube.org/7.9/" target="_blank">SonaQube</a>.</li>
<li><em style="text-decoration: underline">Validate Quality Gate</em> - This task is used to validate the quality gate results by quering its API and ensure that the code has no major issues.</li>
<li><em style="text-decoration: underline">Build-Package</em> - This task is used to create the unified package for the S2I build.</li>
<li>
<p><em style="text-decoration: underline">Build-Image</em> - This task is used to run the S2I build using the binary approach to create a the container image and store it directly in the internal repository.</p>
<p class="notice"><strong>Note</strong>
We could use here the current builders implementation that are available in the Tekton’s and Openshift’s catalogs, but that would require us to provision priviledged containers to run the image builds. Which if in your case this something unecessary, you can change that and go back and use the regular builders. But, I recommend you to not choose this path if you can the alternative to run the s2i builds inside your cluster that does not require you to attenuate the security of your cluster. For more information regarding s2i builds see <a href="https://docs.openshift.com/container-platform/4.4/builds/build-strategies.html#build-strategy-s2i_build-strategies" target="_blank">this documentation</a>.</p>
</li>
<li><em style="text-decoration: underline">Setup-App</em> - This task is used to provision the app’s Kubernetes configurations in the cluster without any triggers.</li>
<li><em style="text-decoration: underline">Deploy</em> - This task is used to execute the rollout using the latest version of the deployment.</li>
</ul>
<p class="notice"><strong>NOTE</strong>
One thing that it’s going to be used on other versions of this tutorial is to have a way to validate the deployment success.</p>
<p>These tasks are chained as in the following diagram:</p>
<!-- markdownlint-disable MD033 -->
<figure>
<a href="/assets/images/how-to-build-cloud-native-pipeline/tekton-pipeline.png"><img src="/assets/images/how-to-build-cloud-native-pipeline/tekton-pipeline.png" /></a>
</figure>
<!-- markdownlint-enable MD033 -->
<h2 id="installing-and-configuring-tekton">Installing and configuring Tekton</h2>
<p>First, we will need the <code class="language-plaintext highlighter-rouge">tkn</code> client to perform some necessary tasks during the article. You can install it by following this instructions <a href="https://github.com/tektoncd/cli" target="_blank">here</a>.</p>
<p>Verify if it is installed correctly:</p>
<div class="language-shell highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nv">$ </span>tkn version
Client version: 0.9.0
Pipeline version: v0.11.3
</code></pre></div></div>
<p>In order to deploy and test our pipeline setup in a Kubernetes environment we will use the <em>Code Ready Containers</em> (CRC) environment. Code Ready Containers will spin up a cluster, which is an all-in-one cluster or both master and worker node, in your local machine. So, if you don’t have it already there you can follow the installation process in this link <a href="https://code-ready.github.io/crc/#installation_gsg" target="_blank">here</a>.</p>
<p>After having the CRC up and running, let’s install the <em>Openshift Pipelines Operator</em> witch is based on version <code class="language-plaintext highlighter-rouge">v0.11.2</code> of Tekton. The process that we will go here is using the <code class="language-plaintext highlighter-rouge">kubectl</code> and <code class="language-plaintext highlighter-rouge">oc</code> command-line interface (CLI).</p>
<div class="language-shell highlighter-rouge"><div class="highlight"><pre class="highlight"><code>kubectl get packagemanifests/openshift-pipelines-operator <span class="se">\</span>
<span class="nt">-n</span> openshift-marketplace <span class="nt">-o</span> yaml | yq r - <span class="se">\</span>
<span class="s2">"status.channels[*].name"</span> <span class="nt">--collect</span>
</code></pre></div></div>
<p>The result will be something like:</p>
<div class="language-shell highlighter-rouge"><div class="highlight"><pre class="highlight"><code>- canary
- dev-preview
</code></pre></div></div>
<p>Subscribe to the <code class="language-plaintext highlighter-rouge">canary</code> channel of <code class="language-plaintext highlighter-rouge">openshift-pipelines-operator</code>. This is going to create the ClusterServiceVersion of the version <code class="language-plaintext highlighter-rouge">0.11.2</code>.</p>
<div class="language-shell highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nv">$ </span><span class="nb">cat</span> <span class="o"><<</span> <span class="no">EOF</span><span class="sh"> | kubectl apply -f -
apiVersion: operators.coreos.com/v1alpha1
kind: Subscription
metadata:
name: openshift-pipelines-operator
namespace: openshift-operators
spec:
channel: canary
name: openshift-pipelines-operator
source: community-operators
sourceNamespace: openshift-marketplace
</span><span class="no">EOF
</span></code></pre></div></div>
<p>The result will be something like:</p>
<div class="language-shell highlighter-rouge"><div class="highlight"><pre class="highlight"><code>subscription.operators.coreos.com/openshift-pipelines-operator created
</code></pre></div></div>
<p>And after a few minutes you will see a new namespace called <code class="language-plaintext highlighter-rouge">openshift-pipelines</code> in the list of existing namespaces which is where the tekton operators will be running.</p>
<div class="language-shell highlighter-rouge"><div class="highlight"><pre class="highlight"><code>kubectl get pods <span class="nt">-n</span> openshift-pipelines
</code></pre></div></div>
<p>The result will be something like:</p>
<div class="language-shell highlighter-rouge"><div class="highlight"><pre class="highlight"><code>NAME READY STATUS RESTARTS AGE
tekton-pipelines-controller-b6856f8cb-27w8k 1/1 Running 0 3d21h
tekton-pipelines-webhook-6d8b4c5644-4qf9w 1/1 Running 0 3d21h
tekton-triggers-controller-75c764b8b8-jmwj4 1/1 Running 0 3d21h
tekton-triggers-webhook-5c8ffbc99d-vslv4 1/1 Running 0 3d21h
</code></pre></div></div>
<h2 id="source-code">Source Code</h2>
<p>All source codes that we are going to use in this article are stored in these two repositories:</p>
<ul>
<li><em>#1 Repo</em> is the application located on <a href="https://github.com/mgo-dev-mode/basic-quarkus-app" target="_blank">github</a>.</li>
<li><em>#2 Repo</em> is the CI/CD and the pipeline files located on <a href="https://github.com/mgohashi/deepdive-tekton" target="_blank">github</a>.</li>
</ul>
<h2 id="cicd-tools">CI/CD Tools</h2>
<p>Before we start installing the pipeline we need to install some tools that will support our pipeline execution. These tools will do some complementary work there in order to validate the application quality and speed up the building process before deploying the application. And these tools are:</p>
<ul>
<li><strong>SonarQube Community</strong>: It will execute the validations on our source code according to our rules.</li>
<li><strong>Sonatype Nexus Community</strong>: It will cache our dependencies, so we won’t need to download them from their original repositories everytime we start a build.</li>
</ul>
<p>Now let’s install them.</p>
<h3 id="1st-step---clone-de-repository">1st Step - Clone de repository</h3>
<p>Clone the <em>#2 Repo</em> to your local machine in the folder that we will call <code class="language-plaintext highlighter-rouge">[PIPELINE_HOME]</code>.</p>
<div class="language-shell highlighter-rouge"><div class="highlight"><pre class="highlight"><code>git clone https://github.com/mgohashi/deepdive-tekton.git
</code></pre></div></div>
<p>There will be many files and folders there, but you are only interested in these here:</p>
<div class="language-console highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="go">[PIPELINE_HOME]
</span><span class="c">...
</span><span class="go">├── 12-complex-pipeline
│ ├── 0-operators
│ │ └── subscriptions.yml
│ ├── 1-cicd-tools
│ │ ├── 0-namespace.yml
│ │ ├── 1-nexus.yaml
│ │ └── 2-sonaqube.yml
│ ├── 2-pipeline
│ │ ├── 0-tasks.yml
│ │ ├── 1-complex-pipeline.yml
│ │ └── quality-gate-result.groovy
│ ├── 3-trigger
│ │ ├── 4-complex-trigger-template.yml
│ │ ├── 5-complex-eventlistener.yml
│ │ ├── 6-complex-trigger-binding.yml
│ │ └── 7-route.yml
│ ├── cleanup.sh
│ ├── gitpush.json
│ ├── simulate-webhook.sh
│ └── update.sh
</span><span class="c">...
</span><span class="go">12 directories, 43 files
</span></code></pre></div></div>
<p class="notice"><strong>NOTE</strong>
Now make sure you execute the following the commands from inside your copy of the <code class="language-plaintext highlighter-rouge">[PIPELINE_HOME]/12-complex-pipeline</code> folder.</p>
<h3 id="2nd-step---install-the-nexus-operator">2nd Step - Install the Nexus Operator</h3>
<p>To install the operator we need to certify that we have the subscription for it.</p>
<p>The yaml file that is in the <code class="language-plaintext highlighter-rouge">0-operators</code> folder is already with the right channel and configuration that we are looking for, so you can just apply it to your cluster using the following command:</p>
<div class="language-shell highlighter-rouge"><div class="highlight"><pre class="highlight"><code>oc apply <span class="nt">-f</span> 0-operators
</code></pre></div></div>
<p class="notice"><strong>NOTE</strong> We used <code class="language-plaintext highlighter-rouge">oc</code> because we have some yamls files that has some openshift only CRs otherwise we would be totally able to do it with our friend <code class="language-plaintext highlighter-rouge">kubectl</code>.</p>
<p>After executing this command you should see another <code class="language-plaintext highlighter-rouge">ClusterServiceVersion</code> being provisioned in the <code class="language-plaintext highlighter-rouge">openshift-operators</code> namespace.</p>
<div class="language-shell highlighter-rouge"><div class="highlight"><pre class="highlight"><code>kubectl get clusterserviceversion
</code></pre></div></div>
<p>The result will be something like:</p>
<div class="language-shell highlighter-rouge"><div class="highlight"><pre class="highlight"><code>NAME DISPLAY VERSION REPLACES PHASE
nxrm-operator-certified.v3.23.0-1 Nexus Repository Operator 3.23.0-1 Succeeded
openshift-pipelines-operator.v0.11.2 OpenShift Pipelines Operator 0.11.2 Succeeded
</code></pre></div></div>
<h3 id="3rd-step---install-the-cicd-tools">3rd Step - Install the CI/CD tools</h3>
<p>So, now we should install the CI/CD tools and for this we are going to create a namespace called <code class="language-plaintext highlighter-rouge">cicd-tools</code> and install the above-mentioned tools there.</p>
<div class="language-shell highlighter-rouge"><div class="highlight"><pre class="highlight"><code>oc apply <span class="nt">-f</span> 1-cicd-tools
</code></pre></div></div>
<p class="notice"><strong>NOTE</strong> Likewise the previous command that we used <code class="language-plaintext highlighter-rouge">oc</code> we needed to do this because this yaml files has some openshift only CRs otherwise we would be totally able to do it with our friend <code class="language-plaintext highlighter-rouge">kubectl</code>.</p>
<p>You should see that the apps are starting in the namespace <code class="language-plaintext highlighter-rouge">cicd-tools</code>.</p>
<div class="language-shell highlighter-rouge"><div class="highlight"><pre class="highlight"><code>kubectl get pods <span class="nt">-n</span> cicd-tools
</code></pre></div></div>
<p>The result will be something like:</p>
<div class="language-shell highlighter-rouge"><div class="highlight"><pre class="highlight"><code>NAME READY STATUS RESTARTS AGE
my-nexusrepo-sonatype-nexus-69ccdd4d44-zsdbw 1/1 Running 0 4d1h
my-sonarqube-1-7bcmk 1/1 Running 0 19m
my-sonarqube-1-deploy 0/1 Completed 0 19m
postgresql-my-sonarqube-1-deploy 0/1 Completed 0 19m
postgresql-my-sonarqube-1-q7258 1/1 Running 0 19m
</code></pre></div></div>
<p>And to check the ingress endpoint to access these tools run the following command:</p>
<p>For Nexus</p>
<div class="language-shell highlighter-rouge"><div class="highlight"><pre class="highlight"><code>kubectl get route/my-nexus <span class="nt">-o</span> <span class="nv">jsonpath</span><span class="o">=</span><span class="s1">'http://{$.spec.host}'</span>
</code></pre></div></div>
<p>And for SonarQube</p>
<div class="language-shell highlighter-rouge"><div class="highlight"><pre class="highlight"><code>kubectl get route/my-sonarqube <span class="nt">-o</span> <span class="nv">jsonpath</span><span class="o">=</span><span class="s1">'http://{$.spec.host}'</span>
</code></pre></div></div>
<p>The results will be something like:</p>
<div class="language-shell highlighter-rouge"><div class="highlight"><pre class="highlight"><code>http://my-nexus-cicd-tools.<span class="o">{</span>FQDN<span class="o">}</span>
http://my-sonarqube-cicd-tools.<span class="o">{</span>FQDN<span class="o">}</span>
</code></pre></div></div>
<ul>
<li><code class="language-plaintext highlighter-rouge">{FQDN}</code> can be whatever <code class="language-plaintext highlighter-rouge">CRC</code> has configured for you. For example: <code class="language-plaintext highlighter-rouge">apps-crc.testing</code>.</li>
</ul>
<h3 id="4th-step---tools-configuration">4th Step - Tools configuration</h3>
<p>Now that you have properly installed tools, we can configure them. Since this is not an article about the demonstration of how to configure these tools, I will just paste here the expected results for some activities.</p>
<p>The default user and password for each tool will be:</p>
<table style="display: inline-table;">
<thead>
<tr>
<th>Tool</th>
<th>User / Password</th>
</tr>
</thead>
<tbody>
<tr>
<td>Nexus Community</td>
<td>admin / admin123</td>
</tr>
</tbody>
<tbody>
<tr>
<td>SonarQube Community</td>
<td>admin / admin</td>
</tr>
</tbody>
</table>
<p>There’s not so much we can do to configure here. But, the tools can have many different analysers or repositories private or not, that would be required to be there. But, to run this demonstration the default is sufficient.</p>
<h2 id="pipeline-environment">Pipeline environment</h2>
<p>The pipeline we are going to create has a few files that are worth detailing a bit more:</p>
<h3 id="considerations">Considerations</h3>
<p><em>Tasks</em> - As we already mentioned before <code class="language-plaintext highlighter-rouge">tasks</code> are there to be used as a reusable code. These specific <code class="language-plaintext highlighter-rouge">tasks</code> are in some situations either just an extended version of others from the catalogs that I either had to customize to make then more flexible or to build one from scratch due to the very specialized situation of mine, such as, create a special package for the <a href="quarkus.io" target="_blank">quarkus app</a>.</p>
<ul>
<li><code class="language-plaintext highlighter-rouge">git-clone</code>: Generic task to clone the parameterized repository. You can check the list of parameters <a href="https://github.com/mgohashi/deepdive-tekton/blob/6f20a3ee621b26e20c874a469597c11789082dbb/12-complex-pipeline/2-pipeline/0-tasks.yml#L9-L51" target="_blank">here</a>;</li>
<li><code class="language-plaintext highlighter-rouge">maven</code>: Generic task to build maven projects. It is already taking a intermediaty <em>Nexus Community</em> mirror to download the artifacts. You can check the list of parameters <a href="https://github.com/mgohashi/deepdive-tekton/blob/6f20a3ee621b26e20c874a469597c11789082dbb/12-complex-pipeline/2-pipeline/0-tasks.yml#L107-L164" target="_blank">here</a>;</li>
<li><code class="language-plaintext highlighter-rouge">openshift-client</code>: A task to execute some commands to the Openshift API using the Openshift CLI or <code class="language-plaintext highlighter-rouge">oc</code>. You can check the list of parameters <a href="https://github.com/mgohashi/deepdive-tekton/blob/6f20a3ee621b26e20c874a469597c11789082dbb/12-complex-pipeline/2-pipeline/0-tasks.yml#L281-L290" target="_blank">here</a>;</li>
<li><code class="language-plaintext highlighter-rouge">tar-quarkus</code>: Generic task to build a single artifact for quarkus applications due to the fact that quarkus is not built using a single jar file, in other works, it does not generate a <code class="language-plaintext highlighter-rouge">uberjar</code> or a <code class="language-plaintext highlighter-rouge">fatjar</code> if you like;</li>
<li><code class="language-plaintext highlighter-rouge">validate-quality-gate</code>: Generic task to validate the code scan into against the <em>SonarQube Community</em>. This task shows how easy it is to create your own task and do whatever is necessary. I used a groovy image to do what I needed. You can check the list of parameters <a href="https://github.com/mgohashi/deepdive-tekton/blob/6f20a3ee621b26e20c874a469597c11789082dbb/12-complex-pipeline/2-pipeline/0-tasks.yml#L326-L337" target="_blank">here</a>;</li>
</ul>
<p>The structure of a task is as follows. For the sake of simplicity I have removed some less important information, but you can find it in the repository in this <a href="https://raw.githubusercontent.com/mgohashi/deepdive-tekton/master/12-complex-pipeline/2-pipeline/0-tasks.yml" target="_blank">link</a>:</p>
<div class="language-yaml highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="na">apiVersion</span><span class="pi">:</span> <span class="s">tekton.dev/v1beta1</span>
<span class="na">kind</span><span class="pi">:</span> <span class="s">Task</span>
<span class="na">metadata</span><span class="pi">:</span>
<span class="na">name</span><span class="pi">:</span> <span class="s">maven</span>
<span class="na">spec</span><span class="pi">:</span>
<span class="na">params</span><span class="pi">:</span>
<span class="s">...</span>
<span class="na">steps</span><span class="pi">:</span>
<span class="pi">-</span> <span class="na">image</span><span class="pi">:</span> <span class="s">registry.access.redhat.com/ubi8/ubi-minimal:latest</span>
<span class="na">name</span><span class="pi">:</span> <span class="s">mvn-settings</span>
<span class="na">resources</span><span class="pi">:</span> <span class="pi">{}</span>
<span class="na">script</span><span class="pi">:</span> <span class="pi">|</span>
<span class="s">#!/usr/bin/env bash</span>
<span class="s">[[ -f $(workspaces.maven-settings.path)/settings.xml ]] && \</span>
<span class="s">echo 'using existing $(workspaces.maven-settings.path)/settings.xml' && \</span>
<span class="s">cat $(workspaces.maven-settings.path)/settings.xml && exit 0</span>
<span class="s">...</span>
<span class="s">[[ -f $(workspaces.maven-settings.path)/settings.xml ]] && cat $(workspaces.maven-settings.path)/settings.xml</span>
<span class="s">[[ -f $(workspaces.maven-settings.path)/settings.xml ]] || echo skipping settings</span>
<span class="pi">-</span> <span class="na">args</span><span class="pi">:</span>
<span class="pi">-</span> <span class="s">-s</span>
<span class="pi">-</span> <span class="s">$(workspaces.maven-settings.path)/settings.xml</span>
<span class="pi">-</span> <span class="s">$(params.GOALS)</span>
<span class="na">command</span><span class="pi">:</span>
<span class="pi">-</span> <span class="s">/usr/bin/mvn</span>
<span class="na">image</span><span class="pi">:</span> <span class="s">gcr.io/cloud-builders/mvn</span>
<span class="na">name</span><span class="pi">:</span> <span class="s">mvn-goals</span>
<span class="na">resources</span><span class="pi">:</span> <span class="pi">{}</span>
<span class="na">workingDir</span><span class="pi">:</span> <span class="s">$(workspaces.source.path)</span>
<span class="na">workspaces</span><span class="pi">:</span>
<span class="pi">-</span> <span class="na">name</span><span class="pi">:</span> <span class="s">source</span>
<span class="pi">-</span> <span class="na">name</span><span class="pi">:</span> <span class="s">maven-settings</span>
<span class="pi">-</span> <span class="na">name</span><span class="pi">:</span> <span class="s">local-repo</span>
</code></pre></div></div>
<p>Let’s break it down to its main sections:</p>
<ul>
<li><code class="language-plaintext highlighter-rouge">workspace</code>: This is just a definition that we will need a persistent storage to put our files.</li>
<li><code class="language-plaintext highlighter-rouge">params</code>: This is the section that we add our parameters. We can have here some default values, so don’t worry if you define a lot to make it flexible enough for the use in a pipeline.</li>
<li><code class="language-plaintext highlighter-rouge">steps</code>: This is the section that we define what the step is going to actually do. It has some important attributes:
<ul>
<li><code class="language-plaintext highlighter-rouge">name</code>: The name of the step.</li>
<li><code class="language-plaintext highlighter-rouge">image</code>: The container image. Yes you can define your own container mage has what you need to do. You will notice that the <code class="language-plaintext highlighter-rouge">validate-quality-gate</code> task has a groovy image <code class="language-plaintext highlighter-rouge">quay.io/dockerlibrary/groovy</code> in its second step.</li>
<li><code class="language-plaintext highlighter-rouge">script</code>: A script that is executed inside the container.</li>
<li><code class="language-plaintext highlighter-rouge">command</code>: Can not be used in conjunection with script field. A command that is expected to run in the container.</li>
<li><code class="language-plaintext highlighter-rouge">args</code>: Can only used in conjunction with the <code class="language-plaintext highlighter-rouge">command</code> attribute. It defines a list of arguments to be appended to the command.</li>
</ul>
</li>
<li><code class="language-plaintext highlighter-rouge">results</code>: Some tasks define this attribute. This is used to store some important information in the container termination log. Up to this moment it has 2048 bytes or 80 lines as a defined limit, whichever is smaller.</li>
<li><code class="language-plaintext highlighter-rouge">resources</code>: Can be used to do perform some builtin tasks. Such as, clone a git repository or export the image to a repository. But, after reviewing many of the advantages and disadivantages, that will be described in the conclusion of our article, I chose not to use them.</li>
</ul>
<p class="notice"><strong>NOTE</strong>
There are a lot more attributes to check it out at tekton’s documentation in the <a href="https://tekton.dev/docs/pipelines/tasks/#configuring-a-task">link</a>.</p>
<p>Now let’s delve into a pipeline structure. Just like the other above for the sake of simplicity I have removed some less important information. But, you can find it in the repository in this <a href="https://raw.githubusercontent.com/mgohashi/deepdive-tekton/master/12-complex-pipeline/2-pipeline/1-complex-pipeline.yml" target="_blank">link</a>:</p>
<div class="language-yaml highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="na">apiVersion</span><span class="pi">:</span> <span class="s">tekton.dev/v1beta1</span>
<span class="na">kind</span><span class="pi">:</span> <span class="s">Pipeline</span>
<span class="na">metadata</span><span class="pi">:</span>
<span class="na">name</span><span class="pi">:</span> <span class="s">complex-pipeline</span>
<span class="na">spec</span><span class="pi">:</span>
<span class="na">params</span><span class="pi">:</span>
<span class="pi">-</span> <span class="na">name</span><span class="pi">:</span> <span class="s">gitrepositoryurl</span>
<span class="na">type</span><span class="pi">:</span> <span class="s">string</span>
<span class="na">description</span><span class="pi">:</span> <span class="s">Git Repo</span>
<span class="pi">-</span> <span class="na">name</span><span class="pi">:</span> <span class="s">gitrevision</span>
<span class="na">type</span><span class="pi">:</span> <span class="s">string</span>
<span class="na">description</span><span class="pi">:</span> <span class="s">Git Repo</span>
<span class="na">default</span><span class="pi">:</span> <span class="s">master</span>
<span class="na">workspaces</span><span class="pi">:</span>
<span class="pi">-</span> <span class="na">name</span><span class="pi">:</span> <span class="s">maven-settings</span>
<span class="pi">-</span> <span class="na">name</span><span class="pi">:</span> <span class="s">shared-workspace</span>
<span class="pi">-</span> <span class="na">name</span><span class="pi">:</span> <span class="s">shared-repo</span>
<span class="na">tasks</span><span class="pi">:</span>
<span class="pi">-</span> <span class="na">name</span><span class="pi">:</span> <span class="s">git-clone</span>
<span class="na">taskRef</span><span class="pi">:</span>
<span class="na">name</span><span class="pi">:</span> <span class="s">git-clone</span>
<span class="na">params</span><span class="pi">:</span>
<span class="s">...</span>
<span class="na">workspaces</span><span class="pi">:</span>
<span class="pi">-</span> <span class="na">name</span><span class="pi">:</span> <span class="s">output</span>
<span class="na">workspace</span><span class="pi">:</span> <span class="s">shared-workspace</span>
<span class="pi">-</span> <span class="na">name</span><span class="pi">:</span> <span class="s">build</span>
<span class="na">taskRef</span><span class="pi">:</span>
<span class="na">name</span><span class="pi">:</span> <span class="s">maven</span>
<span class="s">...</span>
<span class="na">runAfter</span><span class="pi">:</span>
<span class="pi">-</span> <span class="s">git-clone</span>
<span class="na">workspaces</span><span class="pi">:</span>
<span class="pi">-</span> <span class="na">name</span><span class="pi">:</span> <span class="s">maven-settings</span>
<span class="na">workspace</span><span class="pi">:</span> <span class="s">maven-settings</span>
<span class="pi">-</span> <span class="na">name</span><span class="pi">:</span> <span class="s">source</span>
<span class="na">workspace</span><span class="pi">:</span> <span class="s">shared-workspace</span>
<span class="pi">-</span> <span class="na">name</span><span class="pi">:</span> <span class="s">local-repo</span>
<span class="na">workspace</span><span class="pi">:</span> <span class="s">shared-repo</span>
<span class="pi">-</span> <span class="na">name</span><span class="pi">:</span> <span class="s">tests</span>
<span class="na">taskRef</span><span class="pi">:</span>
<span class="na">name</span><span class="pi">:</span> <span class="s">maven</span>
<span class="s">...</span>
<span class="na">runAfter</span><span class="pi">:</span>
<span class="pi">-</span> <span class="s">build</span>
<span class="s">...</span>
<span class="pi">-</span> <span class="na">name</span><span class="pi">:</span> <span class="s">static-code-analysis</span>
<span class="na">taskRef</span><span class="pi">:</span>
<span class="na">name</span><span class="pi">:</span> <span class="s">maven</span>
<span class="s">...</span>
<span class="na">runAfter</span><span class="pi">:</span>
<span class="pi">-</span> <span class="s">build</span>
<span class="s">...</span>
<span class="pi">-</span> <span class="na">name</span><span class="pi">:</span> <span class="s">validate-quality-gate</span>
<span class="na">taskRef</span><span class="pi">:</span>
<span class="na">name</span><span class="pi">:</span> <span class="s">validate-quality-gate</span>
<span class="s">...</span>
<span class="na">runAfter</span><span class="pi">:</span>
<span class="pi">-</span> <span class="s">static-code-analysis</span>
<span class="s">...</span>
<span class="pi">-</span> <span class="na">name</span><span class="pi">:</span> <span class="s">build-package</span>
<span class="na">taskRef</span><span class="pi">:</span>
<span class="na">name</span><span class="pi">:</span> <span class="s">tar-quarkus</span>
<span class="na">runAfter</span><span class="pi">:</span>
<span class="pi">-</span> <span class="s">tests</span>
<span class="pi">-</span> <span class="s">validate-quality-gate</span>
<span class="s">...</span>
<span class="pi">-</span> <span class="na">name</span><span class="pi">:</span> <span class="s">build-image</span>
<span class="na">taskRef</span><span class="pi">:</span>
<span class="na">name</span><span class="pi">:</span> <span class="s">openshift-client</span>
<span class="s">...</span>
<span class="na">runAfter</span><span class="pi">:</span>
<span class="pi">-</span> <span class="s">build-package</span>
<span class="s">...</span>
<span class="pi">-</span> <span class="na">name</span><span class="pi">:</span> <span class="s">setup-app</span>
<span class="na">taskRef</span><span class="pi">:</span>
<span class="na">name</span><span class="pi">:</span> <span class="s">openshift-client</span>
<span class="s">...</span>
<span class="na">runAfter</span><span class="pi">:</span>
<span class="pi">-</span> <span class="s">build-image</span>
<span class="s">...</span>
<span class="pi">-</span> <span class="na">name</span><span class="pi">:</span> <span class="s">deploy</span>
<span class="na">taskRef</span><span class="pi">:</span>
<span class="na">name</span><span class="pi">:</span> <span class="s">openshift-client</span>
<span class="s">...</span>
<span class="na">runAfter</span><span class="pi">:</span>
<span class="pi">-</span> <span class="s">setup-app</span>
<span class="s">...</span>
</code></pre></div></div>
<p>After breaking down this pipeline we will find the following sections:</p>
<ul>
<li><code class="language-plaintext highlighter-rouge">params</code>: This section is where we define our parameters. Parameters in Tekton can be of two types: <code class="language-plaintext highlighter-rouge">String</code> and <code class="language-plaintext highlighter-rouge">Array</code> of <code class="language-plaintext highlighter-rouge">Strings</code></li>
<li>
<p><code class="language-plaintext highlighter-rouge">workspace</code>: This section is where we define that we are going to need some persistent storage to save our files between each task.</p>
<p class="notice"><strong>INFO</strong>
Remember that we mentioned that each task is going to be a pod? Right, and that’s because each pod can start and run in different nodes and even if they have been eventually provisioned in the same node, their filesystem are volatile, so it disappear after it completes its execution.</p>
</li>
<li><code class="language-plaintext highlighter-rouge">tasks</code>: This section is where we define our tasks. We can refer to those tasks above-mentioned or we can create one from scratch right in this section. Each task will have its set of parameters and volumes, but how we tell then work like a chain? That’s what the <code class="language-plaintext highlighter-rouge">runAfter</code> attribute does. This attribute accepts a list of tasks. So, you can have multiple tasks running in parallel. And that’s what we do with the tasks <code class="language-plaintext highlighter-rouge">tests</code>, and the other two <code class="language-plaintext highlighter-rouge">static-code-analysis</code> and <code class="language-plaintext highlighter-rouge">validate-quality-gate</code>. These both sets of tasks will run in parallel.</li>
</ul>
<h3 id="creating-the-pipeline-environment">Creating the pipeline environment</h3>
<p>So, let’s create a namespace for our pipeline. Which can also be where we provision our applications components in a real environment.</p>
<div class="language-shell highlighter-rouge"><div class="highlight"><pre class="highlight"><code>kubectl create ns pipelines-tutorial
</code></pre></div></div>
<p>Just for the sake of not making any mistakes. Let’s now change our context to the namespace we have just created.</p>
<p class="notice"><strong>NOTE</strong>
Here you can use a kubectl plugin to make it easier <code class="language-plaintext highlighter-rouge">kubectl ns {namespace}</code> or just <code class="language-plaintext highlighter-rouge">oc project {namepsace}</code>, if you prefer.</p>
<div class="language-shell highlighter-rouge"><div class="highlight"><pre class="highlight"><code>kubectl config set-context <span class="nt">--current</span> <span class="nt">--namespace</span><span class="o">=</span>pipelines-tutorial
</code></pre></div></div>
<p>Once you create this namespace the <em>Openshift Pipelines Operator</em> automatically adds and configures a <code class="language-plaintext highlighter-rouge">pipeline</code> service account that has permissions to build and push an image. This service account is used by <code class="language-plaintext highlighter-rouge">pipelineruns</code>.</p>
<h3 id="creating-pipeline">Creating pipeline</h3>
<p>At this point you can start creating the pipeline. This command will create <em>Tasks</em>, the <em>Pipeline</em>, and <em>Persistent Volumes</em> that we are going to require.</p>
<div class="language-shell highlighter-rouge"><div class="highlight"><pre class="highlight"><code>kubectl apply <span class="nt">-f</span> 2-pipeline
</code></pre></div></div>
<p>And the result will be something like:</p>
<div class="language-shell highlighter-rouge"><div class="highlight"><pre class="highlight"><code>persistentvolumeclaim/shared-workspace created
persistentvolumeclaim/shared-repo created
buildconfig.build.openshift.io/basic-quarkus-app created
imagestream.image.openshift.io/basic-quarkus-app created
task.tekton.dev/git-clone created
task.tekton.dev/maven created
task.tekton.dev/openshift-client created
task.tekton.dev/tar-quarkus created
task.tekton.dev/validate-quality-gate created
pipeline.tekton.dev/complex-pipeline created
</code></pre></div></div>
<p>Once you have created your pipeline you can already see what has been created using the tekton cli or <code class="language-plaintext highlighter-rouge">tkn</code>.</p>
<p>We can list the existing cluster tasks</p>
<div class="language-shell highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nv">$ </span>tkn clustertasks <span class="nb">ls
</span>NAME DESCRIPTION AGE
buildah 5 days ago
buildah-v0-11-3 5 days ago
...
</code></pre></div></div>
<p>We can list the existing tasks that are installed in this namespace.</p>
<div class="language-shell highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nv">$ </span>tkn tasks <span class="nb">ls
</span>NAME DESCRIPTION AGE
git-clone 2 minutes ago
maven 2 minutes ago
openshift-client 2 minutes ago
tar-quarkus 2 minutes ago
validate-quality-gate 2 minutes ago
</code></pre></div></div>
<p>We can list the pipelines that were created in this namespace.</p>
<div class="language-shell highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nv">$ </span>tkn pipeline <span class="nb">ls
</span>NAME AGE LAST RUN STARTED DURATION STATUS
complex-pipeline 2 minutes ago <span class="nt">---</span> <span class="nt">---</span> <span class="nt">---</span> <span class="nt">---</span>
</code></pre></div></div>
<p>We can describe de pipeline <code class="language-plaintext highlighter-rouge">complex-pipeline</code> to see its <code class="language-plaintext highlighter-rouge">resources</code>, <code class="language-plaintext highlighter-rouge">parameters</code>, <code class="language-plaintext highlighter-rouge">tasks</code>, and <code class="language-plaintext highlighter-rouge">pipelineruns</code>.</p>
<div class="language-shell highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nv">$ </span>tkn pipeline describe complex-pipeline
Name: complex-pipeline
Namespace: pipelines-tutorial
📦 Resources
No resources
⚓ Params
NAME TYPE DESCRIPTION DEFAULT VALUE
∙ gitrepositoryurl string Git Repo <span class="nt">---</span>
∙ gitrevision string Git Repo master
🗒 Tasks
NAME TASKREF RUNAFTER
∙ git-clone git-clone
∙ build maven git-clone
∙ tests maven build
∙ static-code-analysis maven build
∙ validate-quality-gate validate-quality-gate static-code-analysis
∙ build-package tar-quarkus tests, validate-quality-gate
∙ build-image openshift-client build-package
∙ setup-app openshift-client build-image
∙ deploy openshift-client setup-app
⛩ PipelineRuns
No pipelineruns
</code></pre></div></div>
<p>But, if you try to see if there’s is any pipeline running, you will not find anything yet. This is because we did’t started any one yet. To do so we have two alternatives:</p>
<ol>
<li>Create a <code class="language-plaintext highlighter-rouge">PipelineRun</code> with all the required inputs, outputs, and persistent volumes.</li>
<li>Create a <code class="language-plaintext highlighter-rouge">Trigger - Event listerner</code> and a couple of other configurations to make it ready to receive webhooks from GitHub, for instance.</li>
</ol>
<p>So, you are a lucky guy! We will go through both of them you how to run with the command line and by using triggers. Triggers are going to be covered in the next secion, so let’s try starting our pipeline manually.</p>
<h2 id="start-the-pipelines-using-cli">Start the pipelines using CLI</h2>
<p>Let’s start the pipeline from the command line will be as easy as described in the following command:</p>
<div class="language-shell highlighter-rouge"><div class="highlight"><pre class="highlight"><code>tkn pipeline start complex-pipeline <span class="se">\</span>
<span class="nt">-p</span> <span class="nv">gitrepositoryurl</span><span class="o">=</span>https://github.com/mgo-dev-mode/basic-quarkus-app <span class="se">\</span>
<span class="nt">-w</span> <span class="nv">name</span><span class="o">=</span>maven-settings,emptyDir<span class="o">=</span><span class="s2">""</span> <span class="se">\</span>
<span class="nt">-w</span> <span class="nv">name</span><span class="o">=</span>shared-workspace,claimName<span class="o">=</span>shared-workspace,subPath<span class="o">=</span><span class="s2">""</span> <span class="se">\</span>
<span class="nt">-w</span> <span class="nv">name</span><span class="o">=</span>shared-repo,claimName<span class="o">=</span>shared-repo,subPath<span class="o">=</span><span class="s2">""</span>
</code></pre></div></div>
<ol>
<li><code class="language-plaintext highlighter-rouge">complex-pipeline</code>: is the name of the pipeline;</li>
<li><code class="language-plaintext highlighter-rouge">-p</code>: Informs the only required parameter that is the git repository URL;</li>
<li><code class="language-plaintext highlighter-rouge">-w</code>: Informs all workspaces and their PVCs that we created when we installed the pipeline. The syntax is is like this:
<ul>
<li>For PVCs: <code class="language-plaintext highlighter-rouge">name=[WORKSPACE_NAME],claimName=[PVC_NAME],subPath=[PVC_SUB_PATH_TO_BE_MOUNTED]</code></li>
<li>For Secrets: <code class="language-plaintext highlighter-rouge">name=[WORKSPACE_NAME],secret=[SECRET_NAME]</code></li>
<li>For ConfigMaps: <code class="language-plaintext highlighter-rouge">name=[WORKSPACE_NAME],config=[CONFIG_MAP_NAME],item=[KEY]</code></li>
</ul>
</li>
</ol>
<p>You can find more options <a href="https://github.com/tektoncd/cli/blob/master/docs/cmd/tkn_pipeline_start.md" target="_blank">here</a>.</p>
<p>Once you start the pipeline you can see its logs using the following command:</p>
<p class="notice"><strong>NOTE</strong>
This command comes in handy when you need to see the logs of all tasks runs.</p>
<div class="language-shell highlighter-rouge"><div class="highlight"><pre class="highlight"><code>tkn <span class="nb">pr </span>logs <span class="nt">-L</span> <span class="nt">-f</span>
</code></pre></div></div>
<p>And the result will be similar to this:</p>
<p><img src="/assets/images/how-to-build-cloud-native-pipeline/pipelinerun-tkn-log.gif" alt="tkn-logs" /></p>
<h2 id="creating-triggers">Creating triggers</h2>
<p>Regarding triggers you will see that we have three different files that are worth to have some description.</p>
<ul>
<li><code class="language-plaintext highlighter-rouge">TriggerTemplate</code>: is an element that define what will be the parameters, and PipelineRuns templates (here you can have as many as you want), and of course as we are talking about PipelineRuns we need to define all the resources (input and outputs) and the PersistenceVolumeClaims that are going to be bound to the defined workspaces, if there is any requirement for that, and our pipeline does require a few;</li>
<li><code class="language-plaintext highlighter-rouge">TriggerBinding</code>: is an element that extract which attributes are necessary to bootstrap the pipeline. It uses a syntax very similar to JsonPath with <code class="language-plaintext highlighter-rouge">$(body.key)</code> or <code class="language-plaintext highlighter-rouge">$(header.key)</code> and uses <code class="language-plaintext highlighter-rouge">.</code> (dots) to access each key individualy inside the json. For more details see this <a href="https://tekton.dev/docs/triggers/triggerbindings/" target="_blank">link</a>.</li>
<li><code class="language-plaintext highlighter-rouge">EventListers</code>: is an element that is used to process HTTP based events with JSON payloads. This element needs to be combined with both elements mentioned above: <code class="language-plaintext highlighter-rouge">TriggerTemplate</code> and <code class="language-plaintext highlighter-rouge">TriggerBindning</code>.</li>
</ul>
<p>So now let’s create the triggers.</p>
<div class="language-shell highlighter-rouge"><div class="highlight"><pre class="highlight"><code>kubectl apply <span class="nt">-f</span> 3-trigger
</code></pre></div></div>
<h2 id="simulate-a-push-event-from-github">Simulate a push event from Github</h2>
<p>Now let’s simulate the webhook:</p>
<p class="notice"><strong>NOTE</strong>
We are going to use a tool called <a href="https://httpie.org" target="_blank">httpie</a>.</p>
<div class="language-shell highlighter-rouge"><div class="highlight"><pre class="highlight"><code>http POST complex-pipeline-listener-interceptor-pipelines-tutorial.[FQDN] <span class="se">\</span>
<span class="s1">'Content-Type'</span>:<span class="s1">'application/json'</span> @gitpush.json
</code></pre></div></div>
<p>And the result will be something like:</p>
<div class="language-http highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="err">Content-Type':'application/json' @gitpush.json
</span><span class="k">HTTP</span><span class="o">/</span><span class="m">1.1</span> <span class="m">201</span> <span class="ne">Created</span>
<span class="na">content-length</span><span class="p">:</span> <span class="s">109</span>
<span class="na">content-type</span><span class="p">:</span> <span class="s">text/plain; charset=utf-8</span>
<span class="na">date</span><span class="p">:</span> <span class="s">Mon, 25 May 2020 22:44:19 GMT</span>
<span class="na">set-cookie</span><span class="p">:</span> <span class="s">b91e1d0c23621fb276e284974607bad4=ecbe8730ee157a849b9ce38e10cdf2ac; path=/; HttpOnly</span>
{
"eventID": "87sb6",
"eventListener": "complex-pipeline-listener-interceptor",
"namespace": "pipelines-tutorial"
}
</code></pre></div></div>
<p>In the following video we have three terminals: <strong>(1)</strong> is with the existing pods in the namespace; <strong>(2)</strong> is with the taskruns that are being created; <strong>(3)</strong> is the pipeline logs being showed. It begins with the webhook being triggered using the command above and then changes to the pipelinerun log.</p>
<p><img src="/assets/images/how-to-build-cloud-native-pipeline/eventlistener-tkn-log.gif" alt="tkn-logs" class="align-center" /></p>
<h2 id="web-console">Web Console</h2>
<p>In the Web Console it is possible to see the entirely new section for the pipelines.</p>
<p>This is how pipelines are listed:</p>
<!-- markdownlint-disable MD033 -->
<figure>
<a href="/assets/images/how-to-build-cloud-native-pipeline/webconsole-pipeline-list.png"><img src="/assets/images/how-to-build-cloud-native-pipeline/webconsole-pipeline-list.png" /></a>
</figure>
<!-- markdownlint-enable MD033 -->
<p>This is how the details of the pipeline is shown:</p>
<!-- markdownlint-disable MD033 -->
<figure>
<a href="/assets/images/how-to-build-cloud-native-pipeline/webconsole-pipeline-details.png"><img src="/assets/images/how-to-build-cloud-native-pipeline/webconsole-pipeline-details.png" /></a>
</figure>
<!-- markdownlint-enable MD033 -->
<p>This is how pipelineruns are listed:</p>
<!-- markdownlint-disable MD033 -->
<figure>
<a href="/assets/images/how-to-build-cloud-native-pipeline/webconsole-pipelinerun-list.png"><img src="/assets/images/how-to-build-cloud-native-pipeline/webconsole-pipelinerun-list.png" /></a>
</figure>
<!-- markdownlint-enable MD033 -->
<p>This is how pipelineruns look like after a successful run:</p>
<!-- markdownlint-disable MD033 -->
<figure>
<a href="/assets/images/how-to-build-cloud-native-pipeline/webconsole-pipelinerun-details.png"><img src="/assets/images/how-to-build-cloud-native-pipeline/webconsole-pipelinerun-details.png" /></a>
</figure>
<!-- markdownlint-enable MD033 -->
<p>This is how pipelineruns look like after an error occur:</p>
<!-- markdownlint-disable MD033 -->
<figure>
<a href="/assets/images/how-to-build-cloud-native-pipeline/webconsole-pipelinerun-details-error.png"><img src="/assets/images/how-to-build-cloud-native-pipeline/webconsole-pipelinerun-details-error.png" /></a>
</figure>
<!-- markdownlint-enable MD033 -->
<p>You can also see the logs in the PipelineRun</p>
<p><img src="/assets/images/how-to-build-cloud-native-pipeline/webconsole-pipelinerun-logs.gif" alt="tkn-logs" class="align-center" /></p>
<h2 id="conclusion">Conclusion</h2>
<p>It is really worth it to try this new way of creating a Cloud Native pipeline. It is easy, less centralized, light weight, and much more flexible then other tools cause each step can create a totally different environment for your application.</p>
<p>On the other hand there are some minor issues like there is no support for human approval like the one we commonly have been using in Jenkins. There is a github <a href="https://github.com/tektoncd/pipeline/issues/2217" target="_blank">issue</a> in the backlog to provide this feature.</p>
<p>Things that I didn’t use in this implementation were the <code class="language-plaintext highlighter-rouge">PipelineResources</code>. They really seem to be very useful, but there are some issues that made me rethink its use in this specific scenario. And here follow my findings:</p>
<ul>
<li><em>Git Resource</em>: The issue was regarding its current features, it doesn’t have the same features as the ones provided by the <a href="https://github.com/tektoncd/catalog/tree/v1beta1/git#git-tasks" target="_blank">Tektons catalog</a>. But I think that Tekton community is probably going to improve this. Either way as stated by the group in this repository they will be offering replacement tasks for pipeline resources due to the decision of keeping PipelineResources in alpha as mentioned <a href="https://github.com/tektoncd/pipeline/issues/1369" target="_blank">here</a>.</li>
<li><em>Image Resource</em>: The issue was that all build images require us to relax the security to let them execute in a priviledged environment. This is something that I rather didn’t want to do. So, I chose to create a binary <strong>source-to-image</strong> build that does not have any requirement for priviledged access. It has also many other advantages. We also have s2i builds there in the Openshift and Tekton catalog, but they seem to also require priviledged access.</li>
</ul>
<p>Despite those minor issues I think that creating those pipelines were fun and absolutely worth the time spent. The Tekton’s community has been doing a great job and of course Red Hat is also working hard to integrate their tools with it. Let’s see what the future of these technologies hold.</p>
<p>And you stay tuned for more articles like this. If you liked it, please share!</p>Marcelo OhashiThis post will show you details on how to provision, deploy and use some Tekton features and give you an overview of what is the new age of CI/CDQuick Start Knative2020-04-18T00:00:00-03:002020-04-18T00:00:00-03:00http://mohashi.io/blog/kubernetes/knative/setup-a-cloud-native-quarkus-app-in-knative<p>In this article, I am going to show you how to deploy a <em>Cloud-Native Application</em> in <em>Knative.</em> There are several advantages that we could use to make better usage of resources of your Kubernetes cluster.</p>
<p><em>Knative<sup id="fnref:1" role="doc-noteref"><a href="#fn:1" class="footnote" rel="footnote">1</a></sup></em> is a Kubernetes native platform built to deploy and manage serverless workloads. Knative was created by Google with the contribution of several companies, such as IBM, SAP, Red Hat, Pivotal, and others. It has some capabilities, such as scale to zero, stand up, a pluggable architecture, and it can be used with almost every kind of application.</p>
<p>A <em>Cloud-Native Application<sup id="fnref:2" role="doc-noteref"><a href="#fn:2" class="footnote" rel="footnote">2</a></sup></em> is a sort of application that uses technologies that run smoothly in a cloud environment and have implemented some requirements that would not only make it more reliable in dynamic environments, but also in platforms, such as Kubernetes, which I would include: health checks, circuit breakers, bulkhead patterns, and so on.</p>
<p>In order to deploy and test our serverless setup in a Kubernetes environment we will use the <em>Code Ready Containers</em> (CRC) environment. Code Ready Containers will spin up a cluster, which is an all-in-one cluster or both master and worker node, in your local machine. So, if you don’t have it already there you can follow the installation process in this link <a href="https://code-ready.github.io/crc/#installation_gsg" target="_blank">here</a>.</p>
<h2 id="installing-and-configuring-knative">Installing and configuring Knative</h2>
<p>After having the CRC up and running, let’s install the <em>Openshift Serverless Operator</em> witch is based on the version <code class="language-plaintext highlighter-rouge">0.13.1</code> of Knative. The process that we will go here is using the <code class="language-plaintext highlighter-rouge">kubectl</code> command-line interface (CLI).</p>
<div class="language-shell highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nv">$ </span>kubectl get packagemanifests/serverless-operator <span class="se">\</span>
<span class="nt">-n</span> openshift-marketplace <span class="nt">-o</span> yaml | yq r - <span class="se">\</span>
<span class="s2">"status.channels[*].name"</span> <span class="nt">--collect</span>
preview-4.3
techpreview
</code></pre></div></div>
<p>The command above returned all the supported channels that we are going to use. For the purpose of this test, we will pick the channel <code class="language-plaintext highlighter-rouge">preview-4.3</code> which will result in the following command to add the subscription in our Kubernetes.</p>
<div class="language-shell highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nv">$ </span><span class="nb">cat</span> <span class="o"><<</span> <span class="no">EOF</span><span class="sh"> | kubectl apply -f -
apiVersion: operators.coreos.com/v1alpha1
kind: Subscription
metadata:
name: serverless-operator
namespace: openshift-operators
spec:
channel: preview-4.3
name: serverless-operator
source: redhat-operators
sourceNamespace: openshift-marketplace
</span><span class="no">EOF
</span></code></pre></div></div>
<p>To install <code class="language-plaintext highlighter-rouge">KnativeServing</code> we must create it inside a <code class="language-plaintext highlighter-rouge">knative-serving</code> namespace.</p>
<div class="language-shell highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nv">$ </span><span class="nb">cat</span> <span class="o"><<</span> <span class="no">EOF</span><span class="sh"> | kubectl apply -f -
apiVersion: v1
kind: Namespace
metadata:
name: knative-serving
---
apiVersion: operator.knative.dev/v1alpha1
kind: KnativeServing
metadata:
name: knative-serving
namespace: knative-serving
</span><span class="no">EOF
</span></code></pre></div></div>
<p>You can validate the installation by running the following command and see all the operator’s pods running.</p>
<div class="language-shell highlighter-rouge"><div class="highlight"><pre class="highlight"><code>kubectl get pods <span class="nt">-n</span> knative-serving
</code></pre></div></div>
<p>You can see in the output that there are two <code class="language-plaintext highlighter-rouge">controllers</code> for this basic installation.</p>
<div class="language-shell highlighter-rouge"><div class="highlight"><pre class="highlight"><code>NAME READY STATUS RESTARTS AGE
activator-5c7cd46767-6gdqj 1/1 Running 0 5m13s
autoscaler-b8b4cb4b5-d79cg 1/1 Running 0 5m12s
autoscaler-hpa-86ccbc78d-7wvdn 1/1 Running 0 5m2s
autoscaler-hpa-86ccbc78d-f4ptr 1/1 Running 0 5m2s
controller-695597996-xxgls 1/1 Running 0 5m11s
controller-695597996-z6x6g 1/1 Running 0 5m11s
webhook-76bb99f856-wlclf 1/1 Running 0 5m10s
</code></pre></div></div>
<p>The controllers will be scanning all CR’s that we will be creating throughout the article.</p>
<h2 id="creating-the-cloud-native-application">Creating the Cloud-native application</h2>
<p>To create the Cloud-native application we will use the current GA version of <a href="https://quarkus.io" target="_blank">Quarkus framework</a>.</p>
<p>To create this application you need to have the following versions:</p>
<ul>
<li><a href="https://www.graalvm.org/" target="_blank">GraalVM</a> 19.3.1 or 20.0.0 for native compilation</li>
<li><a href="https://maven.apache.org/" target="_blank">Apache Maven</a> 3.6.2+</li>
</ul>
<p>So, let’s create our application by using <code class="language-plaintext highlighter-rouge">io.quarkus:quarkus-maven-plugin:1.3.2.Final</code> plugin.</p>
<div class="language-shell highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nv">$ </span>mvn io.quarkus:quarkus-maven-plugin:1.3.2.Final:create <span class="se">\</span>
<span class="nt">-DgroupId</span><span class="o">=</span>io.mohashi.sample <span class="se">\</span>
<span class="nt">-DartifactId</span><span class="o">=</span>quickstart-knative <span class="se">\</span>
<span class="nt">-DclassName</span><span class="o">=</span>io.mohashi.sample.resource.ProductResource <span class="se">\</span>
<span class="nt">-Dpath</span><span class="o">=</span><span class="s2">"/v1/api/product"</span>
</code></pre></div></div>
<p>The previous command is going to create <code class="language-plaintext highlighter-rouge">quickstart-knative</code> folder with the following contents inside:</p>
<div class="language-console highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="go">quickstart-knative
├── mvnw
├── mvnw.cmd
├── pom.xml
├── README.md
└── src
├── main
│ ├── docker
│ │ ├── Dockerfile.jvm
│ │ └── Dockerfile.native
│ ├── java
│ │ └── io
│ │ └── mohashi
│ │ └── sample
│ │ └── resource
│ │ └── ProductResource.java
│ └── resources
│ ├── application.properties
│ └── META-INF
│ └── resources
│ └── index.html
└── test
</span></code></pre></div></div>
<ul>
<li><code class="language-plaintext highlighter-rouge">mvnw and mvnw.cmd</code>: Maven command</li>
<li><code class="language-plaintext highlighter-rouge">Dockerfile.jvm</code>: Dockerfile used to build the Quarkus app in JVM mode</li>
<li><code class="language-plaintext highlighter-rouge">Dockerfile.native</code>: Dockerfile used to build the Quarkus app in native mode</li>
<li><code class="language-plaintext highlighter-rouge">ProductResource.java</code>: Class that implements a JAX-RS resource</li>
<li><code class="language-plaintext highlighter-rouge">application.properties</code>: A configuration file for the Quarkus app</li>
</ul>
<p>Now that we’ve created the app you can run it using the following command:</p>
<p class="notice--info"><strong>NOTE:</strong> This command also keeps watching your source code for any changes and when detected these changes are automatically compiled and your application reloaded. So, after you run it, you can leave it there running to take all the changes we will make here.</p>
<div class="language-shell highlighter-rouge"><div class="highlight"><pre class="highlight"><code>./mvn clean quarkus:dev
</code></pre></div></div>
<p>That will open the port <code class="language-plaintext highlighter-rouge">8080</code> on your local machine to access your new Cloud-native application. Try to open the application in your preferred browser by accessing <a href="http://localhost:8080" target="_blank">http://localhost:8080</a>.</p>
<p><img src="/assets/images/setup-a-cloud-native-quarkus-app-in-knative/quakus-app-web-browser.png" alt="Cloud-native-app" /></p>
<p>You can also use the <a href="http://httpie.org" target="_blank"><code class="language-plaintext highlighter-rouge">httpie</code></a> command to test the default service created at <code class="language-plaintext highlighter-rouge">localhost:8080/v1/api/product</code>.</p>
<div class="language-shell highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nv">$ </span>http localhost:8080/v1/api/product
HTTP/1.1 200 OK
Content-Length: 5
Content-Type: text/plain<span class="p">;</span><span class="nv">charset</span><span class="o">=</span>UTF-8
hello
</code></pre></div></div>
<p>After creating the application, we can start improving its endpoints. For the sake of simplicity, I will create a POJO <code class="language-plaintext highlighter-rouge">Product</code> as our model inside a package <code class="language-plaintext highlighter-rouge">io.mohashi.sample.model</code>.</p>
<div class="language-java highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kn">package</span> <span class="nn">io.mohashi.sample.model</span><span class="o">;</span>
<span class="kn">import</span> <span class="nn">java.math.BigDecimal</span><span class="o">;</span>
<span class="kn">import</span> <span class="nn">io.quarkus.runtime.annotations.RegisterForReflection</span><span class="o">;</span>
<span class="nd">@RegisterForReflection</span>
<span class="kd">public</span> <span class="kd">class</span> <span class="nc">Product</span> <span class="o">{</span>
<span class="kd">public</span> <span class="nc">Integer</span> <span class="n">id</span><span class="o">;</span>
<span class="kd">public</span> <span class="nc">String</span> <span class="n">name</span><span class="o">;</span>
<span class="kd">public</span> <span class="nc">String</span> <span class="n">sku</span><span class="o">;</span>
<span class="kd">public</span> <span class="nc">BigDecimal</span> <span class="n">value</span><span class="o">;</span>
<span class="kd">public</span> <span class="nf">Product</span><span class="o">()</span> <span class="o">{</span>
<span class="o">}</span>
<span class="kd">public</span> <span class="nf">Product</span><span class="o">(</span><span class="nc">Integer</span> <span class="n">id</span><span class="o">,</span> <span class="nc">String</span> <span class="n">name</span><span class="o">,</span>
<span class="nc">String</span> <span class="n">sku</span><span class="o">,</span> <span class="nc">BigDecimal</span> <span class="n">value</span><span class="o">)</span> <span class="o">{</span>
<span class="k">this</span><span class="o">.</span><span class="na">id</span> <span class="o">=</span> <span class="n">id</span><span class="o">;</span>
<span class="k">this</span><span class="o">.</span><span class="na">name</span> <span class="o">=</span> <span class="n">name</span><span class="o">;</span>
<span class="k">this</span><span class="o">.</span><span class="na">sku</span> <span class="o">=</span> <span class="n">sku</span><span class="o">;</span>
<span class="k">this</span><span class="o">.</span><span class="na">value</span> <span class="o">=</span> <span class="n">value</span><span class="o">;</span>
<span class="o">}</span>
<span class="o">}</span>
</code></pre></div></div>
<ul>
<li><code class="language-plaintext highlighter-rouge">@RegisterForReflection</code>: This annotation makes this class eligible for reflection event after we generate the native binary for this class.</li>
</ul>
<p class="notice"><strong>Note</strong>
At the moment, when JSON-B or Jackson tries to get the list of fields of a class, if the class is not registered for reflection, no exception will be thrown. GraalVM will simply return an empty list of fields.<br />
Hopefully, this will change in the future and make the error more obvious.</p>
<p>Let’s change a little bit the JAX-RS endpoint to return a list of objects of <code class="language-plaintext highlighter-rouge">Product</code>.</p>
<div class="language-java highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kn">package</span> <span class="nn">io.mohashi.sample.resource</span><span class="o">;</span>
<span class="kn">import</span> <span class="nn">io.mohashi.sample.model.Product</span><span class="o">;</span>
<span class="kn">import</span> <span class="nn">javax.ws.rs.GET</span><span class="o">;</span>
<span class="kn">import</span> <span class="nn">javax.ws.rs.Path</span><span class="o">;</span>
<span class="kn">import</span> <span class="nn">javax.ws.rs.Produces</span><span class="o">;</span>
<span class="kn">import</span> <span class="nn">javax.ws.rs.core.MediaType</span><span class="o">;</span>
<span class="kn">import</span> <span class="nn">javax.ws.rs.core.Response</span><span class="o">;</span>
<span class="kn">import</span> <span class="nn">java.math.BigDecimal</span><span class="o">;</span>
<span class="kn">import</span> <span class="nn">java.util.LinkedList</span><span class="o">;</span>
<span class="kn">import</span> <span class="nn">java.util.List</span><span class="o">;</span>
<span class="nd">@Path</span><span class="o">(</span><span class="s">"/v1/api/product"</span><span class="o">)</span>
<span class="kd">public</span> <span class="kd">class</span> <span class="nc">ProductResource</span> <span class="o">{</span>
<span class="kd">private</span> <span class="nc">List</span><span class="o"><</span><span class="nc">Product</span><span class="o">></span> <span class="n">products</span><span class="o">;</span>
<span class="kd">public</span> <span class="nf">ProductResource</span><span class="o">()</span> <span class="o">{</span>
<span class="n">products</span> <span class="o">=</span> <span class="k">new</span> <span class="nc">LinkedList</span><span class="o"><>();</span>
<span class="n">products</span><span class="o">.</span><span class="na">add</span><span class="o">(</span><span class="k">new</span> <span class="nc">Product</span><span class="o">(</span><span class="mi">1</span><span class="o">,</span> <span class="s">"Laptop"</span><span class="o">,</span>
<span class="s">"P00001"</span><span class="o">,</span> <span class="k">new</span> <span class="nc">BigDecimal</span><span class="o">(</span><span class="mi">10</span><span class="o">)));</span>
<span class="n">products</span><span class="o">.</span><span class="na">add</span><span class="o">(</span><span class="k">new</span> <span class="nc">Product</span><span class="o">(</span><span class="mi">2</span><span class="o">,</span> <span class="s">"Keyboard"</span><span class="o">,</span>
<span class="s">"P00002"</span><span class="o">,</span> <span class="k">new</span> <span class="nc">BigDecimal</span><span class="o">(</span><span class="mi">20</span><span class="o">)));</span>
<span class="o">}</span>
<span class="nd">@GET</span>
<span class="nd">@Produces</span><span class="o">(</span><span class="nc">MediaType</span><span class="o">.</span><span class="na">TEXT_PLAIN</span><span class="o">)</span>
<span class="kd">public</span> <span class="nc">Response</span> <span class="nf">list</span><span class="o">()</span> <span class="o">{</span>
<span class="k">return</span> <span class="nc">Response</span><span class="o">.</span><span class="na">ok</span><span class="o">(</span><span class="n">products</span><span class="o">).</span><span class="na">build</span><span class="o">();</span>
<span class="o">}</span>
<span class="o">}</span>
</code></pre></div></div>
<p class="notice--info"><strong>Note</strong>: Don’t forget to fix the unit tests after changing the resource.</p>
<p>In order to add features to a Quarkus application we use <em>extensions</em>. Extensions are just pluggable components that can be added to a Quarkus app. A Quarkus application comes out-of-the-box with a minimal set of extensions <code class="language-plaintext highlighter-rouge">quarkus-resteasy</code> and <code class="language-plaintext highlighter-rouge">quarkus-junit5</code>. So, we will add some new extensions that will provide the functionalities that we will need in this example. And here follows:</p>
<ul>
<li><code class="language-plaintext highlighter-rouge">resteasy-jsonb</code>: To provide automatic JSON object mapping conversion;</li>
<li><code class="language-plaintext highlighter-rouge">openshift</code>: To provide Knative support for our application;</li>
<li><code class="language-plaintext highlighter-rouge">smallrye-health</code>: To enable health checks in the cluster;</li>
</ul>
<p>To add an extension to our Quarkus application we will use the plugin <code class="language-plaintext highlighter-rouge">quarkus-maven-plugin</code>.</p>
<div class="language-shell highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nv">$ </span>./mvnw quarkus:add-extension <span class="se">\</span>
<span class="nt">-Dextensions</span><span class="o">=</span><span class="s2">"resteasy-jsonb, openshift, smallrye-health"</span>
</code></pre></div></div>
<p>The result is:</p>
<div class="language-shell highlighter-rouge"><div class="highlight"><pre class="highlight"><code>...
<span class="o">[</span>INFO] <span class="nt">---</span> quarkus-maven-plugin:1.3.2.Final:add-extension <span class="o">(</span>default-cli<span class="o">)</span> @ quickstart-knative <span class="nt">---</span>
✅ Adding extension io.quarkus:quarkus-smallrye-health
✅ Adding extension io.quarkus:quarkus-openshift
✅ Adding extension io.quarkus:quarkus-resteasy-jsonb
<span class="o">[</span>INFO] <span class="nt">------------------------------------------------------------------------</span>
<span class="o">[</span>INFO] BUILD SUCCESS
<span class="o">[</span>INFO] <span class="nt">------------------------------------------------------------------------</span>
<span class="o">[</span>INFO] Total <span class="nb">time</span>: 2.456 s
<span class="o">[</span>INFO] Finished at: 2020-04-17T12:14:51-03:00
<span class="o">[</span>INFO] <span class="nt">------------------------------------------------------------------------</span>
</code></pre></div></div>
<p>Now that we’ve added the extensions let’s check our endpoint again.</p>
<div class="language-shell highlighter-rouge"><div class="highlight"><pre class="highlight"><code>http localhost:8080/v1/api/product
</code></pre></div></div>
<p>The result is:</p>
<div class="language-http highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">HTTP</span><span class="o">/</span><span class="m">1.1</span> <span class="m">200</span> <span class="ne">OK</span>
<span class="na">Content-Length</span><span class="p">:</span> <span class="s">105</span>
<span class="na">Content-Type</span><span class="p">:</span> <span class="s">application/json</span>
<span class="p">[</span><span class="w">
</span><span class="p">{</span><span class="w">
</span><span class="nl">"id"</span><span class="p">:</span><span class="w"> </span><span class="mi">1</span><span class="p">,</span><span class="w">
</span><span class="nl">"name"</span><span class="p">:</span><span class="w"> </span><span class="s2">"Laptop"</span><span class="p">,</span><span class="w">
</span><span class="nl">"sku"</span><span class="p">:</span><span class="w"> </span><span class="s2">"P00001"</span><span class="p">,</span><span class="w">
</span><span class="nl">"value"</span><span class="p">:</span><span class="w"> </span><span class="mi">10</span><span class="w">
</span><span class="p">},</span><span class="w">
</span><span class="p">{</span><span class="w">
</span><span class="nl">"id"</span><span class="p">:</span><span class="w"> </span><span class="mi">2</span><span class="p">,</span><span class="w">
</span><span class="nl">"name"</span><span class="p">:</span><span class="w"> </span><span class="s2">"Keyboard"</span><span class="p">,</span><span class="w">
</span><span class="nl">"sku"</span><span class="p">:</span><span class="w"> </span><span class="s2">"P00002"</span><span class="p">,</span><span class="w">
</span><span class="nl">"value"</span><span class="p">:</span><span class="w"> </span><span class="mi">20</span><span class="w">
</span><span class="p">}</span><span class="w">
</span><span class="p">]</span><span class="w">
</span></code></pre></div></div>
<h2 id="deploying-in-knative">Deploying in Knative</h2>
<p>So, now that we have our bare minimal application built let’s move on to deploy it into our Kubernete instance. But, we need to prepare the configuration files with additional properties:</p>
<div class="language-properties highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="py">quarkus.container-image.registry</span><span class="p">=</span><span class="s">image-registry.openshift-image-registry.svc:5000</span>
<span class="py">quarkus.container-image.group</span><span class="p">=</span><span class="s">knative-app</span>
<span class="py">quarkus.kubernetes.deployment-target</span><span class="p">=</span><span class="s">knative</span>
</code></pre></div></div>
<ul>
<li><code class="language-plaintext highlighter-rouge">quarkus.container-image.registry</code>: This property is to inform from where the Knative is going to retrieve the container image</li>
<li><code class="language-plaintext highlighter-rouge">quarkus.container-image.group</code>: This property informs the namespace that the deployment is going to be created</li>
<li><code class="language-plaintext highlighter-rouge">quarkus.kubernetes.deployment-target</code>: This property informs the <code class="language-plaintext highlighter-rouge">quarkus-maven-plugin</code> what is going to be our deployment target platform</li>
</ul>
<p>And then, let’s create a new namespace <code class="language-plaintext highlighter-rouge">knative-app</code> for our deployment.</p>
<div class="language-shell highlighter-rouge"><div class="highlight"><pre class="highlight"><code>kubectl create namespace knative-app
</code></pre></div></div>
<p>And let’s set the current namespace to <code class="language-plaintext highlighter-rouge">knative-app</code>.</p>
<div class="language-shell highlighter-rouge"><div class="highlight"><pre class="highlight"><code>kubectl config set-context <span class="nt">--current</span> <span class="nt">--namespace</span><span class="o">=</span>knative-app
</code></pre></div></div>
<p>We are going to need a container image repository for that. For the sake of simplicity, we are going to use the CRC’s internal repository.</p>
<div class="language-shell highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nv">$ </span>./mvnw clean package <span class="nt">-Dquarkus</span>.container-image.build<span class="o">=</span><span class="nb">true</span> <span class="se">\</span>
<span class="nt">-Dquarkus</span>.kubernetes-client.trust-certs<span class="o">=</span><span class="nb">true</span> <span class="nt">-DskipTests</span> <span class="se">\</span>
<span class="nt">-Pnative</span>
</code></pre></div></div>
<ul>
<li><code class="language-plaintext highlighter-rouge">native</code>: the native profile makes the build process to generate a native compilation using GraalVM</li>
</ul>
<p>The result is long and you should see the image being pushed to the repository:</p>
<div class="language-shell highlighter-rouge"><div class="highlight"><pre class="highlight"><code>...
<span class="o">[</span>INFO] <span class="o">[</span>io.quarkus.container.image.s2i.deployment.S2iProcessor] Writing manifest to image destination
<span class="o">[</span>INFO] <span class="o">[</span>io.quarkus.container.image.s2i.deployment.S2iProcessor] Storing signatures
<span class="o">[</span>INFO] <span class="o">[</span>io.quarkus.container.image.s2i.deployment.S2iProcessor] Successfully pushed image-registry.openshift-image-registry.svc:5000/knative-app/quickstart-knative@sha256:cbefd02c096002aaf0718b61299280825d392c866ddd03e933a58399b1980ea2
<span class="o">[</span>INFO] <span class="o">[</span>io.quarkus.container.image.s2i.deployment.S2iProcessor] Push successful
<span class="o">[</span>INFO] <span class="o">[</span>io.quarkus.deployment.QuarkusAugmentor] Quarkus augmentation completed <span class="k">in </span>137493ms
<span class="o">[</span>INFO] <span class="nt">------------------------------------------------------------------------</span>
<span class="o">[</span>INFO] BUILD SUCCESS
<span class="o">[</span>INFO] <span class="nt">------------------------------------------------------------------------</span>
<span class="o">[</span>INFO] Total <span class="nb">time</span>: 02:22 min
<span class="o">[</span>INFO] Finished at: 2020-04-17T12:19:19-03:00
<span class="o">[</span>INFO] <span class="nt">------------------------------------------------------------------------</span>
</code></pre></div></div>
<p>Now that we have there the image there we will be able to create the deployment of the Knative service.</p>
<p class="notice--warning"><strong>Small fix for <code class="language-plaintext highlighter-rouge">quarkus.maven.plugin</code> version <code class="language-plaintext highlighter-rouge">1.3.2.Final</code></strong><br />
This fix is necessary due to a newer mandatory <a href="https://github.com/knative/serving/blob/master/docs/runtime-contract.md#protocols-and-ports" target="_blank">Knative’s protocol requirement</a>. And the latest quarkus-maven-plugin is not fully compliant. So, to fix that we will need to manually rename a container port from <code class="language-plaintext highlighter-rouge">http</code> to <code class="language-plaintext highlighter-rouge">http1</code> and remove the liveness and readiness ports from probes in the generated file <code class="language-plaintext highlighter-rouge">target/kubernetes/knative.yml</code>, before deploying it.</p>
<div class="language-shell highlighter-rouge"><div class="highlight"><pre class="highlight"><code>kubectl apply <span class="nt">-f</span> ./target/kubernetes/knative.yml
</code></pre></div></div>
<p>As a result you are going to see the resources being created and the pods starting up.</p>
<div class="language-console highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="go">serviceaccount/quickstart-knative created
service.serving.knative.dev/quickstart-knative created
</span></code></pre></div></div>
<p>The list of pods that will appear in your console</p>
<div class="language-console highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="go">NAME READY STATUS RESTARTS AGE
quickstart-knative-1-build 0/1 Completed 0 29m
quickstart-knative-t5xgf-deployment-76f5d67bfd-jr5gt 2/2 Running 0 51s
</span></code></pre></div></div>
<p class="notice--info"><strong>Note</strong> If we wait for 90 seconds you will see that the pod is going to terminate due to the scale to zero feature of knative.</p>
<div class="language-console highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="go">quickstart-knative-...-jr5gt 2/2 Terminating 0 90s
quickstart-knative-...-jr5gt 2/2 Terminating 0 111s
quickstart-knative-...-jr5gt 2/2 Terminating 0 112s
quickstart-knative-...-jr5gt 0/2 Terminating 0 112s
quickstart-knative-...-jr5gt 0/2 Terminating 0 112s
quickstart-knative-...-jr5gt 0/2 Terminating 0 2m2s
quickstart-knative-...-jr5gt 0/2 Terminating 0 2m2s
</span></code></pre></div></div>
<p>Once we deploy our app and the app runs smoothly knative creates an ingress to our application and a deployment. You can check using the following command:</p>
<div class="language-shell highlighter-rouge"><div class="highlight"><pre class="highlight"><code>kubectl get route.serving/quickstart-knative
</code></pre></div></div>
<p>The output will be:</p>
<div class="language-console highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="go">NAME URL READY REASON
quickstart-knative http://quickstart-knative.knative-app.apps-crc.testing True
</span></code></pre></div></div>
<p>So, this is our ingress route to the Knative app. Let’s test it.</p>
<div class="language-shell highlighter-rouge"><div class="highlight"><pre class="highlight"><code>http <span class="si">$(</span>kc get route.serving/quickstart-knative <span class="nt">-o</span> yaml | yq r - <span class="s1">'status.url'</span><span class="si">)</span>/v1/api/product
</code></pre></div></div>
<p>After making this call you should see the application starting up in the Kubernete cluster:</p>
<div class="language-console highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="go">NAME READY STATUS RESTARTS AGE
quickstart-knative-...-9cx58 0/2 Pending 0 0s
quickstart-knative-...-9cx58 0/2 Pending 0 0s
quickstart-knative-...-9cx58 0/2 ContainerCreating 0 0s
quickstart-knative-...-9cx58 0/2 ContainerCreating 0 2s
quickstart-knative-...-9cx58 1/2 Running 0 4s
quickstart-knative-...-9cx58 2/2 Running 0 17s
</span></code></pre></div></div>
<p>And you can see the deployment count incrementing by one:</p>
<div class="language-console highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="go">NAME READY UP-TO-DATE AVAILABLE AGE
quickstart-knative-t5xgf-deployment 0/1 1 0 85m
quickstart-knative-t5xgf-deployment 1/1 1 1 86m
</span></code></pre></div></div>
<p>After a few seconds we can see the application response comming out:</p>
<div class="language-http highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">HTTP</span><span class="o">/</span><span class="m">1.1</span> <span class="m">200</span> <span class="ne">OK</span>
<span class="na">Cache-control</span><span class="p">:</span> <span class="s">private</span>
<span class="na">Set-Cookie</span><span class="p">:</span> <span class="s">79c0449662bd552659559da2f83e5009=036dc4d7b97d7c3b08ba671fa1a966bb; path=/; HttpOnly</span>
<span class="na">content-length</span><span class="p">:</span> <span class="s">105</span>
<span class="na">content-type</span><span class="p">:</span> <span class="s">application/json</span>
<span class="na">date</span><span class="p">:</span> <span class="s">Fri, 17 Apr 2020 17:11:57 GMT</span>
<span class="na">server</span><span class="p">:</span> <span class="s">envoy</span>
<span class="na">x-envoy-upstream-service-time</span><span class="p">:</span> <span class="s">6775</span>
<span class="p">[</span><span class="w">
</span><span class="p">{</span><span class="w">
</span><span class="nl">"id"</span><span class="p">:</span><span class="w"> </span><span class="mi">1</span><span class="p">,</span><span class="w">
</span><span class="nl">"name"</span><span class="p">:</span><span class="w"> </span><span class="s2">"Laptop"</span><span class="p">,</span><span class="w">
</span><span class="nl">"sku"</span><span class="p">:</span><span class="w"> </span><span class="s2">"P00001"</span><span class="p">,</span><span class="w">
</span><span class="nl">"value"</span><span class="p">:</span><span class="w"> </span><span class="mi">10</span><span class="w">
</span><span class="p">},</span><span class="w">
</span><span class="p">{</span><span class="w">
</span><span class="nl">"id"</span><span class="p">:</span><span class="w"> </span><span class="mi">2</span><span class="p">,</span><span class="w">
</span><span class="nl">"name"</span><span class="p">:</span><span class="w"> </span><span class="s2">"Keyboard"</span><span class="p">,</span><span class="w">
</span><span class="nl">"sku"</span><span class="p">:</span><span class="w"> </span><span class="s2">"P00002"</span><span class="p">,</span><span class="w">
</span><span class="nl">"value"</span><span class="p">:</span><span class="w"> </span><span class="mi">20</span><span class="w">
</span><span class="p">}</span><span class="w">
</span><span class="p">]</span><span class="w">
</span></code></pre></div></div>
<p>And after 90 seconds the deployment is updated again:</p>
<div class="language-console highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="go">NAME READY UP-TO-DATE AVAILABLE AGE
quickstart-knative-t5xgf-deployment 1/1 1 1 86m
quickstart-knative-t5xgf-deployment 1/0 1 1 87m
quickstart-knative-t5xgf-deployment 1/0 1 1 87m
quickstart-knative-t5xgf-deployment 0/0 0 0 87m
</span></code></pre></div></div>
<p>In the web console you can see the update happening as well.</p>
<!-- Courtesy of embedresponsively.com //-->
<div class="responsive-video-container">
<iframe src="https://www.youtube-nocookie.com/embed/KyCbkXF3Qpw" frameborder="0" allowfullscreen=""></iframe>
</div>
<h2 id="conclusion">Conclusion</h2>
<p>With this introductory article, we have created a Cloud-native application and deployed in a Knative Ready Kubernetes Cluster. Knative applications are good to optimize hardware utilization and save money and resources just when it is really necessary. Regardless, we need to understand that this comes with the price that these kinds of applications need to be not only very lightweight but also a fast startup time otherwise the API consumer is gonna have a long wait time to get their responses from the server.</p>
<p>Stay tuned for more articles related to developing Cloud-native apps.</p>
<p>You can see this sample code in the following repository: <a href="https://github.com/mgohashi/quickstart-knative" target="_blank">https://github.com/mgohashi/quickstart-knative</a>.</p>
<div class="footnotes" role="doc-endnotes">
<ol>
<li id="fn:1" role="doc-endnote">
<p><a href="https://knative.dev/" target="_blank">knative.dev</a> <a href="#fnref:1" class="reversefootnote" role="doc-backlink">↩</a></p>
</li>
<li id="fn:2" role="doc-endnote">
<p><a href="https://www.cncf.io/blog/2017/05/15/developing-cloud-native-applications/" target="_blank">Developing Cloud Native Applications</a> <a href="#fnref:2" class="reversefootnote" role="doc-backlink">↩</a></p>
</li>
</ol>
</div>Marcelo OhashiThis post will show you details on how to provision, deploy and use some Knative features and give you an overview of what is serverless