Technology Listing for CI-CD Implementation (Feb 2020)

Note: CI/CD stands for continuous integration and continuous delivery (aka continuous deployment)

1: Apache Ant

Apache Ant is a software tool for automating software build processes which originated from the Apache Tomcat project in early 2000. It was a replacement for the Make build tool of Unix, and was created due to a number of problems with Unix's make. It is similar to Make but is implemented using the Java language, requires the Java platform, and is best suited to building Java projects.

The most immediately noticeable difference between Ant and Make is that Ant uses XML to describe the code build process and its dependencies, whereas Make uses the Makefile format. By default, the XML file is named build.xml.

Ref: https://en.wikipedia.org/wiki/Apache_Ant

2: Apache Maven

Maven is a build automation tool used primarily for Java projects. Maven can also be used to build and manage projects written in C#, Ruby, Scala, and other languages. The Maven project is hosted by the Apache Software Foundation, where it was formerly part of the Jakarta Project.

Maven addresses two aspects of building software: how software is built, and its dependencies. Unlike earlier tools like Apache Ant, it uses conventions for the build procedure, and only exceptions need to be written down. An XML file describes the software project being built, its dependencies on other external modules and components, the build order, directories, and required plug-ins. It comes with pre-defined targets for performing certain well-defined tasks such as compilation of code and its packaging. Maven dynamically downloads Java libraries and Maven plug-ins from one or more repositories such as the Maven 2 Central Repository, and stores them in a local cache. This local cache of downloaded artifacts can also be updated with artifacts created by local projects. Public repositories can also be updated.

Maven is built using a plugin-based architecture that allows it to make use of any application controllable through standard input. A plugin for the .NET framework exists and is maintained, and a C/C++ native plugin is maintained for Maven 2.

Alternative technologies like Gradle and sbt as build tools do not rely on XML, but keep the key concepts Maven introduced. With Apache Ivy, a dedicated dependency manager was developed as well that also supports Maven repositories.

Apache Maven has support for reproducible builds.

Ref: https://en.wikipedia.org/wiki/Apache_Maven

3: Gradle

Gradle is an open-source build-automation system that builds upon the concepts of Apache Ant and Apache Maven and introduces a Groovy-based domain-specific language (DSL) instead of the XML form used by Apache Maven for declaring the project configuration. Gradle uses a directed acyclic graph ("DAG") to determine the order in which tasks can be run.

Gradle was designed for multi-project builds, which can grow to be quite large. It supports incremental builds by intelligently determining which parts of the build tree are up to date; any task dependent only on those parts does not need to be re-executed.

Ref: https://en.wikipedia.org/wiki/Gradle

4: Flutter SDK

Flutter is an open-source UI software development kit created by Google. It is used to develop applications for Android, iOS, Windows, Mac, Linux, Google Fuchsia and the web.

The major components of Flutter include:

1. Dart platform
2. Flutter engine
3. Foundation library
4. Design-specific widgets

Dart platform:
Flutter apps are written in the Dart language and make use of many of the language's more advanced features.

On Windows, macOS and Linux via the semi-official Flutter Desktop Embedding project, Flutter runs in the Dart virtual machine which features a just-in-time execution engine. While writing and debugging an app, Flutter uses Just In Time compilation, allowing for "hot reload", with which modifications to source files can be injected into a running application. Flutter extends this with support for stateful hot reload, where in most cases changes to source code can be reflected immediately in the running app without requiring a restart or any loss of state. This feature as implemented in Flutter has received widespread praise.

Release versions of Flutter apps are compiled with ahead-of-time (AOT) compilation on both Android and iOS, making Flutter's high performance on mobile devices possible.

Ref: https://en.wikipedia.org/wiki/Flutter_(software)

5: Jenkins

Jenkins is a free and open source automation server. Jenkins helps to automate the non-human part of the software development process, with continuous integration and facilitating technical aspects of continuous delivery. It is a server-based system that runs in servlet containers such as Apache Tomcat. It supports version control tools, including AccuRev, CVS, Subversion, Git, Mercurial, Perforce, TD/OMS, ClearCase and RTC, and can execute Apache Ant, Apache Maven and sbt based projects as well as arbitrary shell scripts and Windows batch commands. The creator of Jenkins is Kohsuke Kawaguchi. Released under the MIT License, Jenkins is free software.

Builds can be triggered by various means, for example by commit in a version control system, by scheduling via a cron-like mechanism and by requesting a specific build URL. It can also be triggered after the other builds in the queue have completed. Jenkins functionality can be extended with plugins.

Ref: https://en.wikipedia.org/wiki/Jenkins_(software)

6: Ansible 

Ansible is an open-source software provisioning, configuration management, and application-deployment tool. It runs on many Unix-like systems, and can configure both Unix-like systems as well as Microsoft Windows. It includes its own declarative language to describe system configuration.

Ansible was written by Michael DeHaan and acquired by Red Hat in 2015. Ansible is agentless, temporarily connecting remotely via SSH or remote PowerShell to do its tasks.

Ref: https://en.wikipedia.org/wiki/Ansible_(software)

7: SonarQube 

SonarQube (formerly Sonar) is an open-source platform developed by SonarSource for continuous inspection of code quality to perform automatic reviews with static analysis of code to detect bugs, code smells, and security vulnerabilities on 20+ programming languages. SonarQube offers reports on duplicated code, coding standards, unit tests, code coverage, code complexity, comments, bugs, and security vulnerabilities.

SonarQube can record metrics history and provides evolution graphs. SonarQube provides fully automated analysis and integration with Maven, Ant, Gradle, MSBuild and continuous integration tools (Atlassian Bamboo, Jenkins, Hudson, etc.).

SonarQube includes support for the programming languages Java (including Android), C#, PHP, JavaScript, TypeScript, C/C++, Ruby, Kotlin, Go, COBOL, PL/SQL, PL/I, ABAP, VB.NET, VB6, Python, RPG, Flex, Objective-C, Swift, CSS, HTML, and XML. Some of these are only available via a commercial license.

SonarQube is available for free under the GNU Lesser General Public License. An enterprise version for paid licensing also exists, as well as a data center edition that supports high availability.

SonarQube integrates with Eclipse, Visual Studio, and IntelliJ IDEA development environments through the SonarLint plug-ins, and also integrates with external tools like LDAP, Active Directory, GitHub, and others. SonarQube is expandable with the use of plug-ins.

Ref: https://en.wikipedia.org/wiki/SonarQube

8: Docker  

Docker is a set of platform as a service (PaaS) products that use OS-level virtualization to deliver software in packages called containers. Containers are isolated from one another and bundle their own software, libraries and configuration files; they can communicate with each other through well-defined channels. All containers are run by a single operating-system kernel and are thus more lightweight than virtual machines.

The service has both free and premium tiers. The software that hosts the containers is called Docker Engine. It was first started in 2013 and is developed by Docker, Inc.

Ref: https://en.wikipedia.org/wiki/Docker_(software)

9: EnvoyProxy
Envoy is an open source edge and service proxy, designed for cloud-native applications.

FEATURES
OUT OF PROCESS ARCHITECTURE
Envoy is a self contained, high performance server with a small memory footprint. It runs alongside any application language or framework.

HTTP/2 AND GRPC SUPPORT
Envoy has first class support for HTTP/2 and gRPC for both incoming and outgoing connections. It is a transparent HTTP/1.1 to HTTP/2 proxy.

ADVANCED LOAD BALANCING
Envoy supports advanced load balancing features including automatic retries, circuit breaking, global rate limiting, request shadowing, zone local load balancing, etc.

APIS FOR CONFIGURATION MANAGEMENT
Envoy provides robust APIs for dynamically managing its configuration.

OBSERVABILITY
Deep observability of L7 traffic, native support for distributed tracing, and wire-level observability of MongoDB, DynamoDB, and more.

Ref: https://www.envoyproxy.io/

10: Helm

Source 1: https://kubernetes.io/blog/2016/10/helm-charts-making-it-simple-to-package-and-deploy-apps-on-kubernetes/

Helm Charts: making it simple to package and deploy common applications on Kubernetes.

"Helm is the package manager (analogous to yum and apt) and Charts are packages (analogous to debs and rpms). The home for these Charts is the Kubernetes Charts repository which provides continuous integration for pull requests, as well as automated releases of Charts in the master branch."

Source 2: https://helm.sh/

Helm 3 -- The package manager for Kubernetes

What is Helm?
Helm helps you manage Kubernetes applications — Helm Charts help you define, install, and upgrade even the most complex Kubernetes application.

Charts are easy to create, version, share, and publish — so start using Helm and stop the copy-and-paste.

The latest version of Helm is maintained by the CNCF - in collaboration with Microsoft, Google, Bitnami and the Helm contributor community.

Why Teams Helm

MANAGE COMPLEXITY
Charts describe even the most complex apps, provide repeatable application installation, and serve as a single point of authority.

EASY UPDATES
Take the pain out of updates with in-place upgrades and custom hooks.

SIMPLE SHARING
Charts are easy to version, share, and host on public or private servers.

ROLLBACKS
Use helm rollback to roll back to an older version of a release with ease.

11: Kubernetes

Kubernetes (commonly stylized as k8s) is an open-source container-orchestration system for automating application deployment, scaling, and management. It was originally designed by Google, and is now maintained by the Cloud Native Computing Foundation. It aims to provide a "platform for automating deployment, scaling, and operations of application containers across clusters of hosts". It works with a range of container tools, including Docker. Many cloud services offer a Kubernetes-based platform or infrastructure as a service (PaaS or IaaS) on which Kubernetes can be deployed as a platform-providing service. Many vendors also provide their own branded Kubernetes distributions.

Ref: https://en.wikipedia.org/wiki/Kubernetes

12: Chef Inspec 

Turn your compliance, security, and other policy requirements into automated tests.

WHAT IS IT?
Chef InSpec is a free and open-source framework for testing and auditing your applications and infrastructure. Chef InSpec works by comparing the actual state of your system with the desired state that you express in easy-to-read and easy-to-write Chef InSpec code. Chef InSpec detects violations and displays findings in the form of a report, but puts you in control of remediation.

HOW CHEF INSPEC WORKS
Get started with Chef InSpec in 3 simple steps

1 -- WRITE THE TEST
Create simple Ruby-based tests to verify your expected state against the current state of your systems.

control 'example-1.0' do
  impact 0.9
  title 'Ensure login disabled'
  desc 'An optional description...'
  describe sshd_config do
    its('PermitRootLogin') {
      should_not cmp 'yes'
    }
  end
end

2 -- RUN THE TEST
Execute your test against your target system locally or remotely with one simple command.

$ inspec exec linux-baseline

3 -- SEE THE RESULTS
See which tests failed, passed and skipped and the expected state against the current state of your target system, in one simple output.

Ref: https://www.inspec.io/docs/

13: Spring boot 

Spring Boot is Spring's convention-over-configuration solution for creating stand-alone, production-grade Spring-based Applications that you can "just run". It is preconfigured with the Spring team's "opinionated view" of the best configuration and use of the Spring platform and third-party libraries so you can get started with minimum fuss. Most Spring Boot applications need very little Spring configuration. 

Features:

1. Create stand-alone Spring applications
2. Embed Tomcat or Jetty directly (no need to deploy WAR files)
3. Provide opinionated 'starter' Project Object Models (POMs) to simplify your Maven configuration
4. Automatically configure Spring whenever possible
5. Provide production-ready features such as metrics, health checks and externalized configuration
6. Absolutely no code generation and no requirement for XML configuration

Ref: https://en.wikipedia.org/wiki/Spring_Framework#Spring_Boot

14: Python Django

Django is a Python-based free and open-source web framework, which follows the model-template-view (MTV) architectural pattern. It is maintained by the Django Software Foundation (DSF), an independent organization established as a 501(c)(3) non-profit.

Django's primary goal is to ease the creation of complex, database-driven websites. The framework emphasizes reusability and "pluggability" of components, less code, low coupling, rapid development, and the principle of don't repeat yourself. Python is used throughout, even for settings files and data models. Django also provides an optional administrative create, read, update and delete interface that is generated dynamically through introspection and configured via admin models.

Some well-known sites that use Django include the Public Broadcasting Service, Instagram, Mozilla, The Washington Times, Disqus, Bitbucket, and Nextdoor. It was used on Pinterest, but later the site moved to a framework built over Flask.

Ref: https://en.wikipedia.org/wiki/Django_(web_framework)

15: Python Flask 

Flask is a micro web framework written in Python. It is classified as a microframework because it does not require particular tools or libraries. It has no database abstraction layer, form validation, or any other components where pre-existing third-party libraries provide common functions. However, Flask supports extensions that can add application features as if they were implemented in Flask itself. Extensions exist for object-relational mappers, form validation, upload handling, various open authentication technologies and several common framework related tools. Extensions are updated far more frequently than the core Flask program.

Applications that use the Flask framework include Pinterest, LinkedIn, and the community web page for Flask itself.

Ref: https://en.wikipedia.org/wiki/Flask_(web_framework)

16: Swagger / OpenAPI Specification

The OpenAPI Specification, originally known as the Swagger Specification, is a specification for machine-readable interface files for describing, producing, consuming, and visualizing RESTful web services. Originally part of the Swagger framework, it became a separate project in 2016, overseen by the OpenAPI Initiative, an open-source collaboration project of the Linux Foundation. Swagger and some other tools can generate code, documentation and test cases given an interface file.

Applications implemented based on OpenAPI interface files can automatically generate documentation of methods, parameters and models. This helps keep the documentation, client libraries, and source code in sync.

Ref: https://en.wikipedia.org/wiki/OpenAPI_Specification

17: Kafka 

Apache Kafka is an open-source stream-processing software platform developed by LinkedIn and donated to the Apache Software Foundation, written in Scala and Java. The project aims to provide a unified, high-throughput, low-latency platform for handling real-time data feeds. Kafka can connect to external systems (for data import/export) via Kafka Connect and provides Kafka Streams, a Java stream processing library.

Kafka uses a binary TCP-based protocol that is optimized for efficiency and relies on a "message set" abstraction that naturally groups messages together to reduce the overhead of the network roundtrip. This "leads to larger network packets, larger sequential disk operations, contiguous memory blocks [...] which allows Kafka to turn a bursty stream of random message writes into linear writes."

Apache Kafka is based on the commit log, and it allows users to subscribe to it and publish data to any number of systems or real-time applications. Example applications include managing passenger and driver matching at Uber, providing real-time analytics and predictive maintenance for British Gas’ smart home, and performing numerous real-time services across all of LinkedIn.

Ref: https://en.wikipedia.org/wiki/Apache_Kafka

18: Prometheus

Prometheus is a free software application used for event monitoring and alerting. It records real-time metrics in a time series database (allowing for high dimensionality) built using a HTTP pull model, with flexible queries and real-time alerting. The project is written in Go and licensed under the Apache 2 License, with source code available on GitHub, and is a graduated project of the Cloud Native Computing Foundation, along with Kubernetes and Envoy.

Ref: https://en.wikipedia.org/wiki/Prometheus_(software)

19: Grafana 

Grafana is an open-source, general purpose dashboard and graph composer, which runs as a web application. It supports graphite, InfluxDB, Prometheus or opentsdb as backends.

Ref: https://wiki.archlinux.org/index.php/Grafana

Grafana is an open source metric analytics & visualization suite. It is most commonly used for visualizing time series data for infrastructure and application analytics but many use it in other domains including industrial sensors, home automation, weather, and process control.

Ref: https://grafana.com/docs/grafana/v4.3/

20: Jaeger tracing 

Jaeger, inspired by Dapper and OpenZipkin, is a distributed tracing system released as open source by Uber Technologies. It is used for monitoring and troubleshooting microservices-based distributed systems, including:

$ Distributed context propagation
$ Distributed transaction monitoring
$ Root cause analysis
$ Service dependency analysis
$ Performance / latency optimization

Uber published a blog post, Evolving Distributed Tracing at Uber, where they explain the history and reasons for the architectural choices made in Jaeger.

Features
$ OpenTracing compatible data model and instrumentation libraries in Go, Java, Node, Python and C++
$ Uses consistent upfront sampling with individual per service/endpoint probabilities
$ Multiple storage backends: Cassandra, Elasticsearch, memory.
$ Adaptive sampling (coming soon)
$ Post-collection data processing pipeline (coming soon)

Ref: https://www.jaegertracing.io/docs/1.11/