REST API Management, Monitoring & Analytics using Kong 3

NOTE:

• This is NOT a coding course. The focus is on API management tools using off-the-shelf (ready to use) softwares

• You don't need any credit card or payment to use tools in this course. All softwares (including API manager itself) are available as free edition. Some might even open source their codes.

Update January 2024

Major Update

• Use Kong 3, Elastic 8, and latest software version (as of 2024)

• Improved audio for ALL lectures

Update November 2020

Major Update

• Use Kong 2, Elastic 7, and latest software version (as of 2020)

• Added : OAuth2

• Added : Proxy Cache

• Improved audio for ALL lectures

  
  

Before We Begin

If you want to learn, you simply need a web browser. For running tools and hands-on, specific hardware and software required. Please see requirements on Lecture One (with title Readme First : Hardware & Software Required).

Prerequisites

We will use Docker (which is container technology) to install tools required for this API management course. In this course, you will learn basic knowledge of Docker. To install docker, you need to be familiar with:

• Windows : basic command prompt usage & command, administrator access

• Mac / Linux : basic terminal usage & command, root / sudo access

I provide Docker installation instructions for Windows 10 Professional Edition. For any other operating system -including other version of Windows-, I provide reference to official documentation, but not the video. I'm sorry about this, since I don't have Mac / Linux machine.

Don't worry, the official documentation is complete and easy enough to follow.

Works with REST API

Your workplace adopt modern architecture and relies on REST API. Having REST API give benefits, but the benefits comes at cost. For most teams, the processes of setting and enforcing standards or policies across their APIs is entirely manual. Often that process involves multiple teams or team members and becomes even more painful as the number of APIs grows. 

Now that you have provisioned your APIs, you also need to have visibility on your APIs.  How is your API traffic trending over time? Which API used most? Which API has large error rate? Having this data will help you analyze and take correct decision for further API development.

Implementing policies, securities, or analytics can be done by writing additional logic on your APIs. You can even write your own analytics tools for monitoring & analyzing your APIs. But those process will be redundant and time-consuming. Why don't we find a more efficient way so developers can focus on business needs?  That will ultimately maximize the productivity of the developers who build on the API, and in turn give benefits to company.   

API Management Software

API management tools can be an answer. API manager tools in market likely provides common API capabilities, like:

• API gateway, a single communication point for accessing API

• Basic security for accessing API

• Traffic control, like rate limiting or quota

• Monitor API health (is API accessible or down?)

• Load balancing

• API Analytics (hit rate, error rate)

• Logging your traffic

• Developer portal, act as API documentation. A common API documentation format is swagger / Open API specification

  
  

Big players like Google, IBM, or Axway provides good features for API management -if you are willing to pay some amount of money-. But if you prefer open source, start-small approach (but still good), there are also several API manager tools in the market,one of it is Kong API Manager (which we use in this course).

In this course, we will learn API Management using several tools. All software available as free edition. Some comes with option to upgrade into premium edition with more features. The tools we will learn here are:

• Kong API management

  This is the main focus of this course.

• Docker and docker compose

  Popular container technology to speed up our installation and configuration. However, this course is NOT a docker course, so we will no go deep on docker.

• Zipkin

  An open source REST API tracing tools. Useful for distributed tracing on different services, even if the API services scattered on several machines.

• Elasticsearch, Logstash & Kibana

  A Popular search engine and it's visualization tools. This course is an introductory course on how to use these stacks to analyze your API traffic. But this course is not focusing on Elastic stack.

• Prometheus & Grafana

  Open source monitoring tools to gather performance metrics and display those metrics as time-series charts.

Benefit of API Management (for Engineering and Business)

Let's face it : we have time constraints. Business needs those API. And they need it fast.

Writing API that just functionally works but not secure, or technically flawed, will bring trouble sooner or later. Maybe somebody find a security hole in your API, and exploit them. Or you work on a place where API is used in many places by many people / systems? Without having transparency of API analytics (usage, error rate, etc), it is difficult to know whether your API works fine, or need improvement.

Writing codes for implementing API security, analytics, or some other technical aspects are nice. Either veteran programmer or new kids on the house will love the challenge to write such codes and solve complex problems.

But the time constraints speaks loud. Writing such codes can be time-consuming. It is a good thing to focus developers effort to delivering business logic, which in turns, keep the business stays ahead.

API management can be used to helps you solve common API aspects, like security, analytics, or who can access which API. And most of the time, you don't need to write any code. Put your API behind some API management, and configure it to works with your API.

That means, productivity increases. Whatever role you have in the company, productivity always a good thing, and API management (even the open-source) can helps you to boost it.

Kong API Management

Kong provides API gateway for REST APIs with plug-in architecture to enable users adding functionalities to the core Kong gateway. Kong's API management platform is attractively lean. It should appeal to users with basic

requirements who want to get an initial platform that they can then add as they go, either

with Kong-or-community-provided plug-ins, or with developments of their own (yes, it's possible to develop Kong plugin based on your own requirement, although we don't cover it in this course). Companies aim to use Kong API management platform can start fast as you can see in the course -less than 1 hour to setup and run-.

Users looking to execute digital strategies will find Kong useful for enabling their platforms.

API Monitoring & Analytics

In this course, we will learn to do analytics and monitoring not only using Kong API Manager, but also some other tools, and most importantly how to integrate them. One tools only is quite powerful, but integration will give us more benefits

API management tools usually comes with API analytics module. We can use the analytics to gain visibility of the entire API ecosystem. The analytics will automatically and continuously collect all API-traffic data, like performance time, hit rate (how many hits in second), error rate. By leverage the API analytics, we can know the data about which API has good value, and which one needs to be improved.

In operation side, we can use API monitoring tools to proactively detect API traffic and performance issues. Imagine if a service down at 10 AM, and nobody knows until one hour later, which means problems already arose and some business opportunity might already lost. With leveraging open source Kong API management (and several other free tools), we can get notification in less than 5 minutes after the service troubled.