Console Service

With the Console Service you generate logs from your app for monitoring and debugging.

Private beta

The Console Service is currently in private beta with select users. It is planned for public release in Q4 2019 (see the Q4/19 Roadmap Update for more information). To stay up to date about upcoming releases please check out our blog, where we post release notes and roadmap updates.

Contents

Overview

Being able to access app logs is a key requirement of any developer building, monitoring or debugging an app, and of any administrator running an app in production. The Console Service provides you—as an app developer—with the tools necessary to create and access your app’s logs.

Key capabilities:

  • Provide developer APIs for producing structured or unstructured logs
  • Automatically capture and log stdout and stderr output (only Docker Runtime)
  • Automatically capture and log window.console output (only WebView Runtime)
  • Enable local or remote access to historical or real-time logs

Local access only (until 2020)

The Console Service currently only supports local access to logs, i.e. when you are in the same network as the edge device. As noted in our Q4/19 Roadmap Update, remote logging will be introduced in 2020. Stay tuned for more.

Basics

Structured vs. unstructured logs

Traditionally, logs were produced, persisted, and consumed as lines of text. This way of logging is referred to as unstructured logging. A traditional log file might look like this:

# File app.log
MyApp-1.0.3::console | 2019-09-11T21:46:12.106Z [info] Starting app...
MyApp-1.0.3::console | 2019-09-11T21:46:12.113Z [debug] Setting route '/activities'
MyApp-1.0.3::console | 2019-09-11T21:46:12.113Z [debug] 34 activities loaded
MyApp-1.0.3::console | 2019-09-11T21:46:12.113Z [info] User 'jdoe' registered
MyApp-1.0.3::console | 2019-09-11T21:46:12.114Z [debug] Setting route '/preferences'

This file contains unstructured logs—one per line— generated by an app. Metadata and actual content are all packed into a single line. Whilst these types of logs are easy to read as a human, they are not ideal for machine analysis.

Whilst ActyxOS supports this type of logging, we favor structured logging, where metadata and content are cleanly separated and each log message is produced in a machine-readable format. Consider the first log line shown above, this time in a structured format (with the information highlighted):

{
    "insertId": "1a18886587c2efa7b720554ff646d482",
    "insertTimestamp": "2019-09-11T21:46:12.106Z",
    "labels": {
        "ax.schemas.logging.level": "info",
        "ax.schemas.logging.producer.sourceId": "9d6083b3-3e06-4e59-b59c-bdfee1eae8cb",
        "ax.schemas.logging.producer.sourceName": "EdgeDevice1",
        "ax.schemas.logging.producer.appId": "8c243589-dd8a-4122-8c00-4e81bbc7b11b",
        "ax.schemas.logging.producer.appName": "MyApp",
        "ax.schemas.logging.producer.appVersion": "1.0.3",
        "com.app-builder-ltd.gdpr-relevant": false,
    },
    "logName": "com.app-builder-ltd.console",
    "payload": {
        "message": "Starting app..."
    }
}

Reserved namespaces ax.* and actyx.*

Both ax.* and actyx.* are reserved namespaces and aliases for com.actyx.*. This is done to save valuable bytes going through the network.

This structured log provides significantly more information than could be contained in a single line in any readable fashion. Given its JSON-format, it is also easy to programmatically parse for automated analysis or interactive investigation.

You can generate unstructured logs with ActyxOS

Whilst not recommended, the Console Service supports unstructured logging. Refer to the usage examples below for more information about how that works.

Log lifecycle and persistence

Logs are generated when your app runs. As you may not always be available to immediately capture ephemeral logs, the Console Service automatically persists logs for you. This persistence means that you can access logs at a later point in time.

Given disk constraints and configuration options, ActyxOS may at some point delete logs from the edge device. The deletion happens in a FIFO fashion, with the oldest logs being deleted first, irrespective of level.

Infinite persistence with the upcoming Actyx Console

The Actyx Console is planned for release in 2020 (see the Q4/19 Roadmap Update). It will allow for logs to automatically be sent from the edge device to the scalable, cloud-based Console. Then you will be able to access logs long after they have been deleted from the edge device. Stay tuned for release updates.

Usage

Creating high-quality applications and providing associated support means you must be able to see what is going on with your apps. The Console Service gives you the tools to do so.

Unstructured logging

Unstructured logging is supported for both docker apps and single-page web applications.

Docker apps

The business logic running in a docker container can output to stdout or stderr. The ActyxOS Docker Runtime automatically captures these outputs and transforms them into structured log messages of the following standard shape:

{
    "insertId": "<automatically-added>",
    "insertTimestamp": "<automatically-added>",
    "labels": {
        "ax.schemas.logging.level": "info",
        "ax.schemas.logging.producer.sourceId": "<automatically-added>",
        "ax.schemas.logging.producer.sourceName": "<automatically-added>",
        "ax.schemas.logging.producer.appId": "<automatically-added>",
        "ax.schemas.logging.producer.appName": "<automatically-added>",
        "ax.schemas.logging.producer.appVersion": "<automatically-added>",
        "ax.schemas.logging.producer.dockerTag": "<automatically-added>"
    },
    "logName": "stdout/stderr",
    "payload": {
        "message": "Starting app..."
    }
}

Note

All data streamed to stdout or stderr will have the same info log level. This is because the Docker interleaves the stdout and stderr into a single stream.

Single-page apps

Single-page applications have—via JavaScript—access to the console object (see also the WebView Runtime guide). The Console Service captures messages logged with the following functions:

  • console.debug mapped to debug log level
  • console.warn mapped to warn log level
  • console.info mapped to info log level
  • console.log mapped to info log level
  • console.error mapped to error log level

The logged string is transformed in a similar fashion as shown in the previous section.

Structured logging

Structured logging—the preferred method—is possible using the Console Service’s local HTTP API. You publish logs by submitting POST requests to the http://localhost:4457/api/v1/logs endpoint.

Important

The API can only be accessed locally, i.e. at localhost. It is not meant to be accessed from other devices. The Console Service automatically distributes published logs in the background. As a developer, you only need to interact with the local service and API.

The following code example shows how you can do this from a Node.js-based docker app.

const fetch = require('node-fetch')

const myMessage = {
    labels: {
        "ax.schemas.logging.level": "info",
        "com.app-builder-ltd.gdpr-relevant": false,
    },
    logName: "com.app-builder-ltd.console",
    payload: {
        message: "Starting app..."
    }
}

fetch('http://localhost:4454/api/v1/logs', {
    method: 'POST',
    body: JSON.stringify(myMessage),
    headers: { 'Content-Type': 'application/json' },
})

Using cURL from bash as an alternative, you would post a structured message using the following command:

echo '
{
    "labels": {
        "ax.schemas.logging.level": "info",
        "com.app-builder-ltd.gdpr-relevant": false,
    },
    "logName": "com.app-builder-ltd.console",
    "payload": {
        "message": "Starting app..."
    }
}
'\
| curl \
    -X "POST" \
    -d @- \
    -H "Content-Type: application/json" \
    -H "Accept: application/json" \
    http://localhost:4457/api/v1/logs

Auto-generated/-added metadata

The Console Service will automatically add the following metadata

  • insertId
  • insertTimestamp
  • labels from the com.actyx.* namespace

Accessing log messages

Until the release of the Actyx Console in 2020, logs can only be accessed using the Actyx CLI with the ax logs command. You can either retrieve all logs on the edge device or tail all logs generated from the time you execute the command.

Note

Until the release of the Actyx Console, logs can only be accessed locally. In order for this to work, you must use the --local flag when calling Actyx CLI commands.

Example:

$ ax logs --tail --local EdgeDevice1
> MyApp-1.0.3::console | 2019-09-11T21:46:12.106Z [info] Starting app...
> MyApp-1.0.3::console | 2019-09-11T21:46:12.113Z [debug] Setting route '/activities'
> MyApp-1.0.3::console | 2019-09-11T21:46:12.113Z [debug] 34 activities loaded
> MyApp-1.0.3::console | 2019-09-11T21:46:12.113Z [info] User 'jdoe' registered
> MyApp-1.0.3::console | 2019-09-11T21:46:12.114Z [debug] Setting route '/preferences'

You can also retrieve logs in their structured format using the --format=json flag with the ax logs command:

$ ax logs --tail --local --format=json EdgeDevice1 | jq
>{
    "insertId": "1a18886587c2efa7b720554ff646d482",
    "insertTimestamp": "2019-09-11T21:46:12.106Z",
    "labels": {
        "ax.schemas.logging.level": "info",
        "ax.schemas.logging.producer.sourceId": "9d6083b3-3e06-4e59-b59c-bdfee1eae8cb",
        "ax.schemas.logging.producer.sourceName": "EdgeDevice1",
        "ax.schemas.logging.producer.appId": "8c243589-dd8a-4122-8c00-4e81bbc7b11b",
        "ax.schemas.logging.producer.appName": "MyApp",
        "ax.schemas.logging.producer.appVersion": "1.0.3",
        "com.app-builder-ltd.gdpr-relevant": false,
    },
    "logName": "com.app-builder-ltd.console",
    "payload": {
        "message": "Starting app..."
    }
}