ServiceLens: A Visual Overview

ServiceLens bridges the gap between X-Ray and CloudWatch by providing a “single pane of glass” unifying X-Ray traces with CloudWatch features such as metrics and alarms. The resulting experience definitely feels X-Ray-esque, built around the concept of a “service map” you can drill down into to get more information about a poorly-performing request. The new console experience makes it easy to slice and dice requests by dimensions, such as client IP or URL to quickly discover and understand your systems. - Forrest Brazeal

Prepping our test case

In order to effectively understand ServiceLens, we want a diversified test case. Below is a repository provided that leverages a bunch of different services and points of interaction between those services. This will serve as our test bed for playing with everything that ServiceLens has to offer.

The services and interactions involved are as follows. Everything is deployed in via AWS Serverless Application Model (SAM).

• AWS Lambda
• Amazon DynamoDB
• Amazon API Gateway
• X-Ray

You can check out the repository, and even deploy it easily to your own account with sam build && sam deploy if you would like!

The basics

You will notice we have turned out pretty extensive logging capabilities, as well as instrumented a bit of the code.

Globals:
  Function:
    Runtime: nodejs10.x
    MemorySize: 256
    Timeout: 3
    Tracing: Active
    Environment:
      Variables:
        SAMPLE_TABLE: !Ref SampleTable
  Api:
    EndpointConfiguration: REGIONAL
    TracingEnabled: True
    MethodSettings:
      - LoggingLevel: INFO
        MetricsEnabled: False
        DataTraceEnabled: True
        ResourcePath: "/*"
        HttpMethod: "*"

const AWSXRay = require('aws-xray-sdk');

// Create a DocumentClient that represents the query to add an item
const AWS = require('aws-sdk');
const docClient = new AWS.DynamoDB.DocumentClient({
  service: new AWS.DynamoDB()
});

AWSXRay.captureAWSClient(docClient.service);

With just these few bits of things, this opens up quite a bit of functionality and capabilities in ServiceLens.

First things first, we get a "ServiceMap" that we can filter in different ways. For our "listing" endpoint this is what it looks like.

Check out looking at our request map. Can you understand where errors may be in the system?

Let's dive into those errors

So we quickly surface that there is a problem, let's see if we can figure out WHAT went wrong.

Yep. We found that error saw it was a POST request from API Gateway.

The body was {"id": 1, "key2":"value2"} and our error was One or more parameter values were invalid: Type mismatch for key id expected: S actual: N. Oops! Our id was supposed be a string.

Easy fix!

Understanding our users

Let's pretend we just got a massive spike of traffic. How can we quickly understand the genesis of this?

Let's play with our traces screen.

Well there we go, someone (me) decided to slam our service with ApacheBench and generated 94% of our traffic. We can see everyone else is behaving correctly.

If desired we can drill down into what specifically a particular IP or user agent is doing, filter by response codes or URLs, etc. Once we've got our interesting batch of traces, we are free to start investigating them in detail to understand call for call what the actors are doing.

What you put in is what you get out

With X-Ray and by extensions, ServiceLens, you get out the value based on what you invest ahead of time. The more instrumented your calls are, and the more annotations and metadata you add to your X-Ray, the more value you will be able to extract.

I think ServiceLens is one of the largest observability leaps we have seen in the native AWS tooling yet.