AWS X-Ray with Client Instrumentation

Documentation: X-Ray SDK for Java Sample Application

If you haven't used X-Ray with Scorekeep yet, try the xray-gettingstarted branch first.

This branch includes the full content of the xray branch with the addition of client instrumentation with Amazon Cognito. Cognito integration lets users save their user ID, username and password in AWS. When a user logs in to the web app, it authenticates them to Cognito and retrieves their user ID, allowing them to use the same account accross sessions. When a user is signed in, the app uses their Cognito credentials to upload trace data to X-Ray.

Before you deploy this branch, follow the instructions in the next section to create the required Amazon Cognito resources.

Amazon Cognito Integration

From branch: cognito

Create a user pool, user pool client, and the Lambda function that the pool uses to confirm users, with the create-userpool.sh script in the _cognito directory.

~/eb-java-scorekeep/_cognito$ ./create-userpool.sh

This script uses a CloudFormation template named userpool to create the required resources. The template outputs the pool ID and client ID and saves the values in CloudFormation exports, which can be read from other stacks' templates. The env.config file then uses the Fn::ImportValue function to read these values and pass them to the API through environment variables. The API, in turn, provides a route named /api/userpool to retrieve the data, which the web app calls when you load the cognito demo page. This is all done to avoid hard coding the values in the web app.

Next, create an identity pool. The identity pool allows signed-in users to get credentials for use with the AWS SDK.

~/eb-java-scorekeep/_cognito$ ./create-identitypool.sh

This script creates another stack in CloudFormation that creates an identity pool associated with the Scorekeep user pool, and a role with write-only permissions for X-Ray that users assume when they sign in to their user account.

Finally, associate the role with the identity pool.

~/eb-java-scorekeep/_cognito$ ./associate-role.sh

This script retrieves information about the user pool and identity pool from the output of the CloudFormation stack and uses it to run an AWS CLI command that associates the role with the identity pool. While you can create a role association in a CloudFormation template, one of the attribute keys needs to contain resource IDs. You can't use functions like Ref to get key values in a CloudFormation template, so you need to use a script to associate the role to avoid hard coding values in the template.

AWS Lambda Integration

From branch: lambda

In the UserFactory class, Scorekeep calls a Node.js AWS Lambda function to generate random usernames. If the call to Lambda fails, Scorekeep falls back on a public API to generate names.

Run the script in the _lambda folder to create the AWS Lambda function that Scorekeep calls to generate random names: eb-java-scorekeep/_lambda$ ./create-random-name.sh

The script uses a CloudFormation template and the AWS CLI to create the function and its execution role:

• _lambda/random-name.yml - Template that defines the role and function
• _lambda/create-random-name.sh - Script to create the role and function
• _lambda/delete-random-name.sh - Script to delete the role and function

If you don't have the AWS CLI, install it or use the CloudFormation console to create a stack with the template.

Next, add Lambda permission to your instance profile (aws-elasticbeanstalk-ec2-role) to let the environment update the Lambda function:

• AWSLambdaFullAccess

Note: In the lambda branch, Scorekeep creates the Lambda function with a configuration file. In this branch, you create the function independently with the same template that creates the role. This lets the xray branch work even if the Lambda function and role have not been created, whereas in the lambda branch, the deployment fails if you haven't created the required role.

Finally, redeploy this branch to your environment. During deployment, the update script in lambda-function.config builds the function and uploads the source code to Lambda.

Create a new user in the web app and refresh your service map to see the two Lambda nodes.

Amazon RDS Integration

From branch: sql Documentation: Instrumenting Calls to a PostgreSQL Database

In application-pgsql.properties, Scorekeep adds the X-Ray SDK tracing interceptor to the JDBC data source used by Hibernate.

Add a PostgreSQL database to your Elastic Beanstalk environment to enable SQL tracing on the X-Ray demo page (/#/xray).

Hibernate also calls the database during application startup. No segment is available to the X-Ray SDK during startup, so we create one manually in RdsWebConfig.java by overriding Hibernate's SchemaExport class.

Scorekeep

Scorekeep is a RESTful web API implemented in Java that uses Spring to provide an HTTP interface for creating and managing game sessions and users. This project includes the Scorekeep API and a front-end web app that consumes it. The front end and API can run on the same server and domain or separately, with the API running in Elastic Beanstalk and the front end served statically by a CDN.

The master branch shows the use of Spring, Angular, nginx, the AWS SDK for Java, Amazon DynamoDB, Gradle, CORS, and AWS Elastic Beanstalk features that enable you to:

• Run both components on the same Amazon EC2 instance.
• Create required DynamoDB and Amazon SNS resources as part of the Elastic Beanstalk environment.
• Build the API from source on instance during deployment.

Other branches extend the application's functionality and show the use of other AWS services. See the readme in each branch for details about the integration and instructions for use.

Branches

• cognito - Support login and store users in an Amazon Cognito user pool. Get AWS SDK credentials and make service calls with a Cognito identity pool.
• cognito-basic - Use Cognito for user ID storage. User pool only, no identity pool.
• lambda - Call an AWS Lambda function to generate random names.
• lambda-worker - Run a Lambda function periodically to process game records and store the output in Amazon S3.
• sql - Use JDBC to store game histories in an attached PostgreSQL database instance.
• xray - Use the AWS X-Ray SDK for Java to instrument incoming requests, functions, SDK clients, SQL queries, HTTP clients, startup code, and AWS Lambda functions.
• xray-cognito - Use AWS credentials obtained with Amazon Cognito to upload trace data to X-Ray from the browser.
• xray-gettingstarted (tutorial) - Use the AWS X-Ray SDK for Java to instrument incoming requests and SDK clients (no additional configuration required).
• xray-worker - Instrumented Python Lambda worker function from the lambda-worker branch.

Use the procedures in the following sections to run the project on Elastic Beanstalk and configure it for local testing and development.

Sections

• Prerequisites
• Deploying the application
• Configuring notifications
• How it works
• Running the project locally
• Contributing

Prerequisites

• User permissions - Elastic Beanstalk, IAM (If you don't have permission to manage permissions in IAM, get someone who does to add DynamoDB and SNS permissions to the default Elastic Beanstalk instance profile.)
• Instance profile with permission to use DynamoDB and SNS
• Elastic Beanstalk environment running Java 8

Get permission to use Elastic Beanstalk

If you're using an IAM user with limited permissions, good work! Add Elastic Beanstalk permissions to your user account to get started.

To add Elastic Beanstalk permissions to an IAM user

1. Sign in to a user or role with administrator permissions.
2. Open the users page of the IAM console.
3. Choose a user.
4. Choose Add permissions.
5. Add the AWSElasticBeanstalkFullAccess managed policy.

Create an Elastic Beanstalk environment

Create a Java 8 SE environment in Elastic Beanstalk to host the application.

To create an Elastic Beanstalk environment running the Java 8 SE platform

1. Open the AWS Elastic Beanstalk console with this preconfigured link: console.aws.amazon.com/elasticbeanstalk/#/newApplication?applicationName=scorekeep...
2. Choose Create application to create an application with an environment running the Java 8 SE platform.
3. When your environment is ready, the console redirects you to the environment Dashboard.
4. Click the URL at the top of the page to open the site.

Give the application permission to use DynamoDB and SNS

When the Scorekeep API runs in Elastic Beanstalk, it uses the permissions of its EC2 instance to call AWS. Elastic Beanstalk provides a default instance profile that you can extend to grant the application the permissions it needs to read from and write to resource tables in DynamoDB, and send notifications with SNS.

To add DynamoDB and SNS permissions to the instances in your Elastic Beanstalk environment

1. Open the Elastic Beanstalk instance profile in the IAM console: aws-elasticbeanstalk-ec2-role
2. Choose Attach Policy.
3. Select AmazonDynamoDBFullAccess, and then choose Attach Policy.
4. Select AmazonSNSFullAccess, and then choose Attach Policy.

Deploying the application

Deploy the source code for the project to your Elastic Beanstalk environment.

To deploy the source code

1. Download the source bundle: eb-java-scorekeep-v1.zip
2. Open the Elastic Beanstalk console.
3. Choose your environment's name to open the Dashboard.
4. Choose Upload and Deploy.
5. Upload eb-java-scorekeep-v1.zip and choose Deploy.
6. Open the environment URL.

Click through the app to explore its functionality. Use the network console in your browser to see the HTTP requests that it sends to the API to read and write users, sessions, games, moves, and game state to DynamoDB via the API.

Configuring notifications

The API uses SNS to send a notification email when a game ends. To enable notifications, configure your email address in an environment variable.

To enable notifications

1. Open your environment's page in the environment management console.
2. Choose Configuration.
3. Choose Software Configuration
4. Under Environment Properties, set NOTIFICATION_TOPIC to your email address.
5. Check your email for a subscription confirmation.
6. Complete a game to trigger a notification.

How it works

The project includes two independent components

• An HTML and JavaScript front end in Angular 1.5
• A Java backend that uses Spring to provide a public API

Backend

The API runs at paths under /api that provide access to user, session, game, state, and move resources stored as JSON documents in DynamoDB. The API is RESTful, so you can create resources by sending HTTP POST requests to the resource path, for example /api/session. See the test script for example requests with cURL.

The application includes a configuration file that creates a DynamoDB table for each resource type.

The Buildfile tells Elastic Beanstalk to run gradle build during deployment to create an executable JAR file. The Procfile tells Elastic Beanstalk to run that JAR on port 5000.

Front end

The front end is an Angular 1.5 web app that uses $resource objects to perform CRUD operations on resources defined by the API. Users first encounter the main view and controller and progress through session and game views at routes that include the IDs of resources that the user creates.

The front end is served statically by the nginx proxy at the root path. The nginx.conf file in the source code overwrites the default configuration provided by the Elastic Beanstalk Java platform to serve the front end statically, and forward requests to paths starting with /api to the API running on port 5000.

Running the project locally

You can run both the API and front end locally with Gradle and the Spring Boot CLI. To get started, clone this repository or extract the contents of the source bundle that you downloaded earlier.

Run the Scorekeep API with Gradle

The API requires DynamoDB tables to exist in AWS to run locally. These tables are created by this configuration file when you launch the application in Elastic Beanstalk. If you terminate the environment, the tables are deleted. To create the tables without a running environment, use the configuration file as a template to create an AWS CloudFormation stack.

The application reads an environment variable named AWS_REGION to determine which region to connect to for calls to DynamoDB. In Elastic Beanstalk, this variable is set in a configuration file that reads the current region from CloudFormation and creates an Elastic Beanstalk environment property.

Set the environment variable and then use Gradle to build the API and run it locally on port 5000.

~/eb-java-scorekeep$ export AWS_REGION=us-west-2
~/eb-java-scorekeep$ gradle bootrun

The application needs AWS credentials to communicate with DynamoDB. In Elastic Beanstalk, Scorekeep gets credentials from the instance profile, which is the IAM role that is attached to the EC2 instance that runs the code. When you run the application locally, the AWS SDK for Java can retrieve credentials from files in ~/.aws/ or environment variables.

Follow the instructions in the AWS SDK for Java Developer Guide to provide access keys to the application: Set up AWS Credentials for Development.

Use the test script to verify that the API works.

~/eb-java-scorekeep$ ./bin/test-api.sh

The script targets localhost:5000. However, you can point it at the API running on any host by modifying the API variable at the top of the file.

Run the front end and configure it for local testing

Use the Spring Boot CLI to run the front end on port 8080.

~/eb-java-scorekeep$ spring run app.groovy

Open the app in a browser: localhost:8080

The app loads but can't hit the API, because it's trying to call paths relative to its own root, localhost:8080, but the API is running on localhost:5000 (or in Elastic Beanstalk). To fix this, configure the app with the full URL of the API.

To configure the web app with an absolute path to the API

1. Open eb-java-scorekeep/public/app/scorekeep.js.

2. Set the value of the api module to the full URL of the API.

   • Use the domain name of your environment to test changes to the front end without running the backend locally.

        module.value('api', 'http://scorekeep.XXXXXXXX.elasticbeanstalk.com/api/');
     

   • Use localhost:5000 to test both front end and backend changes when running both locally.

        module.value('api', 'http://localhost:5000/api/');
     

3. Refresh the app in your browser to load the updated script.

Configure the API for CORS

The API includes a CORS (cross-origin resource sharing) filter that allows traffic from localhost:8080.

  private static UrlBasedCorsConfigurationSource configurationSource() {
    CorsConfiguration config = new CorsConfiguration();
    config.setAllowCredentials(true);
    // Modify allowed origins if you run the client at a different endpoint
    config.addAllowedOrigin("http://localhost:8080");
    config.addAllowedHeader("*");
    config.addAllowedMethod("*");
    UrlBasedCorsConfigurationSource source = new UrlBasedCorsConfigurationSource();
    source.registerCorsConfiguration("/**", config);
    return source;
  }

This lets requests originating from a front end hosted locally on port 8080 to send requests to the API hosted on a different local port (i.e., localhost:5000) or on Elastic Beanstalk. This enables you to work on the front end locally and see changes immediately, without needing to deploy the source to your environment. When you make changes to the HTML or CSS in the app, simply refresh the browser.

When you run both the API and front end in the same Elastic Beanstalk environment, CORS is not required because the scripts that contact the API and the API itself are hosted on the same domain. To run the front end on a different port locally, or even host it on a completely different domain, add an allowed origin to the filter to whitelist it in the API.

To extend the CORS configuration to allow cross-origin requests from specific domains

1. Open src/main/java/scorekeepSimpleCORSFilter.java.

2. Add an allowed origin with the URL serving the front end.

    config.addAllowedOrigin("http://localhost:8080");
    config.addAllowedOrigin("http://scorekeep.XXXXXXXX.elasticbeanstalk.com");
   

3. Save the files and commit your changes.

    ~/eb-java-scorekeep$ git commit -am "Update API domain"
   

4. Create an archive of the updated source code.

    ~/eb-java-scorekeep$ git archive -o scorekeep-v1.zip HEAD
   

5. Open your environment's Dashboard in the Elastic Beanstalk console and deploy the updated code.

Contributing

This sample application could be better with your help!

• Add a new game!
  • Implement game logic in the game class. See TicTacToe.java.
  • Add the class to RulesFactory.java.
• Create your own client front end!
  • Web frameworks - Angular 2, React, ember, etc.
  • Mobile app
  • Desktop application
• Integrate with other AWS services!
  • CICD with AWS CodeCommit, AWS CodePipeline, AWS CodeBuild, and AWS CodeDeploy
  • Analytics with Amazon Kinesis, Amazon Athena, Amazon EMR, or Amazon QuickSight
  • Security with Amazon VPC
  • Performance with Amazon ElastiCache
  • Scalability with Amazon CloudFront
  • Portability with Amazon ECS or CloudFormation
  • Accessibility with Amazon Polly
• Write tests!
  • Unit tests
  • Integration tests
  • Functional tests
  • Load tests
• File an issue to report a bug or request new features.