Multi-Node Quick Start: Install and run Vespa on AWS ECS

Requirements:

This document explains how to set up a multinode system by running the Vespa Docker image on AWS ECS.

Configure an AWS ECS cluster

  1. Create a cluster
    • Log in to AWS and the EC2 Container Service. Click Create Cluster, using:
      Cluster namevespa
      EC2 instance typet2.medium
      Number of instances4
      Key pairSelect or create your keypair
      Security group inbound rules - Port range0 - 65535
    • Click Create and wait for the tasks to succeed
    • Click View Cluster
  2. Configure ECS instances
    • Click the ECS Instances tab - this should show 4 container instances
    • Select the first Container instance checkbox, then Actions -> View/Edit attributes
    • Click Add attribute. Set Name=type and Value=configserver and click the green checkbox on the right. Then click Close.
    • Select the bottom 3 Container instance checkboxes, then Actions -> View/Edit attributes
    • Click Add attribute. Set Name=type and Value=services and click the green checkbox on the right. Then click Close
    • Click on the first Container Instance name
    • Take a note of the config server Private DNS address on the screen (e.g. ip-10-0-1-194.us-east-2.compute.internal) as we will use this in the following configuration.
  3. Start the config server task
    • Click Task Definitions
    • Click Create new Task Definition
    • Click Configure via JSON and replace the content with:
    • {
          "networkMode": "host",
          "containerDefinitions": [
              {
                  "name": "configserver",
                  "environment": [
                      {
                          "name": "VESPA_CONFIGSERVERS",
                          "value": "PUT_CONFIGSERVER_HOSTNAME_HERE"
                      }
                  ],
                  "image": "vespaengine/vespa",
                  "command": [
                      "configserver"
                  ],
                  "privileged": true,
                  "memoryReservation": 1024
              }
          ],
          "placementConstraints": [
              {
                  "expression": "attribute:type == configserver",
                  "type": "memberOf"
              }
          ],
          "family": "configserver"
      }
      
    • Replace the PUT_CONFIGSERVER_HOSTNAME_HERE with the config server hostname found previously
    • Click Save
    • Click Create
    • Choose Actions -> Run task and configure:
      Clustervespa
      Number of tasks1
      Placement templatesOne Task Per Host
    • Click Run Task
  4. Start the services tasks
    • Click Task Definitions
    • Click Create new Task Definition
    • Click Configure via JSON and replace the content with:
    • {
          "networkMode": "host",
          "containerDefinitions": [
              {
                  "name": "services",
                  "environment": [
                      {
                          "name": "VESPA_CONFIGSERVERS",
                          "value": "PUT_CONFIGSERVER_HOSTNAME_HERE"
                      }
                  ],
                  "image": "vespaengine/vespa",
                  "command": [
                      "services"
                  ],
                  "privileged": true,
                  "memoryReservation": 1024
              }
          ],
          "placementConstraints": [
              {
                  "expression": "attribute:type == services",
                  "type": "memberOf"
              }
          ],
          "family": "services"
      }
      
    • Replace the PUT_CONFIGSERVER_HOSTNAME_HERE with the config server hostname found previously
    • Click Save
    • Click Create
    • Choose Actions -> Run task and configure:
      Clustervespa
      Number of tasks3
      Placement templatesOne Task Per Host
    • Click Run Task

Configure and deploy Vespa

  1. Gather required host information
    Navigate to Clusters -> vespa and select the Tasks tab. You should now see one configserver and 3 services tasks running. Select the ECS Instances tab and click on each Container Instance and note the external (Public DNS) and internal (Private DNS) hostnames. Example:
    Task typeExternal hostnameInternal hostname
    configserver ec2-52-14-175-194.us-east-2.compute.amazonaws.com ip-10-0-1-194.us-east-2.compute.internal
    services ec2-52-15-184-219.us-east-2.compute.amazonaws.com ip-10-0-0-219.us-east-2.compute.internal
    services ec2-52-14-69-95.us-east-2.compute.amazonaws.com ip-10-0-1-57.us-east-2.compute.internal
    services ec2-18-220-230-182.us-east-2.compute.amazonaws.com ip-10-0-0-38.us-east-2.compute.internal
  2. Connect to the public hostname of the config server
    $ ssh -i my-aws.key ec2-user@ec2-52-14-175-194.us-east-2.compute.amazonaws.com
    
  3. Find the configserver container name (e.g. ecs-configserver-7-configserver-aa8991a7c2ebe0a44800)
    [ec2-user@ip-10-0-1-194 ~]$ docker ps
    
  4. Enter the container
    [ec2-user@ip-10-0-1-194 ~]$ docker exec -ti <container name> bash
    [root@ip-10-0-1-194 ]#
    
  5. Follow the multinode guide to configure and deploy a sample application

Next Steps