Multi-Node Quick Start: Install and run Vespa on AWS ECS

Requirements:

• An AWS account
• Familiarity with setting up a single-node system using Docker.

This document explains how to set up a multinode system by running the Vespa Docker image on AWS ECS. See Getting Started for troubleshooting, next steps and other guides.

Configure an AWS ECS cluster

1. Create a cluster
   
   • Log in to AWS and the EC2 Container Service. Click Create Cluster, using:

     Cluster name	vespa
     EC2 instance type	t2.medium
     Number of instances	5
     Key pair	Select or create your keypair
     Security group inbound rules - Port range	0 - 65535

   • Click Create and wait for the tasks to succeed
   • Click View Cluster
2. Configure ECS instances
   
   • Click the ECS Instances tab - this should show 5 container instances
   • Select the first Container instance checkbox, then Actions -> View/Edit attributes
   • Click Add attribute. Set Name=type and Value=configserver and click the green checkbox on the right. Then click Close.
   • Select the bottom 3 Container instance checkboxes, then Actions -> View/Edit attributes
   • Click Add attribute. Set Name=type and Value=services and click the green checkbox on the right. Then click Close
   • Click on the first Container Instance name
   • Take a note of the config server Private DNS address on the screen (e.g. ip-10-0-1-194.us-east-2.compute.internal) as we will use this in the following configuration.
3. Start the config server task
   
   • Click Task Definitions
   • Click Create new Task Definition
   • Click Configure via JSON and replace the content with:

     {
         "networkMode": "host",
         "containerDefinitions": [
             {
                 "name": "configserver",
                 "environment": [
                     {
                         "name": "VESPA_CONFIGSERVERS",
                         "value": "PUT_CONFIGSERVER_HOSTNAME_HERE"
                     }
                 ],
                 "image": "vespaengine/vespa",
                 "command": [
                     "configserver"
                 ],
                 "privileged": true,
                 "memoryReservation": 1024
             }
         ],
         "placementConstraints": [
             {
                 "expression": "attribute:type == configserver",
                 "type": "memberOf"
             }
         ],
         "family": "configserver"
     }
     

   • Replace the PUT_CONFIGSERVER_HOSTNAME_HERE with the config server hostname found previously
   • Click Save
   • Click Create
   • Choose Actions -> Run task and configure:

     Cluster	vespa
     Number of tasks	1
     Placement templates	One Task Per Host

   • Click Run Task
4. Start the services tasks
   
   • Click Task Definitions
   • Click Create new Task Definition
   • Click Configure via JSON and replace the content with:

     {
         "networkMode": "host",
         "containerDefinitions": [
             {
                 "name": "services",
                 "environment": [
                     {
                         "name": "VESPA_CONFIGSERVERS",
                         "value": "PUT_CONFIGSERVER_HOSTNAME_HERE"
                     }
                 ],
                 "image": "vespaengine/vespa",
                 "command": [
                     "services"
                 ],
                 "privileged": true,
                 "memoryReservation": 1024
             }
         ],
         "placementConstraints": [
             {
                 "expression": "attribute:type == services",
                 "type": "memberOf"
             }
         ],
         "family": "services"
     }
     

   • Replace the PUT_CONFIGSERVER_HOSTNAME_HERE with the config server hostname found previously
   • Click Save
   • Click Create
   • Choose Actions -> Run task and configure:

     Cluster	vespa
     Number of tasks	3
     Placement templates	One Task Per Host

   • Click Run Task

Configure and deploy Vespa

1. Gather required host information
   Navigate to Clusters -> vespa and select the Tasks tab. You should now see one configserver and 3 services tasks running. Select the ECS Instances tab and click on each Container Instance and note the external (Public DNS) and internal (Private DNS) hostnames. Example:

   Task type	External hostname	Internal hostname
   configserver	ec2-52-14-175-194.us-east-2.compute.amazonaws.com	ip-10-0-1-194.us-east-2.compute.internal
   services	ec2-52-15-184-219.us-east-2.compute.amazonaws.com	ip-10-0-0-219.us-east-2.compute.internal
   services	ec2-52-14-69-95.us-east-2.compute.amazonaws.com	ip-10-0-1-57.us-east-2.compute.internal
   services	ec2-18-220-230-182.us-east-2.compute.amazonaws.com	ip-10-0-0-38.us-east-2.compute.internal
   services	ec2-18-220-230-183.us-east-2.compute.amazonaws.com	ip-10-0-0-39.us-east-2.compute.internal

2. Connect to the public hostname of the config server

   $ ssh -i my-aws.key ec2-user@ec2-52-14-175-194.us-east-2.compute.amazonaws.com
   

3. Find the configserver container name (e.g. ecs-configserver-7-configserver-aa8991a7c2ebe0a44800)

   [ec2-user@ip-10-0-1-194 ~]$ docker ps
   

4. Enter the container

   [ec2-user@ip-10-0-1-194 ~]$ docker exec -ti <container name> bash
   [root@ip-10-0-1-194 ]#
   

5. Follow the multinode guide to configure and deploy a sample application