Multi-Node Quick Start: Install and run Vespa on AWS ECS

edit page

Requirements:

• An AWS account
• Familiarity with setting up a single-node system using Docker.

This document explains how to set up a multinode system by running the Vespa Docker image on AWS ECS. See Getting Started for troubleshooting, next steps and other guides.

Configure an AWS ECS cluster

1. Create a cluster
   
   • Log in to AWS and the EC2 Container Service. Click Create Cluster, using:

     Cluster name	vespa
     EC2 instance type	t2.medium
     Number of instances	5
     Key pair	Select or create your keypair
     Security group inbound rules - Port range	0 - 65535

   • Click Create and wait for the tasks to succeed
   • Click View Cluster
2. Configure ECS instances
   
   • Click the ECS Instances tab - this should show 5 container instances
   • Select the first Container instance checkbox, then Actions -> View/Edit attributes
   • Click Add attribute. Set Name=type and Value=configserver and click the green checkbox on the right. Then click Close.
   • Select the bottom 3 Container instance checkboxes, then Actions -> View/Edit attributes
   • Click Add attribute. Set Name=type and Value=services and click the green checkbox on the right. Then click Close
   • Click on the first Container Instance name
   • Take a note of the config server Private DNS address on the screen (e.g. ip-10-0-1-194.us-east-2.compute.internal) as we will use this in the following configuration.
3. Start the config server task
   
   • Click Task Definitions
   • Click Create new Task Definition
   • Click Configure via JSON and replace the content with:

   {
       "networkMode": "host",
       "containerDefinitions": [
           {
               "name": "configserver",
               "environment": [
                   {
                       "name": "VESPA_CONFIGSERVERS",
                       "value": "PUT_CONFIGSERVER_HOSTNAME_HERE"
                   }
               ],
               "image": "vespaengine/vespa",
               "command": [
                   "configserver"
               ],
               "privileged": true,
               "memoryReservation": 1024
           }
       ],
       "placementConstraints": [
           {
               "expression": "attribute:type == configserver",
               "type": "memberOf"
           }
       ],
       "family": "configserver"
   }
   

   • Replace the PUT_CONFIGSERVER_HOSTNAME_HERE with the config server hostname found previously
   • Click Save
   • Click Create
   • Choose Actions -> Run task and configure:

     Cluster	vespa
     Number of tasks	1
     Placement templates	One Task Per Host

   • Click Run Task
4. Start the services tasks
   
   • Click Task Definitions
   • Click Create new Task Definition
   • Click Configure via JSON and replace the content with:

   {
       "networkMode": "host",
       "containerDefinitions": [
           {
               "name": "services",
               "environment": [
                   {
                       "name": "VESPA_CONFIGSERVERS",
                       "value": "PUT_CONFIGSERVER_HOSTNAME_HERE"
                   }
               ],
               "image": "vespaengine/vespa",
               "command": [
                   "services"
               ],
               "privileged": true,
               "memoryReservation": 1024
           }
       ],
       "placementConstraints": [
           {
               "expression": "attribute:type == services",
               "type": "memberOf"
           }
       ],
       "family": "services"
   }
   

   • Replace the PUT_CONFIGSERVER_HOSTNAME_HERE with the config server hostname found previously
   • Click Save
   • Click Create
   • Choose Actions -> Run task and configure:

     Cluster	vespa
     Number of tasks	3
     Placement templates	One Task Per Host

   • Click Run Task

Configure and deploy Vespa

1. Gather required host information
   Navigate to Clusters -> vespa and select the Tasks tab. You should now see one configserver and 3 services tasks running. Select the ECS Instances tab and click on each Container Instance and note the external (Public DNS) and internal (Private DNS) hostnames. Example:

   Task type	External hostname	Internal hostname
   configserver	ec2-52-14-175-194.us-east-2.compute.amazonaws.com	ip-10-0-1-194.us-east-2.compute.internal
   services	ec2-52-15-184-219.us-east-2.compute.amazonaws.com	ip-10-0-0-219.us-east-2.compute.internal
   services	ec2-52-14-69-95.us-east-2.compute.amazonaws.com	ip-10-0-1-57.us-east-2.compute.internal
   services	ec2-18-220-230-182.us-east-2.compute.amazonaws.com	ip-10-0-0-38.us-east-2.compute.internal
   services	ec2-18-220-230-183.us-east-2.compute.amazonaws.com	ip-10-0-0-39.us-east-2.compute.internal

2. Connect to the public hostname of the config server

   $ ssh -i my-aws.key ec2-user@ec2-52-14-175-194.us-east-2.compute.amazonaws.com
   

3. Find the configserver container name (e.g. ecs-configserver-7-configserver-aa8991a7c2ebe0a44800)

   [ec2-user@ip-10-0-1-194 ~]$ docker ps
   

4. Enter the container

   [ec2-user@ip-10-0-1-194 ~]$ docker exec -ti <container name> bash
   [root@ip-10-0-1-194 ]#
   

5. Follow the multinode guide to configure and deploy a sample application