Amazon Web Services

Table of Contents

Amazon Web Services (AWS) is a large cloud platform with a wide range of services. It scales well and is reliable, and a lot of teams run their applications on it.

Managing those resources directly can get complicated. Daestro gives you a simple interface to launch and manage EC2 instances, so you do not have to be an AWS expert. If you also use DigitalOcean or Linode, you can run jobs on those providers and AWS from the same place. You spend less time configuring infrastructure and more time on the jobs and applications.

If you are evaluating AWS for batch processing, see Daestro vs AWS Batch.

Step 1: Create an Access Key in Your AWS Account

Daestro needs an IAM user in your AWS account with permission to manage resources. Give that user only the access Daestro needs.

Log in to your AWS Console.
Open the IAM (Identity and Access Management) service. Type IAM in the main search bar if you cannot find it.
In the IAM dashboard, click Users in the left-hand menu, then click Create user.
Set user details:
- Give the user a name you will recognize, like daestro-integration-user.
- Do not check “Provide user access to the AWS Management Console”. This user is for programmatic access only.
- Click Next.
Set permissions:
- Select Attach policies directly.
- For most people, the simplest policy is AmazonEC2FullAccess. Search for it and check the box next to it.
- Click Next.

Advanced: Creating a Custom IAM Policy (Recommended)

You can create a custom policy that grants only the EC2 permissions Daestro needs, limited to one region.

On the Set permissions page, select Create policy. This opens a new tab.
Click the JSON tab.

Delete the existing content and paste the JSON below. Replace "us-east-1" with the AWS region you intend to use.

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "DaestroCorePermissions",
      "Effect": "Allow",
      "Action": ["ec2:*"],
      "Resource": "*",
      "Condition": {
        "StringEquals": {
          "ec2:Region": "us-east-1"
        }
      }
    }
  ]
}

Click Next: Tags, then Next: Review.
Give the policy a name, like Daestro-EC2-Restricted-Policy, and click Create policy.
Close this tab and return to the IAM user creation tab. Click the refresh button and search for the policy you just created. Check the box next to it.

Important: If you use a custom region-locked policy, when you create Compute Environments in Daestro you must select the region your policy is authorized for. Otherwise your jobs will fail.

Review the details on the final screen and click Create user.
AWS then shows the Access key ID and Secret access key. This is the only time you will see the secret key.
- Click Show under Secret access key.
- Copy both the Access key ID and the Secret access key and store them somewhere safe. You will need them in the next step.

Step 2: Add Your AWS Credentials to Daestro

Log in to your Daestro Console.
Open Cloud Auth from the main menu.
Click Add New and select AWS from the list of providers.
Fill in the form:
- Cloud Auth Name: a name you will recognize, like My AWS Account.
- Access Key ID: the Access key ID you copied from AWS.
- Secret Access Key: the Secret access key you copied from AWS.

Credentials you save in Daestro are encrypted with AES-256 when stored (at rest) and sent over TLS (in transit).

Click Save.

Your AWS account is now linked to Daestro.

Step 3: Create Your First AWS Compute Environment

A Compute Environment in Daestro is the virtual server where your jobs run.

In the Daestro Console, open Compute Environment.
Click Create New.
Configure the server:
- Name: something descriptive, like aws-us-east-prod.
- Cloud Auth: the AWS credential you just created (for example, My AWS Account).
- Instance Type: the size of the server. This sets CPU, RAM, and cost. For testing, t2.micro or t3.micro is a cheap starting point.
- Location: the AWS region to launch in (for example, us-east-1, eu-west-2).
- Storage (GB): the size of the server’s hard drive (EBS volume) in gigabytes.
- Assign Public IPv4: check this if the job needs to be reachable from the public internet. AWS now charges for public IPv4 addresses, so only enable it if you need it.
Click Create.

Daestro provisions an EC2 instance in your AWS account with those settings. When the status is Ready, you can assign jobs to it.