Job Queue

A Job Queue is a powerful tool that helps you organize and control how your jobs are run. Think of it as a waiting line for your tasks, where you can set the rules. A queue decides which jobs are most important, how many can run at the same time, and what kind of servers to use.

By using Job Queues, you can make sure your most critical tasks are always run first and that you are using your server resources efficiently.

How to Create a Job Queue

1. Go to the Job Queue page in the Daestro Console.
2. Click on Create Job Queue.
3. Fill out the form with the following details:
   • Name: A unique name for your queue. It can only contain letters, numbers, and hyphens.
   • Description (Optional): A brief note to remind you what this queue is for.
   • Priority: A number from 1 to 1000. Jobs in a queue with a lower number (like 10) will run before jobs in a queue with a higher number (like 100).
   • Max Concurrency: The maximum number of jobs that can run at the same time in this queue. Set it to 0 for no limit.
   • Max Compute Spawn (Required): 0 means no limit. Limits the number of instances that can be spawned in a Job Queue.
   • Max Idle Time (in seconds): This controls costs. It’s the number of seconds a cloud server can sit idle (with no job to run) before Daestro automatically shuts it down.
   • SSH Public Key (Optional): User can add their SSH public key here to log into the Compute Spawns. (e.g. ssh daestro@<compute-spawn-ip>)

After you save it, the new Job Queue is created but will be in a disabled state. You need to add a Compute Environment to it before you can enable it.

Adding a Compute Environment to a Queue

A Job Queue needs to know what kind of server to use for its jobs. You connect it to one or more of your Compute Environments .

From the Job Queue’s detail page, you can easily add or remove Compute Environments. If you plan to run many jobs, it’s a good idea to add multiple Compute Environments to a single queue. Daestro will then spread the work across them.

Note: A Job Queue cannot be enabled and will not process any jobs until it has at least one Compute Environment linked to it.

How to Update a Job Queue

You can change the settings of a Job Queue at any time from its detail page. You can update its:

• Description
• Priority
• Max Concurrency
• Max Idle Time
• Active state (enabling or disabling it)

How to Delete a Job Queue

You can delete a Job Queue if you no longer need it. For safety, you can only delete a queue if it meets two conditions:

1. There are no active servers (Compute Spawns ) currently running for it.
2. No scheduled Cron Job is set to use this queue.