Understanding AWS Batch
Amazon Web Services AWS Batch enables us to run batch computing workloads on AWS. These workloads can be at any scale. AWS Batch allows the end-user to plan, schedule, and execute batch jobs while being able to control costs.
Batch jobs, by their nature, run asynchronously and have inter-dependencies between jobs. These interdependencies builds the ability to control scheduling and sequencing very important. Amazon Web Services Batch provides the tools to control the scheduling and sequencing of jobs.
Amazon Web Services Batch Implementation
Some of the terminologies used in AWS Batch that should be familiar to users of on-premises batch systems:
- Job
- Job definition
- Job queue
- Scheduler
- Compute environment
Amazon Web Services Batch executes Jobs which is the unit of work. Such that jobs can be executed as AWS Lambda functions or as containerized applications running on Amazon EC2.
Job definitions are a similar concept to Amazon ECS task definitions—they specify how jobs are to be run. Some of the attributes specified in a job definition include vCPU requirements, memory requirements, mount points, and environmental variables, among other items.
Process of Implementing Amazon Web Services Batch Job
Following are the steps in implementing a batch job
- Define a job.
- Configure the computing environment.
- Configure the job queue.
- Submit a job.
Amazon Web Services Batch Management
Note that there are service limits to AWS Batch. We can have up to 10 compute environments, 5 job queues, and 3 compute environments per job queue. Such that these limits can be changed via the AWS Management Console or the AWS CLI. We can also have up to 20 job dependencies and 1,000,000 jobs in a submitted state. The maximum job size is 20 KB. Note, these three limits cannot be changed.
Monitoring Amazon Web Services Batch
We can monitor the AWS Batch environment either through the CLI or the AWS Batch dashboard. Logs for jobs (for instance, STDERR and STDOUT) are available in the AWS Management Console and can be written to Amazon CloudWatch Logs. We can also use SSH to access the Amazon EC2 instances that have been spun up to process the Amazon Web Services Batch job.
Amazon Web Services Batch Costs
There is no extra costs for AWS Batch. Such that the only costs are for the Amazon EC2 resources or the AWS Lambda resources used. It is possible to use Spot pricing for all of the compute instances in order to minimize costs.
Amazon Web Services Batch Security
AWS Batch uses IAM to control access to AWS resources that are being used. Through IAM, we can define policies for different users in the organization to have different levels of access. Since we can control the Amazon VPC in which we create the compute resources, we can implement additional layers of security with the use of private subnets, network ACLs, and security groups.
Get ready to boost your learning and prepare for AWS SysOps Administrator Associate with hundreds of practice tests. Try Now!