We live in a world where we need to remember that OS or base packages on the instance need to be updated because security is critically important.
When in our ENV, there are a lot of instances. Sometimes it’s easy to make a mistake or we can forget to update one of the instances.
With AWS SSM we can schedule automatic updates for our EC2 instances.
AWS Systems Manager Agent (SSM Agent) is Amazon software that can be installed and configured on an Amazon EC2 instance, an on-premises server, or a virtual machine (VM). SSM Agent makes it possible for Systems Manager to update, manage, and configure these resources. The agent processes requests from the Systems Manager service in the AWS Cloud, and then runs them as specified in the request. SSM Agent then sends status and execution information back to the Systems Manager service by using the Amazon Message Delivery Service (service prefix: ec2messages).
For this article, I will create a small ENV. For better usage of SSM, we need to add correct tags to EC2 instances and install SSM agent on these instances.
Our update plan:
Before updating instances, AMI Image will be automatically created as a backup solution.
Staging instances will be patched on Saturday at 06:00 AM UTC
Install only Critical and Important updates on the instances
Getting Set Up
In Staging ENV there will be two EC2 instances using Ubuntu 14.04 OS (called stageweb01 and stageweb02).
If we have a large environment, tags are helpful. For our article, we will attach 3 tags to instances:
From these tags we know that instance is from environment staging using OS ubuntu 14 and this instance is part of the Patch Group ubuntu_staging. This is important later on.
SSM Agent is installed by default on the following AMIs:
Windows Server 2003-2012 R2 AMIs published in November 2016 or later
By default, AWS Systems Manager doesn’t have permission to perform actions on your instances. You must grant access by using an AWS Identity and Access Management (IAM) instance profile. An instance profile is a container that passes IAM role information to an Amazon Elastic Compute Cloud (Amazon EC2) instance at launch. You can create an instance profile for Systems Manager by attaching one or more IAM policies that define the necessary permissions to a new role or to a role you have already created.
For following along in this article, you will need two IAM Roles.
The first IAM Role will be called role-for-ssm. This role will be attached to our two instances.
Go to the AWS Console -> IAM -> Roles and hit the “Create role” button
Create a role called role-for-ssm and attach the AWS manager policy called: AmazonEC2RoleforSSM
After you read this article and complete tasks, IAM role ssmautomationservice_role should have an attached policy called “AmazonSSMAutomationRole” and one inline policy.
Check your environment, we need to be sure that SSM agent is working on instances:
For testing purposes, this entire env is launched in eu-central-1 in one AZ.
(Of course for high availbility needs, always try to launch multiple instances in different AZ)
Every instance has added an IAM Role for SSM and has installed SSM agent.
Please go to Managed Instances in AWS Systems Manager to check if we correctly added the IAM Role called role-for-ssm and if SSM Agent is working.
We should see our entire ENV there.
Create a Maintenance Window
Now we need to create a Maintenance Window for our staging instances:
Please go to AWS Console -> Systems Manager -> Maintenance Windows and select create maintenance window.
<pre><code>Name: Ubuntu_StagingDescription: Maintenance Window for staging instances with Ubuntu OSCheck Allow unregistered targetsCheck Cron schedule builderSelect Every Saturday at 06:00Duration: 1 hourStop initiating tasks: 0 hourSchedule timezone: Etc/UTC</code></pre>
Create a Patch baseline
A patch baseline defines which patches are approved for installation on your instances. You can specify approved or rejected patches one by one. You can also create auto-approval rules to specify that certain types of updates (for example, critical updates) should be automatically approved. The rejected list overrides both the rules and the approved list.
Now click Create patch baseline, and if everything is fine we should see:
Assign Patch Group tag with custom Patch baseline
Now, this step is very important, we need to assign our custom Patch baseline with Patch Group tag.
Please, go to AWS Console -> Systems Manager -> Patch Manager
Find our Patch baseline for Ubuntu, Please use search by name: ubuntu-patch-baseline
Select our Patch baseline and click on Action -> Modify patch groups
In Patch groups, add our tag: staging_ubuntu
Now click Close.
In Patch groups, we should see our Patch group, staging_ubuntu, registered with ubuntu-patch-baseline.
Please remember: If we don’t specify patch baseline to patch group, AWS will use the default patch baseline for the desired OS.
Register targets in Maintenance Window
We want SSM to update only the instances with our tag Patch Group:staging_ubuntu
Please go to AWS Systems Manager -> Maintenance windows
Search Maintenance window by name, ubuntu_staging, and click on the Window ID of the ubuntu_staging.
We should see:
Now we need to create maintenance window targets.
Click the Targets tab and register target.
<pre><code>Target name: staging-targetsDescription: window target only for staging instancesIn Targets selection, please choose:Instance Tags and type:Tag key: Patch Group Tag value: staging_ubuntu</code></pre>
Finally, click on Register target
We should see something like this:
Now we have registered our instances with Maintenance Window.
So, if we add new instances with correct tags and IAM roles, new instances will be automatically added to this Maintenance Window.
Create tasks in Maintenance Window
Now we need to add tasks in the Maintenance window for staging ENV.
Please go to the AWS Systems Manager -> Maintenance windows.
Search the Maintenance window by name: ubuntu_staging, and click on Window ID of the ubuntu_staging.
Now click on Tasks.
The first thing we need to do is register the Automated task for creating AMI image before starting patching. This will be our backup plan. If there is something bad after patching, we can revert changes, launching a new instance from backup AMI.
To do this, please:
Click on Register tasks and select Register Automation task.
<pre><code>Name: Create_AMI_imageDescription: This task will create an AMI image of the instances before patching</code></pre>
In Automation document, please select AWS-CreateImage In Document version, select Latest version at runtime. In Task priority, type: 0
Task priority 0 - it’s the highest priority, it will be executed as the first task In Targets, we need to select staging-targets registered targets groups
Here we can set how many instances can run this task at once.
If we have a very big environment, we can use a percentage, but, in our case, we will set 1 instance at a time.
In an error threshold, we set the value when this task should be stopped. So, if errors appear, this task will be stopped.
I recommend using 1.
Now select a custom service role that allows Maintenance Windows to interact with other AWS resources on your behalf.
In our case, it will be Role: ssm_automation_service_role
The last step to finish configuring automation for creating an AMI image is setting Input parameters.