WebDriver and AWS: Use AWS Parameter Store to Guard Your Secrets.

WebDriver and AWS: Use AWS Parameter Store to Guard Your Secrets.

One of the issues with running any software product on a third-party service like AWS or Azure is the need to push sensitive bits of information to those environments. Things like passwords and API access tokens should be closely guarded secrets for sure.

So what do you do when your automated tests need to handle login details for test users. Though you can commit your secrets to configuration files to your source control system like GitHub, you really don’t want to do that.

AWS’ Parameter Store is a useful and very secure tool to store your pieces of secret information. It will even let your code programatically access those secrets freeing you from passing them around as parameters.

Here’s how to do it.

Save Your Secrets to Parameter Store

First let’s take a look at how to save a secret to Parameter Store. You can’t read them if they’re not already there, right?

When it comes to automated tests, we want to save the secrets once, but read them over and over again. In such a case, I just create a bash shell script to handle the saving of the secrets.

This script reads a simple text file in this format:

Test environment config file: StagingServer.txt

BROWSER=chrome
USERNAME=Joey
PASSWORD=SuperSecretPassword

With that done, we just run a script like this one to save the secrets to AWS. It uses the put-parameter command of AWS’s CLI tool.

I’m not going to dwell over this script as the point of this post is to look at how to read them in your Ruby code.

# This script will automatically add and update secrets in the AWS Parameter Store.
# Example: "./set_parameters_to_aws.sh test_environment.txt" will store the secrets at path /WebDriver_Tests/my_parameters
#
# Check if an input file was specified
if [ $# -eq 0 ] || [ ! -f $1 ]; then
echo
echo "No test environment specified or config file not found. Example usage '$ set_parameters_to_aws.sh tesT_environment.txt'"
echo
exit 1
fi
# Strips away everything leaving the name of the input filename (in case you use a full path)
temp=$(sed -e 's#.*/\(\)#\1#' <<< $1)
test_env=$(echo "${temp%%.*}")
# We want to break each line at the "=" to get our key and our value
IFS='='
while read -r key value;
do
# Skip comment lines
if [[ $key != "#"* ]]; then
cat >./file.json <<EOF
{
"Type": "SecureString",
"Name": "/WebDriver_Tests/${test_env}/${key}",
"Value": "${value}",
"KeyId": "alias/test-integration-secrets",
"Overwrite": true
}
EOF
echo "Saving parameter: " $key
aws ssm put-parameter --cli-input-json file://file.json
rm file.json
fi
done < $1
echo
echo "Done saving secrets to AWS."
echo "You can find them at https://console.aws.amazon.com/ec2/v2/home?region=us-east-1#Parameters:Path=%5BOneLevel%5D/WebDriver_Tests/${test_env};sort=Name$/"
echo

Retrieve The Secrets At Runtime

Now we can look at reading the secret into our tests without having to save them as configuration files. Parameter Store has an API allowing us to grab them programatically.

AWS has a couple methods to do this get_parameter and get_parameters_by_path.

1) Using get_parameter

First we setup our AWS credentials. The AWS access key and AWS secret keys seem like the perfect things to store in Parameter Store. Trouble is you kinda need them off the bat to even connect to AWS.

You can setup the shell environment your Ruby code will be running in to get around this, but for now hard-coding them will do.

    Aws.config.update({
      region: 'us-east-1',
      credentials: Aws::Credentials.new('AWS_ACCESS_KEY_GOES_HERE', 'AWS_SECRET_KEY_GOES_HERE')
    })
    ssm = Aws::SSM::Client.new

With the AWS client setup, we can make the call to retrieve a parameter.

    params = {
      names: ["MyTestParameter"],
      with_decryption: true
    }

    resp = ssm.get_parameters(params)

Now you can access the parameters with resp.parameters[0].value.

get_parameter is great on its own, but can be slow when trying to retrieve several secrets because it has to make a connection and authenticate each time.

A faster way is to grab several parameters in the same call. That’s where get_parameters_by_path comes in.

2) Using get_parameters_by_path

get_parameters_by_path only returns 10 parameters at a time, so we need to loop through our calls using a next_token variable.

Each call to get_parameters_by_path includes a next_token field. If next_token contains a value, we hit get_parameters_by_path again, passing in next_token, allowing us to get the next set of secrets. We loop over this until next_token is nil meaning there are no more secrets in the particular path.

With each call, we get a bunch of extra data with each secret. All we’ll really care about are the parameters themselves, so we’ll store those in a class variable called @@ssm_parameters.

    next_token = nil

    while true
      params = {
        path: "/WebDriver_Tests/#{test_environment}",
        recursive: true,
        with_decryption: true
      }
      params[:next_token] = next_token unless next_token.nil?

      resp = ssm.get_parameters_by_path(params)

      unless resp.parameters.empty?
        (@@ssm_parameters << resp.parameters).flatten!
      end

      break if resp.next_token.nil?

      next_token = resp.next_token
    end

Next we just wrap all of this up in a class called AWSHelper and throw in a helper method we use to retrieve the parameters from the @@ssm_parameters class variable.

require 'aws-sdk-ssm'
class AWSHelper
@@ssm_parameters = []
@@test_environment = nil
# Reads and stores Parameter Store values for a particular path.
def self.load_ssm_parameters(test_environment)
@@test_environment = test_environment
Aws.config.update({
region: 'us-east-1',
credentials: Aws::Credentials.new('AWS_ACCESS_KEY_GOES_HERE', 'AWS_SECRET_KEY_GOES_HERE')
})
ssm = Aws::SSM::Client.new
next_token = nil
while true
params = {
path: "/WebDriver_Tests/#{test_environment}",
recursive: true,
with_decryption: true
}
params[:next_token] = next_token unless next_token.nil?
resp = ssm.get_parameters_by_path(params)
unless resp.parameters.empty?
(@@ssm_parameters << resp.parameters).flatten!
end
break if resp.next_token.nil?
next_token = resp.next_token
end
end
# Returns the value of a particular Parameter Store parameter.
def self.get_parameter_value(name)
param = @@ssm_parameters.each.select { |ssm_parameter| ssm_parameter.name == "/WebDriver_Tests/#{@@test_environment}/#{name}"}
param[0].value
end
end

Here’s the usage of this particular method:

AWSHelper.load_ssm_parameters('StagingServer')

and to retrieve a parameter, we use AWSHelper.get_parameter_value(‘USERNAME’), which will return “Joey”.

AWSHelper.get_parameter_value('PARAMETER_NAME_GOES_HERE')

So AWSHelper.get_parameter_value(‘USERNAME’) will return “Joey”.

Easy-peasy.

 

Leave a Reply