Khalid Khaliqi, Principal Architect
When developing any application intended to run somewhere other than your desktop, eventually, you’ll face the problem of how to pass some specific environmental variables for configuration. In a well-designed application, you don’t want to hard code parameters like API URLs, API Keys, or others that would change between environments like QA and Production. AWS offers multiple ways to inject environmental variables into your application, container, or EC2 instance, and they usually fall into one of two categories:
Unsecured Parameters – These variables do not need to be encrypted at rest. These include endpoints, boolean flags, and anything other than “secrets”.
Secured Parameters – These variables are sensitive and must be encrypted at rest and injected into your container or serverless application security. These include API Keys and other sensitive parameters you don’t want to fall into the hands of bad guys.
Furthermore, the implementation of HOW you pass environmental variables depends on the AWS service you are using. This article will focus on a workload implemented in Amazon ECS, but the same techniques apply equally well to other deployment approaches, such as EC2 and Lambdas.
We will focus on three different options for configuring secure and unsecured environmental variables:
AWS Console
Terraform
CloudFormation Template
We’ll conclude this write-up with some pros & cons of each method.
Let’s dive in!
Secrets Manager & AWS SSM (Parameter Store)
You have choices regarding injecting your environmental variables into your serverless app, container, or ec2 instance. Each service allows you to specify the variable as part of the service definition. The biggest issue you’ll run into with this approach is not securing your parameter OR if you want to manage your parameter centrally and have more control over its value, permissions, etc. Not to worry - there are two other services that you can use:
AWS SSM (Parameter Store)
AWS Parameter Store allows you to store parameters in a hierarchy, assign policies, and allow for encrypted values using AWS KMS, which is an excellent way of organizing and controlling access to parameters and referencing them in CloudFormation / Terraform, etc. Later, we’ll go through how to use SSM parameters as environment variables with ECS.
Cost: SSM manager is FREE to use with some limits.
Secrets Manager
AWS Secrets Manager has a similar functionality as Parameter Store, but some differences exist. Secrets Manager encrypts all values, and you cannot store an unsecured parameter. Secrets Manager can generate random passwords via SDK or CloudFormation templates and keep them in Secrets Manager. This feature is not available to SSM. Another feature of Secrets Manager is the ability to rotate secrets. If you are using RDS, the rotated secret propagates to RDS. If you are using another service, you can write custom logic via Lambdas to rotate secrets and propagate those changes to your service. And last but not least, you can share secrets across accounts using IAM roles.
Cost: Secrets Manager costs $0.40 to store a secret and $0.05 per 10k calls.
Now let’s keep digging into how to implement these services with AWS ECS.
AWS ECS
AWS ECS is a fully managed container service that allows you to deploy and manage your containerized applications (think dockerized apps). We’ll walk through how to define environmental variables in the task definition that is unsecured and not sensitive, or use SSM or Secrets Manager to pass sensitive variables. Of course, you could also take the approach of storing insecure configuration items in SSM, but for simplicity, we will demonstrate how to create them as part of the task definition. This reduces the need for the application to have a bootstrap process that kicks off to read the complete configuration data from SSM.
In order to keep this write-up focused on env. variables, we’ll assume you know how to create a docker image of your application, publish it to ECS and deploy it to ECS.
Unsecured Parameters
Let’s assume that you have a dockerized container and you deploy this application to two different AWS environments (QA and PROD) and want to pass the name of the environment to your docker container as ENV_NAME environmental variable. This straightforward case where ECS supports an array of key/value pairs in the task definition of ECS is documented here: AWS ECS Task Definition Options
AWS CLI
In the AWS CLI, you will create the environment tag when you register your task definition. We won’t get into the full definition of a task as that would be beyond the scope of this article. However, the specific command parameter we will use is the
–container-definitions value, which allows us to set key-value pairs associated with the environment in the definition of the container. The following is an example of what the command might look like (abbreviated as it won’t include all required tag definition parameters) to create an environment tag of QA.
String in that command would be where you can add your specific environment name (e.g., QA or PROD), or that could be passed into the CLI command as a parameter. This will add that ENV environment variable to the task when it starts up using the container definition.
Terraform
In your ECS Terraform definition, you’ll need to specify “environment” tag along with key/value pair like:
You should pass environment_name as input to your TF script and pass the actual value QA or PROD as part of your deployment, either manually or through CICD.
CloudFormation Template
Another Infrastructure as Code option for setting your ECS environment variables is AWS cloud formation. To implement the environment variable injection to the ECS task, you include them in the container definition stanza of the cloud formation template. Here is a representative example of what the relevant sections of the Cloud Formation template might look like.
The string on line 12 would be where you inject the environment you’re setting into the variable. This can be passed into the cloud formation as a defined parameter that can be used in place of the static value.
Secured Parameters
Here we can use Secrets Manager or SSM to pass sensitive data to our container. For example, assume we have a CONTENT_API_KEY environment variable with the secret API key to a content provider for our service deployed as an ECS managed container.
AWS CLI
Similarly to the unsecured parameters, we can inject the information for the application into the container definition from the command line. We will inject a reference to the secret by providing the ARN for either the SSM secure parameter or a Secrets Manager secret. We can also pass the container definition into the command line as a file containing a very complex definition. The following is a representative example of what this command line might look like for adding references to secrets.
The values of the secrets would need to be added to SSM or Secrets Manager either manually or via some other aspect of the CI/CD process to protect that information from being disclosed.
Terraform
In our task definition, we would define an array of key/value pairs named “secrets” and specify the name and ARN of our secret (SSM ARN or Secrets Manager). The following is an example of how this declaration might look.
Specific requirements need to be met, and the Execution Role for the task must provide the correct role so the task may access Secrets.
CloudFormation Template
In addition to Terraform and the AWS CLI, CloudFormation is a good alternative for managing the configuration of these ECS task secrets to be retrieved from either SSM or Secrets Manager. The following is an example of how to implement the creation of these secrets in the ECS task configuration.
As with the other methods, the appropriate roles and permissions need to be provided to allow the ECS tasks to retrieve the secrets.
Conclusion
You made it to the end! Now you know how to configure your environmental variables for different AWS workloads. There are a number of ways to handle providing parameters to your applications. In this article, we covered using environment variables to provide the information, which has the advantage of not needing to bootstrap access to something like SSM or Secrets Manager to get to the parameters. This also lends nicely to the parameters provided in the CI/CD process.
Comments