To me, an important part of being an effective engineer is utilizing the best tool for the job. Most times you’ll need to orchestrate a bunch of them together. At Civis Analytics, we use Ansible for a myriad of dissimilar configuration management tasks: spin up AWS CloudFormation templates, install application dependencies, configure AWS CloudWatch alarms, etc. We’ve been using Ansible for a while; we’re quite happy with it and have contributed back to the project. To authenticate Ansible to AWS, you can use an AWS service called Identity and Access Management (IAM). This service provides role-based access controls from one centralized console. Recently, we adopted a new-ish AWS IAM Best Practice: Use IAM Roles.
We’ve open sourced the iam-role-injector script on github.
Using Multiple AWS Accounts for different kinds of resources is becoming an increasingly popular option, and AWS is really embracing it. While having multiple accounts completely isolates your different environments and resources, it also presents an administrative problem: how to distribute and control access. You don’t want to manage multiple users for the same person in different accounts, nor do you want to give out shared credentials. IAM provides the usual swath of access control objects: Users, Groups, Roles and Policies. Users can have their own permissions or ‘assume’ a role and temporarily gain authorization to access whatever that role can. Roles can be used by users from different accounts, and multiple users can assume the same role.
Now, the federated access that IAM roles provide are temporary. The credentials expires a short time after they are requested. How does this work? When requested by an authorized user, the Security Token Service (STS) creates temporary IAM credentials that the user can send with subsequent requests to utilize those elevated permissions. STS also allows the use of Multi-Factor Authentication.
For Civis Analytics, using IAM Roles made a heck of a lot of sense, but we had to crush a few implementation challenges. We had multiple codebases that were not designed to perform a role assumption on their own. We figured the best way to utilize IAM Roles would be to feed the temporary credentials in via an already in-place mechanism: environment variables. Many of our AWS calls use the Boto library, the Python-flavored SDK for AWS. Boto looks in a few places for AWS credentials and looks through these in a defined order. It checks the environment variables first. If we set our new keys there, we can guarantee Boto will use those credentials. It’s important to note that a lot of other tools follow this same paradigm (aws cli, ruby aws sdk, etc).
So, we use the AWS CLI to assume a role. After successful completion, a new set of credentials are returned to you: A SecretAccessKey, SessionToken, and an AccessKeyID. You must then manually export these into your environment. If you used the method described in the above paragraph you also need to perform some manual cleanup in the form of removing the temp credentials when they have expired.
That gets tedious fast, simply because the keys are ephemeral. So the obvious choice was to script this; to have a single command make the call, parse and export the result to the environment. Alongside that, the script needs to handle two other cases: expired role credentials and the potential overwriting of original credentials. The latter is due to the ability to store your credentials in environment variables in the first place, which would be overwritten by our process.
As I mentioned earlier, we’ve open sourced the script and it is now available on github. Please give it a try, and open an issue or pull request if you have suggestions on how we can improve it.