You can not select more than 25 topics
Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
284 lines
10 KiB
284 lines
10 KiB
---
|
|
layout: post
|
|
title: Using Policyfile Cookbooks
|
|
category: chef_infra
|
|
tags: [chef, cookbooks, policyfiles]
|
|
summary: Policy...what is it good for?
|
|
---
|
|
|
|
## Policyfile? What is that?
|
|
|
|
A Policyfile (aka Policyfile.rb) is a file that contains information about a
|
|
node's:
|
|
- Default source for fetching cookbooks
|
|
- Run list (or multiple run lists via `named_run_list`)
|
|
- Cookbook dependencies and sources
|
|
- Optional cookbook attributes
|
|
|
|
Here is an example Policyfile.rb:
|
|
|
|
```ruby
|
|
name 'mycorp_audit'
|
|
default_source :supermarket
|
|
cookbook 'mycorp_audit', path: '.'
|
|
run_list 'mycorp_audit::default'
|
|
```
|
|
|
|
This file is a key component in Policyfile cookbooks.
|
|
|
|
## What are Policyfile cookbooks and why should I use them?
|
|
|
|
Essentially, a Policyfile cookbook is a cookbook that uses a Policyfile.rb to
|
|
combine the functions of Berkshelf, Environments, and Roles into a single
|
|
artifact that can be promoted safely through the software development
|
|
lifecycle.
|
|
|
|
Without using Policyfiles, special care is needed in order to use Chef safely.
|
|
This is due to the shortcomings of runtime dependency solving.
|
|
|
|
Take for example the following scenario:
|
|
- You have 3 cookbooks: `cookbook_a`, `cookbook_b`, and `cookbook_c`
|
|
- `cookbook_a` is deployed and depends on `cookbook_b` with version `~>1.0.0`
|
|
- `cookbook_c` is created and depends on `cookbook_b` with version `~> 1.0.0`
|
|
- `cookbook_c` is tested/deployed (which deploys newest 1.X.X of `cookbook_b`)
|
|
- `cookbook_a` runs and breaks due to a bug in `cookbook_b` version `1.1.1`
|
|
|
|
Now, ideally, strict version pinning and adhering to [Semantic Versioning](https://semver.org/)
|
|
would prevent any mishaps...but that is a lot to trust to place on the
|
|
developer, the pipeline, and the upstream cookbook maintainer.
|
|
|
|
With Policyfile cookbooks, this is prevented because cookbooks are referenced
|
|
by hash and not version at runtime. In order to run a new set of cookbooks a
|
|
new Policyfile.lock must be deployed.
|
|
|
|
---
|
|
|
|
## Policyfile.lock
|
|
|
|
The Policyfile.lock (aka Policyfile.lock.json) is generated from a
|
|
Policyfile.rb and contains all the same information in it as the Policyfile.rb
|
|
in addition to a few other key items needed for safely deploying cookbooks.
|
|
|
|
For the sake of this section of the article, the two most important sections of
|
|
the Policyfile.lock are the `revision_id` which contains the hash of the entire
|
|
lock file and the `cookbook_locks` section.
|
|
|
|
These two items together ensure that cookbooks and recipes are applied
|
|
consistently regardless of the environment that Chef Infra is deployed/ran in.
|
|
|
|
Below is an example `cookbook_locks` section:
|
|
|
|
```json
|
|
"audit": {
|
|
"version": "7.5.0",
|
|
"identifier": "27e6ea8e7bc82f5ba44b77834f5472c96da9c27a",
|
|
"dotted_decimal_identifier": "11231419178928175.25794866915266388.126209453834874",
|
|
"cache_key": "audit-7.5.0-supermarket.chef.io",
|
|
"origin": "https://supermarket.chef.io:443/api/v1/cookbooks/audit/versions/7.5.0/download",
|
|
"source_options": {
|
|
"artifactserver": "https://supermarket.chef.io:443/api/v1/cookbooks/audit/versions/7.5.0/download",
|
|
"version": "7.5.0"
|
|
}
|
|
}
|
|
```
|
|
|
|
As you can see above, not only is useful information such as version and origin
|
|
of the cookbook provided, but also referenced is the exact hash of the upstream
|
|
cookbook that is being used.
|
|
|
|
> In this case, the `audit` cookbook with hash
|
|
> `27e6ea8e7bc82f5ba44b77834f5472c96da9c27a`
|
|
|
|
This is critical as it ensures that not only are we running the correct version
|
|
of the cookbook, but also that the content of that cookbook has not changed
|
|
since the Policyfile.lock was created.
|
|
|
|
## Benefits over Berkshelf (i.e. runtime dependency solving)
|
|
|
|
Since the Policyfile.lock is computed prior to uploading to the Chef Infra
|
|
Server there is no need to compute cookbook dependencies at runtime. This has
|
|
several emergent benefits. Firstly, by moving this computation to the
|
|
workstation (or pipeline), Chef Infra is saved from having to do the same
|
|
computation(s) on each run on every node.
|
|
|
|
More importantly, since dependency resolution has already been completed and
|
|
cookbooks are referenced via hash, it is impossible to unintentionally run a
|
|
cookbook in production.
|
|
|
|
## Deploying/Promoting Policyfiles
|
|
|
|
Policyfiles are deployed to the Chef Infra Server into what are known as Policy
|
|
Groups. Conceptually, these Policy Groups take the place of Environments.
|
|
|
|
This is best illustrated with the output from `chef show-policy`:
|
|
|
|
```text
|
|
mycorp_audit
|
|
=======
|
|
* dev: 98ed3a52a5
|
|
* staging: 98ed3a52a5
|
|
* production: 98ed3a52a5
|
|
```
|
|
|
|
We can see here that the `mycorp_audit` policy with the revision ID
|
|
`98ed3a52a5` is currently deployed in the dev, staging, and production policy
|
|
groups.
|
|
|
|
When bootstrapping a node, the `policy_name` and `policy_group` are specified,
|
|
this ensures that the correct policy is applied during runtime.
|
|
|
|
### Promoting Policyfiles through your environments (Policy Groups)
|
|
|
|
This is done like via `chef push POLICY_GROUP POLICY_NAME`.
|
|
|
|
See:
|
|
|
|
```text
|
|
$ chef update # This updates the Policyfile.lock to use the latest deps
|
|
OUTPUT REMOVED TO SAVE SPACE
|
|
|
|
$ chef push dev mycorp_audit
|
|
Uploading policy mycorp_audit (bfd3af4697) to policy group dev
|
|
Using mycorp_audit 0.3.0 (f37cdfc3)
|
|
Using audit 7.5.0 (27e6ea8e)
|
|
|
|
$ chef show-policy
|
|
mycorp_audit
|
|
=======
|
|
* dev: bfd3af4697
|
|
* staging: 98ed3a52a5
|
|
* production: 98ed3a52a5
|
|
|
|
$ chef push staging mycorp_audit
|
|
Uploading policy mycorp_audit (bfd3af4697) to policy group staging
|
|
Using mycorp_audit 0.3.0 (f37cdfc3)
|
|
Using audit 7.5.0 (27e6ea8e)
|
|
|
|
$ chef show-policy
|
|
mycorp_audit
|
|
=======
|
|
* dev: bfd3af4697
|
|
* staging: bfd3af4697
|
|
* production: 98ed3a52a5
|
|
|
|
$ chef push production mycorp_audit
|
|
Uploading policy mycorp_audit (bfd3af4697) to policy group production
|
|
Using mycorp_audit 0.3.0 (f37cdfc3)
|
|
Using audit 7.5.0 (27e6ea8e)
|
|
|
|
$ chef show-policy
|
|
mycorp_audit
|
|
=======
|
|
* dev: bfd3af4697
|
|
* staging: bfd3af4697
|
|
* production: bfd3af4697
|
|
```
|
|
|
|
As you can see the policy with a `revision_id` of `bfd3af4697` is being moved
|
|
through dev, staging, and production.
|
|
|
|
For more info see: [https://docs.chef.io/ctl_chef.html#policyfile-commands](https://docs.chef.io/ctl_chef.html#policyfile-commands)
|
|
|
|
---
|
|
|
|
## FAQ
|
|
|
|
### How can I create a Policyfile cookbook?
|
|
|
|
Older versions of the ChefDK/Chef Workstation support generating Policyfile
|
|
cookbooks using `chef generate cookbook mycorp_app -P`. Newer versions will
|
|
generate Policyfile cookbooks by default.
|
|
|
|
An easy way to tell if you are using a Policyfile cookbook is that it will not
|
|
contain a Berksfile.
|
|
|
|
### How can I migrate from the current way I'm doing cookbooks?
|
|
|
|
At a high level, if you are using a Berksfile:
|
|
- Swap it out for a Policyfile.rb
|
|
- Use `chef install/push` instead of `berks install/upload`
|
|
- Run `chef-client` on your nodes with `policy_name`/`policy_group` specified
|
|
- Example: `chef-client -j policy.json`
|
|
- Search [here](https://docs.chef.io/ctl_chef_client.html) for `Specify a policy`
|
|
|
|
This is a safe migration since only nodes bootstrapped to use Policyfiles will
|
|
consume them. Also, since cookbooks used via Policyfiles are stored in a
|
|
different way than other cookbooks, you cannot accidentally break the old way
|
|
of using Chef.
|
|
|
|
### Why don't I see Policyfile cookbooks with `knife cookbook list`?
|
|
|
|
Cookbooks consumed by Policyfiles are stored differently on Chef Infra Server
|
|
and thus do not show when using Knife. Instead you should use:
|
|
|
|
`chef show-policy POLICY_NAME POLICY_GROUP`
|
|
|
|
### Policyfile cookbooks vs plain Policyfiles, which is better?
|
|
|
|
Ultimately, the Policyfile.lock is the source of truth. With that in mind, it
|
|
is possible to build a Policyfile.lock using only a Policyfile.rb.
|
|
|
|
In theory, you would build the top level role cookbook as described in
|
|
[this](https://blog.chef.io/2017/02/14/writing-wrapper-cookbooks/) blog post on
|
|
writing wrapper cookbooks, then call that cookbook (and set any desired
|
|
attributes) via a Policyfile.rb, and upload the resulting lock to the Chef
|
|
Infra Server. This removes the need for a top level cookbook entirely.
|
|
|
|
That being said, I personally prefer that the top level artifact be a
|
|
Policyfile cookbook. This way, those that are not familiar with Policyfiles can
|
|
acclimate to the new pattern and you can set attributes via logic (which cannot
|
|
be done in a Policyfile.rb)
|
|
|
|
### Attributes, should I set them in the Policyfile.rb?
|
|
|
|
I personally prefer only specifying dependencies and a `run_list` in the
|
|
Policyfile.rb. I prefer this for many reasons.
|
|
|
|
Mainly:
|
|
- It matches the older Chef patterns so those who usually look in
|
|
recipes/attribute files for attributes will not be surprised
|
|
- It sets the attributes near where they are used
|
|
- It's more intuitive than methods such as hoisting (personal opinion)
|
|
|
|
Also, since the Policyfile.lock.json is compiled before runtime it is not
|
|
possible to set dyanmic attributes that you would normally set during a Chef
|
|
Infra run (e.g. changing behavior based on the domain, region, or Ohai).
|
|
|
|
An example of how I do this is as follows:
|
|
|
|
```ruby
|
|
case node.policy_group
|
|
when 'dev'
|
|
node.default['my_cookbook']['api_url'] = 'https://dev.example.com/api'
|
|
when 'staging'
|
|
node.default['my_cookbook']['api_url'] = 'https://staging.example.com/api'
|
|
when 'production'
|
|
node.default['my_cookbook']['api_url'] = 'https://prod.example.com/api'
|
|
end
|
|
```
|
|
|
|
### Chef Infra Server, do I need it if I'm using Policyfiles?
|
|
|
|
Moving dependency resolution out of runtime has one other major benefit. If
|
|
you are not relying on state stored in the Chef Infra Server (e.g. data bags,
|
|
chef search, etc) then you can remove the need for the Chef Infra Server all
|
|
together.
|
|
|
|
By using `chef export` you can create an artifact that contains the policy and
|
|
all dependent cookbooks. This artifact can then be distributed and executed via
|
|
`chef-client`. In fact, this is a core factor in how Effortless Infra and
|
|
Test Kitchen (when using Policyfiles) functions.
|
|
|
|
### How do I test Policyfile cookbooks in Test Kitchen
|
|
|
|
By default, the generated kitchen.yml will do what you need. If you'd like to
|
|
test multiple suites though, please see the following section of the Chef docs:
|
|
|
|
[https://docs.chef.io/policyfile.html#test-w-kitchen](https://docs.chef.io/policyfile.html#test-w-kitchen)
|
|
|
|
---
|
|
|
|
## Supporting Resources
|
|
- [More justification for Policyfiles](https://docs.chef.io/policyfile.html)
|
|
- [Policyfile.rb documentation](https://docs.chef.io/config_rb_policyfile.html)
|
|
- [Chef Policyfile commands](https://docs.chef.io/ctl_chef.html#policyfile-commands)
|
|
|