こしごぇ(B)

旧:http://d.hatena.ne.jp/koshigoeb/

Chef 実践入門(3) テスト

Chef実践入門 ~コードによるインフラ構成の自動化 (WEB+DB PRESS plus)

Chef実践入門 ~コードによるインフラ構成の自動化 (WEB+DB PRESS plus)

写経しつつ現状にあわせて修正なども。継続的インテグレーションは略。

Test Kitchen

  • CentOS 5.10(x86_64), 6.5(x86_64) で動作
  • analog コマンドが /usr/bin/analog に配置され実行可能
  • Analog のバージョンは 6.0

クックブック作成

$ knife cookbook create analog -o .
$ cd analog

レシピ

$ cat <<\EOF > recipes/default.rb
case node[:platform]
when 'centos'
  platform_version = node[:platform_version].to_f

  if platform_version >= 6.0
    rpmfile = 'analog-6.0.4-1.x86_64.rpm'
  elsif platform_version >= 5.0
    rpmfile = 'analog-6.0.4-1.el5.i386.rpm'
  else
    raise 'This recipe can not be applied to this environment!!'
  end

  remote_file "#{Chef::Config[:file_cache_path]}/#{rpmfile}" do
    source "http://www.iddl.vt.edu/~jackie/analog/#{rpmfile}"
  end

  package 'perl' do
    action :install
  end

  package 'analog' do
    action :install
    source "#{Chef::Config[:file_cache_path]}/#{rpmfile}"
    provider Chef::Provider::Package::Rpm
  end
end
EOF

Test Kitchen インストール

$ cat <<EOF > Gemfile
source 'https://rubygems.org'

gem 'test-kitchen', '~> 1.2.0'
gem 'kitchen-vagrant', group: :integration
gem 'berkshelf'
EOF
$ bundle
$ bundle exec kitchen init
      create  .kitchen.yml
      create  test/integration/default
$ vi .kitchen.yml
$ cat .kitchen.yml
---
driver:
  name: vagrant

provisioner:
  name: chef_solo

platforms:
  - name: centos-6.5
  - name: centos-5.10

suites:
  - name: default
    run_list:
      - recipe[analog::default]
    attributes:

テスト

Bats でテストを記述

$ cat <<\EOF test/integration/default/bats/basic_test.bats
@test "ファイルが正しい場所にインストールされている" {
    ls -la /usr/bin/analog
}

@test "ファイルに実行権限がある" {
    test -x /usr/bin/analog
}

@test "ファイルは正しいバージョンである" {
    /usr/bin/analog 2>/dev/null | grep "analog 6.0"
}
EOF

minitest でテストを記述

$ cat <<\EOF > test/integration/default/minitest/analog_spec.rb
require 'minitest/autorun'

describe 'Winning' do
  it 'should exist' do
    assert File.exists? '/usr/bin/analog'
  end

  it 'should be executable' do
    assert File.executable? '/usr/bin/analog'
  end

  it 'shouls be right version' do
    assert system("/usr/bin/analog 2>/dev/null | grep 'analog 6.0' 1>/dev/null")
  end
end
EOF

serverspec でテストを記述

$ cat <<\EOF > test/integration/default/serverspec/localhost/default_spec.rb
require 'serverspec'

include Serverspec::Helper::Exec
include Serverspec::Helper::DetectOS

RSpec.configure do |c|
  c.before :all do
    c.os = backend(Serverspec::Commands::Base).check_os
  end
  c.path = '/sbin:/usr/sbin'
end

describe package('analog') do
  it { should be_installed }
end

describe file('/usr/bin/analog') do
  it { should be_file }
  it { should be_mode 755 }
end

describe command("/usr/bin/analog 2>/dev/null | grep 'analog 6.0'") do
  it { should return_exit_status 0 }
end
EOF

テストを実行

$ bundle exec kitchen test

kitchen test は以下のコマンドを組み合わせたもの。

  1. kitchen create
  2. kitchen setup
  3. kitchen converge
  4. kitchen verify
  5. kitchen destroy

毎回インスタンス作成から実行すると時間がかかるので、converge -> verify を繰り返すと良いとの事。

  • --destroy=never
  • --destroy=always

Chef 実践入門(2) Vagrant

Chef実践入門 ~コードによるインフラ構成の自動化 (WEB+DB PRESS plus)

Chef実践入門 ~コードによるインフラ構成の自動化 (WEB+DB PRESS plus)

続続・写経。

Vagrant Provision

Vagrantfile

diff --git a/Vagrantfile b/Vagrantfile
index 660dfe6..d169bd9 100644
--- a/Vagrantfile
+++ b/Vagrantfile
@@ -11,6 +11,7 @@ Vagrant.configure(VAGRANTFILE_API_VERSION) do |config|

   # Every Vagrant virtual environment requires a box to build off of.
   config.vm.box = "opscode-centos-6.5"
+  config.vm.box_url = 'http://opscode-vm-bento.s3.amazonaws.com/vagrant/virtualbox/opscode_centos-6.5_chef-provisionerless.box'

   config.vm.define :node01
   config.vm.define :node02
@@ -99,6 +100,28 @@ Vagrant.configure(VAGRANTFILE_API_VERSION) do |config|
   #   # You may also specify custom JSON attributes:
   #   chef.json = { mysql_password: "foo" }
   # end
+  config.omnibus.chef_version = :latest
+  config.vm.provision :chef_solo do |chef|
+    chef.cookbooks_path = './cookbooks'
+    chef.json = {
+      nginx: {
+        env: 'ruby'
+      },
+      fluentd: {
+        installer: 'rpm'
+      },
+      mysql: {
+        server_root_password: 'rootpass'
+      }
+    }
+    chef.run_list = [
+      'recipe[yum-epel]',
+      'recipe[nginx]',
+      'recipe[mysql::server]',
+      'recipe[mysql::client]',
+      'recipe[td-agent]',
+    ]
+  end

   # Enable provisioning with chef server, specifying the chef server URL,
   # and the path to the validation key (relative to this Vagrantfile).

Berkshelf

source "https://supermarket.getchef.com"

cookbook 'yum-epel'
cookbook 'apache2'
cookbook 'mysql'
cookbook 'git'
cookbook 'nginx'
cookbook 'td-agent', git: 'https://github.com/treasure-data/chef-td-agent.git'

※ 写経した Vagrantfile の recipe が行方不明なので berks でごまかす。

$ rm -rf ./cookbooks
$ berks vendor ./cookbooks

vagrant-omnibus プラグイン

$ vagrant plugin install vagrant-omnibus

プロビジョニング実行

起動時

$ vagrant up --provision

随時

$ vagrant provision

Sahara

$ vagrant plugin install sahara

サンドボックスモード

sandbox モードが有効なときに仮想マシンに対して与えた変更は、commit コマンドでコミットしない限り rollback コマンドで巻き戻す事ができる。

$ vagrant sandbox on
[node02] Starting sandbox mode...
0%...10%...20%...30%...40%...50%...60%...70%...80%...90%...100%
[node01] Starting sandbox mode...
0%...10%...20%...30%...40%...50%...60%...70%...80%...90%...100%
$ vagrant sandbox status
[node02] Sandbox mode is on
[node01] Sandbox mode is on

ロールバック

$ vagrant ssh -c "sudo yum install flac -y" node01
$ vagrant ssh -c "sudo rpm -aq | grep flac" node01
flac-1.2.1-6.1.el6.x86_64
Connection to 127.0.0.1 closed.
$ vagrant sandbox rollback
[node02] Rolling back the virtual machine...
0%...10%...20%...30%...40%...50%...60%...70%...80%...90%...100%
0%...10%...20%...30%...40%...50%...60%...70%...80%...90%...100%
[node01] Rolling back the virtual machine...
0%...10%...20%...30%...40%...50%...60%...70%...80%...90%...100%
0%...10%...20%...30%...40%...50%...60%...70%...80%...90%...100%
$ vagrant ssh -c "sudo rpm -aq | grep flac" node01
Connection to 127.0.0.1 closed.

終了

$ vagrant sandbox off
[node02] Stopping sandbox mode...
0%...10%...20%...30%...40%...50%...60%...70%...80%...90%...100%
[node01] Stopping sandbox mode...
0%...10%...20%...30%...40%...50%...60%...70%...80%...90%...100%

Packer で box 作成

$ brew tap homebrew/binary
$ brew install packer

CentOS の box

template.json

{
    "builders": [{
        "type": "virtualbox-iso",
        "guest_os_type": "RedHat_64",
        "iso_url": "http://ftp.iij.ad.jp/pub/linux/centos/6.5/isos/x86_64/CentOS-6.5-x86_64-minimal.iso",
        "iso_checksum": "0d9dc37b5dd4befa1c440d2174e88a87",
        "iso_checksum_type": "md5",
        "ssh_username": "vagrant",
        "ssh_password": "vagrant",
        "ssh_wait_timeout": "3000s",
        "vm_name": "box",
        "http_directory": "./",
        "boot_wait": "30s",
        "boot_command": [
            "<esc><wait>",
            "linux ks=http://{{ .HTTPIP }}:{{ .HTTPPort }}/ks.cfg ",
            "<enter><wait>"
        ],
        "shutdown_command": "sudo /sbin/poweroff"
    }],
    "provisioners": [{
        "type": "shell",
        "scripts": [
            "base.sh"
        ]
    }],
    "post-processors": [{
        "type": "vagrant",
        "output": "./CentOS-6.5-x86_64-ja.box"
    }]
}

ks.cfg

install
cdrom
lang en_US.UTF-8
keyboard us
network --bootproto=dhcp
rootpw --iscrypted $1$damlkd,f$UC/u5pUts5QiU3ow.CSso/
firewall --enabled --service=ssh
authconfig --enableshadow --passalgo=sha512
selinux --disabled
timezone Asia/Tokyo
bootloader --location=mbr

text
skipx
zerombr

clearpart --all --initlabel
autopart

auth --useshadow --enablemd5
firstboot --disabled
reboot

%packages --nobase
@core
%end

%post
/usr/bin/yum -y install sudo
/usr/bin/yum -y upgrade
/usr/sbin/groupadd vagrant
/usr/sbin/useradd vagrant -g vagrant -G wheel
echo "vagrant" | passwd --stdin vagrant
echo "vagrant      ALL=(ALL)       NOPASSWD: ALL" >> /etc/sudoers.d/vagrant
chmod 0440 /etc/sudoers.d/vagrant
%end

base.sh

/usr/sbin/setenforce 0
sudo sed -i "s/^.*requiretty/#Defaults requiretty/" /etc/sudoers
sudo sed -i "s/#UseDNS yes/UseDNS no/" /etc/ssh/sshd_config

cat <<EOF | sudo tee -a /etc/yum.repos.d/epel.repo
[epel]
name=epel
baseurl=http://download.fedoraproject.org/pub/epel/6/\$basearch
enabled=0
gpgcheck=0
EOF

sudo yum -y install gcc make automake autoconf libtool gcc-c++ kernel-headers-`uname -r` kernel-devel-`uname -r` zlib-devel openssl-devel readline-devel sqlite-devel perl wget nfs-utils bind-utils
sudo yum -y --enablerepo=epel install dkms

mkdir -pm 700 /home/vagrant/.ssh
wget --no-check-certificate 'https://raw.github.com/mitchellh/vagrant/master/keys/vagrant.pub' -O /home/vagrant/.ssh/authorized_keys
chmod 0600 /home/vagrant/.ssh/authorized_keys
chown -R vagrant /home/vagrant/.ssh

cd /tmp
sudo mount -o loop /home/vagrant/VBoxGuestAdditions.iso /mnt
sudo sh /mnt/VBoxLinuxAdditions.run
sudo umount /mnt

sudo /etc/rc.d/init.d/vboxadd setup

curl -L https//www.opscode.com/chef/install.sh | sudo bash

ビルド

$ packer build template.json
...
==> Builds finished. The artifacts of successful builds are:
--> virtualbox-iso: 'virtualbox' provider box: ./CentOS-6.5-x86_64-ja.box

Vagrant に box を登録する

$ vagrant box add CentOS-6.5-x86_64-ja ./CentOS-6.5-x86_64-ja.box
==> box: Adding box 'CentOS-6.5-x86_64-ja' (v0) for provider:
    box: Downloading: file:///Users/koshigoe/Projects/chef-practice/CentOS-6.5-x86_64-ja.box
==> box: Successfully added box 'CentOS-6.5-x86_64-ja' (v0) for 'virtualbox'!
$ vagrant box list
CentOS-6.5-x86_64-ja (virtualbox, 0)
opscode-centos-6.5   (virtualbox, 0)

他の provider

など。

Chef 実践入門(1) クックブック

Chef実践入門 ~コードによるインフラ構成の自動化 (WEB+DB PRESS plus)

Chef実践入門 ~コードによるインフラ構成の自動化 (WEB+DB PRESS plus)

今回も写経。

コミュニティクックブック

クックブックを検索する

$ knife cookbook site search apache2
apache:
  cookbook:             http://cookbooks.opscode.com/api/v1/cookbooks/apache
  cookbook_description: various apache server related resource provides (LWRP)
  cookbook_maintainer:  melezhik
  cookbook_name:        apache
apache2:
  cookbook:             http://cookbooks.opscode.com/api/v1/cookbooks/apache2
  cookbook_description: Installs and configures all aspects of apache2 using Debian style symlinks with helper definitions
  cookbook_maintainer:  onehealth
  cookbook_name:        apache2
apache2-windows:
  cookbook:             http://cookbooks.opscode.com/api/v1/cookbooks/apache2-windows
  cookbook_description: Installs and configures Apache on Microsoft Windows platforms.
  cookbook_maintainer:  dlrobinson
  cookbook_name:        apache2-windows
  (略)

クックブックの詳細を見る

$ knife cookbook site show apache2 | head -20
average_rating:
category:       Other
created_at:     2009-10-25T23:47:55.000Z
deprecated:     false
description:    Installs and configures all aspects of apache2 using Debian style symlinks with helper definitions
external_url:   http://github.com/opscode-cookbooks/apache2
latest_version: http://cookbooks.opscode.com/api/v1/cookbooks/apache2/versions/2.0.0
maintainer:     onehealth
metrics:
  downloads:
    total:    11815036
    versions:
      0.10.0: 227842
      0.10.1: 227941
      0.11.0: 227831
      0.12.0: 227835
      0.12.1: 227865
      0.12.2: 227868
      0.12.3: 228022
      0.9.1:  227976

クックブックの一覧

$ knife cookbook site list

Berkshelf

$ cat Berksfile
site :opscode

cookbook 'yum-epel'
cookbook 'apache2'
cookbook 'mysql'
$ berks
DEPRECATED: Your Berksfile contains a site location pointing to the Opscode Community Site (site :opscode). Site locations have been replaced by the source location. Change this to: 'source "https://supermarket.getchef.com"' to remove this warning. For more information visit https://github.com/berkshelf/berkshelf/wiki/deprecated-locations
Resolving cookbook dependencies...
Fetching cookbook index from https://supermarket.getchef.com...
Installing apache2 (2.0.0)
Installing logrotate (1.6.0)
Installing iptables (0.13.2)
Installing mysql (5.3.6)
Installing pacman (1.1.1)
Installing yum (3.2.2)
Installing yum-epel (0.4.0)
Installing yum-mysql-community (0.1.10)
$ vi Berksfile
$ cat Berksfile
source "https://supermarket.getchef.com"

cookbook 'yum-epel'
cookbook 'apache2'
cookbook 'mysql'

default.rb 以外のレシピ

recipe[apache2::mod_deflate]

default.rb の代わりに mod_deflate.rb が使われるルール。

Attribute の上書き

Node オブジェクトで apache2 の Attribute を上書き。

$ cat nodes/webdb.json
{
    "apache": {
        "listen_ports": [ 8080],
        "keepalive": "Off",
        "docroot_dir": "/home/vagrant/htdocs"
    },
    "run_list": [
        "recipe[dstat]",
        "recipe[apache2]",
        "recipe[mysql]"
    ],
    "automatic": {
        "ipaddress": "webdb"
    }
}

Chef Solo で複数ノードを管理する

Vagrant マルチ VM 機能

diff --git a/Vagrantfile b/Vagrantfile
index 0f53f00..660dfe6 100644
--- a/Vagrantfile
+++ b/Vagrantfile
@@ -12,6 +12,9 @@ Vagrant.configure(VAGRANTFILE_API_VERSION) do |config|
   # Every Vagrant virtual environment requires a box to build off of.
   config.vm.box = "opscode-centos-6.5"

+  config.vm.define :node01
+  config.vm.define :node02
+
   # Disable automatic box update checking. If you disable this, then
   # boxes will only be checked for updates when the user runs
   # `vagrant box outdated`. This is not recommended.

ロール

diff --git a/nodes/node01.json b/nodes/node01.json
new file mode 100644
index 0000000..4bcece1
--- /dev/null
+++ b/nodes/node01.json
@@ -0,0 +1,5 @@
+{
+    "run_list": [
+        "role[web]"
+    ]
+}
diff --git a/nodes/node02.json b/nodes/node02.json
new file mode 100644
index 0000000..4bcece1
--- /dev/null
+++ b/nodes/node02.json
@@ -0,0 +1,5 @@
+{
+    "run_list": [
+        "role[web]"
+    ]
+}
diff --git a/roles/web.json b/roles/web.json
new file mode 100644
index 0000000..609467f
--- /dev/null
+++ b/roles/web.json
@@ -0,0 +1,9 @@
+{
+    "name": "web",
+    "chef_type": "role",
+    "json_class": "Chef::Role",
+    "run_list": [
+        "recipe[git]",
+        "recipe[apache2]"
+    ]
+}

Environments

diff --git a/environments/development.json b/environments/development.json
new file mode 100644
index 0000000..489416e
--- /dev/null
+++ b/environments/development.json
@@ -0,0 +1,13 @@
+{
+    "name": "development",
+    "description": "Development environment",
+    "chef_type": "environment",
+    "json_class": "Chef::Environment",
+    "default_attributes": {
+        "apache": {
+            "max_children": "10"
+        }
+    },
+    "override_attributes": {
+    }
+}

Attribute の優先度

  1. Node オブジェクト
  2. ロール
  3. Environments
  4. レシピの中で定義された Attribute
  5. クックブック内の Attribute ファイル

複数ノードへ Chef Solo を実行

$ vagrant ssh-config --host node01 --host node02 >> ~/.ssh/config
$ echo node01 node02 | xargs -n 1 knife solo bootstrap
$ echo node01 node02 | xargs -n 1 knife solo cook

Chef 実践入門(0) 準備

Chef実践入門 ~コードによるインフラ構成の自動化 (WEB+DB PRESS plus)

Chef実践入門 ~コードによるインフラ構成の自動化 (WEB+DB PRESS plus)

写経。重い腰をあげていく。

仮想サーバを準備する

$ brew cask install vagrant virtualbox
$ vagrant -v
Vagrant 1.6.3
$ vagrant box add opscode-centos-6.5 http://opscode-vm-bento.s3.amazonaws.com/vagrant/virtualbox/opscode_centos-6.5_chef-provisionerless.box
==> box: Adding box 'opscode-centos-6.5' (v0) for provider:
    box: Downloading: http://opscode-vm-bento.s3.amazonaws.com/vagrant/virtualbox/opscode_centos-6.5_chef-provisionerless.box
==> box: Successfully added box 'opscode-centos-6.5' (v0) for 'virtualbox'!
$ vagrant init opscode-centos-6.5
A `Vagrantfile` has been placed in this directory. You are now
ready to `vagrant up` your first virtual environment! Please read
the comments in the Vagrantfile as well as documentation on
`vagrantup.com` for more information on using Vagrant.
$ vagrant up
Bringing machine 'default' up with 'virtualbox' provider...
==> default: Importing base box 'opscode-centos-6.5'...
==> default: Matching MAC address for NAT networking...
==> default: Setting the name of the VM: chef-practice_default_1408109406692_62816
==> default: Clearing any previously set network interfaces...
==> default: Preparing network interfaces based on configuration...
    default: Adapter 1: nat
==> default: Forwarding ports...
    default: 22 => 2222 (adapter 1)
==> default: Booting VM...
==> default: Waiting for machine to boot. This may take a few minutes...
    default: SSH address: 127.0.0.1:2222
    default: SSH username: vagrant
    default: SSH auth method: private key
    default: Warning: Connection timeout. Retrying...
==> default: Machine booted and ready!
==> default: Checking for guest additions in VM...
==> default: Mounting shared folders...
    default: /vagrant => /Users/koshigoe/Projects/chef-practice
$ vagrant ssh
[vagrant@localhost ~]$ exit
$ vagrant ssh-config --host webdb
Host webdb
  HostName 127.0.0.1
  User vagrant
  Port 2222
  UserKnownHostsFile /dev/null
  StrictHostKeyChecking no
  PasswordAuthentication no
  IdentityFile /Users/koshigoe/.vagrant.d/insecure_private_key
  IdentitiesOnly yes
  LogLevel FATAL

$ vagrant ssh-config --host webdb >> ~/.ssh/config
$ cp Vagrantfile Vagrantfile.orig
$ vi Vagrantfile
$ diff -u Vagrantfile.orig Vagrantfile
--- Vagrantfile.orig    2014-08-15 22:34:31.000000000 +0900
+++ Vagrantfile 2014-08-15 22:35:00.000000000 +0900
@@ -24,7 +24,7 @@

   # Create a private network, which allows host-only access to the machine
   # using a specific IP.
-  # config.vm.network "private_network", ip: "192.168.33.10"
+  config.vm.network "private_network", ip: "192.168.33.10"

   # Create a public network, which generally matched to bridged network.
   # Bridged networks make the machine appear as another physical device on
$ vagrant halt
==> default: Attempting graceful shutdown of VM...
$ vagrant up
Bringing machine 'default' up with 'virtualbox' provider...
==> default: Clearing any previously set forwarded ports...
==> default: Clearing any previously set network interfaces...
==> default: Preparing network interfaces based on configuration...
    default: Adapter 1: nat
    default: Adapter 2: hostonly
==> default: Forwarding ports...
    default: 22 => 2222 (adapter 1)
==> default: Booting VM...
==> default: Waiting for machine to boot. This may take a few minutes...
    default: SSH address: 127.0.0.1:2222
    default: SSH username: vagrant
    default: SSH auth method: private key
    default: Warning: Connection timeout. Retrying...
==> default: Machine booted and ready!
==> default: Checking for guest additions in VM...
==> default: Configuring and enabling network interfaces...
==> default: Mounting shared folders...
    default: /vagrant => /Users/koshigoe/Projects/chef-practice
==> default: Machine already provisioned. Run `vagrant provision` or use the `--provision`
==> default: to force provisioning. Provisioners marked to run always will still run.

Chef Solo を仮想サーバにインストール

$ vagrant ssh
[vagrant@localhost ~]$ curl -L https://www.opscode.com/chef/install.sh | sudo bash
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 16519  100 16519    0     0  14703      0  0:00:01  0:00:01 --:--:-- 35832
Downloading Chef  for el...
downloading https://www.opscode.com/chef/metadata?v=&prerelease=false&nightlies=false&p=el&pv=6&m=x86_64
  to file /tmp/install.sh.2020/metadata.txt
trying wget...
url     https://opscode-omnibus-packages.s3.amazonaws.com/el/6/x86_64/chef-11.14.2-1.el6.x86_64.rpm
md5     ffeffb67c310e6f76bb5d90ee7e30a3f
sha256  840946bc5f7931346131c0c77f2f5bfd1328245189fc6c8173b01eb040ffb58b
downloaded metadata file looks valid...
downloading https://opscode-omnibus-packages.s3.amazonaws.com/el/6/x86_64/chef-11.14.2-1.el6.x86_64.rpm
  to file /tmp/install.sh.2020/chef-11.14.2-1.el6.x86_64.rpm
trying wget...
Comparing checksum with sha256sum...
Installing Chef
installing with rpm...
警告: /tmp/install.sh.2020/chef-11.14.2-1.el6.x86_64.rpm: ヘッダ V4 DSA/SHA1 Signature, key ID 83ef826a: NOKEY
準備中...                ########################################### [100%]
   1:chef                   ########################################### [100%]
Thank you for installing Chef!
[vagrant@localhost ~]$ chef-solo -v
Chef: 11.14.2

Chef を動かす

[vagrant@localhost ~]$ sudo knife cookbook create hello -o /var/chef/cookbooks
WARNING: No knife configuration file found
** Creating cookbook hello
** Creating README for cookbook: hello
** Creating CHANGELOG for cookbook: hello
** Creating metadata for cookbook: hello
[vagrant@localhost ~]$ find /var/chef/cookbooks
/var/chef/cookbooks
/var/chef/cookbooks/hello
/var/chef/cookbooks/hello/resources
/var/chef/cookbooks/hello/README.md
/var/chef/cookbooks/hello/libraries
/var/chef/cookbooks/hello/metadata.rb
/var/chef/cookbooks/hello/CHANGELOG.md
/var/chef/cookbooks/hello/attributes
/var/chef/cookbooks/hello/definitions
/var/chef/cookbooks/hello/recipes
/var/chef/cookbooks/hello/recipes/default.rb
/var/chef/cookbooks/hello/files
/var/chef/cookbooks/hello/files/default
/var/chef/cookbooks/hello/templates
/var/chef/cookbooks/hello/templates/default
/var/chef/cookbooks/hello/providers
[vagrant@localhost ~]$ sudo cp /var/chef/cookbooks/hello/recipes/default.rb /var/chef/cookbooks/hello/recipes/default.rb.orig
[vagrant@localhost ~]$ sudo vi /var/chef/cookbooks/hello/recipes/default.rb
[vagrant@localhost ~]$ diff -u /var/chef/cookbooks/hello/recipes/default.rb.orig /var/chef/cookbooks/hello/recipes/default.rb
--- /var/chef/cookbooks/hello/recipes/default.rb.orig   2014-08-15 15:28:59.695189333 +0000
+++ /var/chef/cookbooks/hello/recipes/default.rb        2014-08-15 15:29:23.310191245 +0000
@@ -6,3 +6,4 @@
 #
 # All rights reserved - Do Not Redistribute
 #
+log 'Hello, World!'
[vagrant@localhost ~]$ sudo chef-solo -o hello
[2014-08-15T15:30:13+00:00] WARN: *****************************************
[2014-08-15T15:30:13+00:00] WARN: Did not find config file: /etc/chef/solo.rb, using command line options.
[2014-08-15T15:30:13+00:00] WARN: *****************************************
[2014-08-15T15:30:13+00:00] WARN:
* * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * *
SSL validation of HTTPS requests is disabled. HTTPS connections are still
encrypted, but chef is not able to detect forged replies or man in the middle
attacks.

To fix this issue add an entry like this to your configuration file:

```
  # Verify all HTTPS connections (recommended)
  ssl_verify_mode :verify_peer

  # OR, Verify only connections to chef-server
  verify_api_cert true
```

To check your SSL configuration, or troubleshoot errors, you can use the
`knife ssl check` command like so:

```
  knife ssl check -c /etc/chef/solo.rb
```

* * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * *

Starting Chef Client, version 11.14.2
[2014-08-15T15:30:14+00:00] WARN: Run List override has been provided.
[2014-08-15T15:30:14+00:00] WARN: Original Run List: []
[2014-08-15T15:30:14+00:00] WARN: Overridden Run List: [recipe[hello]]
Compiling Cookbooks...
Converging 1 resources
Recipe: hello::default
  * log[Hello, World!] action write


Running handlers:
Running handlers complete
Chef Client finished, 1/1 resources updated in 1.647607218 seconds

/etc/chef/solo.rb

[vagrant@localhost ~]$ sudo mkdir /etc/chef
[vagrant@localhost ~]$ sudo vi /etc/chef/solo.rb
[vagrant@localhost ~]$ cat /etc/chef/solo.rb
# Verify all HTTPS connections (recommended)
ssl_verify_mode :verify_peer

パッケージインストール

[vagrant@localhost ~]$ sudo knife cookbook create dstat -o /var/chef/cookbooks/
WARNING: No knife configuration file found
** Creating cookbook dstat
** Creating README for cookbook: dstat
** Creating CHANGELOG for cookbook: dstat
** Creating metadata for cookbook: dstat
cat <<\EOF | sudo tee -a /var/chef/cookbooks/dstat/recipes/default.rb
package 'dstat' do
  action :install
end
EOF
package 'dstat' do
  action :install
end
[vagrant@localhost ~]$ sudo chef-solo -o hello,dstat
Starting Chef Client, version 11.14.2
[2014-08-15T15:37:10+00:00] WARN: Run List override has been provided.
[2014-08-15T15:37:10+00:00] WARN: Original Run List: []
[2014-08-15T15:37:10+00:00] WARN: Overridden Run List: [recipe[hello], recipe[dstat]]
Compiling Cookbooks...
Converging 2 resources
Recipe: hello::default
  * log[Hello, World!] action write

Recipe: dstat::default
  * package[dstat] action install
    - install version 0.7.0-1.el6 of package dstat

Running handlers:
Running handlers complete
Chef Client finished, 2/2 resources updated in 11.672054207 seconds

knife-solo で chef-solo をリモート実行

$ cat <<\EOF > Gemfile
source 'https://rubygems.org'

gem 'knife-solo'
gem 'berkshelf'
EOF
$ bundle
$ knife solo init .
WARNING: No knife configuration file found
Creating kitchen...
Creating knife.rb in kitchen...
Creating cupboards...
Setting up Berkshelf...

knife-solo で chef-solo を仮想サーバにインストール

$ knife solo bootstrap webdb

クックブックを作成

$ knife cookbook create dstat -o site-cookbooks
$ cat <<\EOF >> site-cookbooks/dstat/recipes/default.rb
package 'dstat' do
  action :install
end
EOF

Node オブジェクト

diff --git a/nodes/webdb.json b/nodes/webdb.json
index cfc3fb0..562f116 100644
--- a/nodes/webdb.json
+++ b/nodes/webdb.json
@@ -1,8 +1,8 @@
 {
   "run_list": [
-
+    "recipe[dstat]"
   ],
   "automatic": {
     "ipaddress": "webdb"
   }
-}
\ No newline at end of file
+}

knife-solo で chef-solo を実行

$ knife solo cook webdb
....
Starting Chef Client, version 11.14.2
Compiling Cookbooks...
Converging 1 resources
Recipe: dstat::default
  * package[dstat] action install (up to date)

Running handlers:
Running handlers complete
Chef Client finished, 0/1 resources updated in 2.992491012 seconds

Apache, MySQL のセットアップ

$ knife cookbook create apache -o site-cookbooks
$ knife cookbook create mysql -o site-cookbooks
$ vi nodes/webdb.json

Apache のレシピ

$ cat <<\EOF >> site-cookbooks/apache/recipes/default.rb
package 'httpd' do
  action :install
end

service 'httpd' do
  action [:enable, :start]
end
EOF

MySQL のレシピ

$ cat <<\EOF >> site-cookbooks/mysql/recipes/default.rb
package 'mysql-server' do
  action :install
end

service 'mysqld' do
  action [:enable, :start]
end
EOF

chef-solo 実行

$ knife solo cook webdb

テンプレート

元となる設定ファイルを仮想サーバからコピー。

[vagrant@localhost ~]$ cp /etc/httpd/conf/httpd.conf /vagrant/site-cookbooks/apache/templates/default/httpd.conf.erb

レシピに追記。

$ cat <<\EOF >> site-cookbooks/apache/recipes/default.rb

template 'httpd.conf' do
  path '/etc/httpd/conf/httpd.conf'
  owner 'root'
  group 'root'
  mode 0644
  notifies :reload, 'service[httpd]'
end
EOF

仮想サーバを作り直して chef-solo 実行

$ vagrant destroy -f
$ vagrant up
$ knife solo bootstrap webdb

AWS を使った仕事もそろそろ一年

IAM Role と CloudFormation 便利ですよね。いい加減 chef も覚えて OpsWorks も理解しないと。

  • VPC
  • IAM
  • EC2
  • S3
  • SQS
  • SNS
  • SES
  • CloudSearch
  • ElastiCache
  • Elastic Beanstalk
  • CloudFormation
  • RDS

一番驚いたのは、EC2インスタンスが安定している事。数年前に触ったときは、いつの間にか EIP が外れていて全然繋がらなかったなんて事もあったのに。

SNS を使ったプッシュ通知も便利だなと思うのですが、大量の送信先リストを用意しておいて特定のリストに対して一括配信する様な運用が難しそうな点で残念な印象。Topic は標準で上限 3,000 ですもんね。

ElastiCache の Redis で Replication Group を作る時に、これを CloudFormation でノード作成から一気にやることができないのは残念。Beanstalk の conf で Redis ノードを作る所まではやれても、Replication Group を作ってエンドポイントを取得するところは別でやることになってしまって面倒。

EC2 インスタンスに IAM Role をアタッチして、必要なサービスにアクセスキーなしにアクセスできるのもいいですよね。

AMI ベースでサーバ構成を管理するという運用は原始的過ぎて、さっさと chef 覚えろよと言う話。あ、でも、cloud-init を使って AMI の内容を起動時に書き換えるのは、案外面白い。 Beanstalk の Docker サポートはまだ使えてないし、immutable infrastructure ってやつは全然習得できてないのが現状。

CloudSearch は案外高い感じで、しかも最近までは海外リージョンでしか使えず性能もいまいちだし、日本語的じゃなかった事もあってあれな感じですが、データ投げれば自動的にインデックスが作られて検索出来る様になるのは便利。インデックス再構築もサービス停止無しにやってくれるので安心。知らないけど Solr とか使ってもサービス停止無しにインデックス再構築はしてくれるものかな?そこら辺まできっちり面倒見てくれれば、楽でいいですね。

Beanstalk は最近さわり始めたばかりで、puma のスタック(1.0.3)を使ってたら、curlhttps のリクエストに失敗する版で最悪の未来を想像したのが思い出深いです。1.0.4 になって解消されたので一安心。puma も全く運用経験がないのですが、勢いです。

SES も海外リージョンだけあって激しく遅いですよね。まあ、普通に考えてメール送信(SMTP)を同期的にやることはないですよね。常識的に考えて。

RDS は、まださわりたてで、Parameter Group を使って文字コードを utf8mb4 にしつつキープレフィクスの調整をしたくらい。

そういえば、Redshift は S3 に CSV にしたログを貯めるだけで、結局使ってないですね。

そういえば Fabric にも手を出していた

ずっと Capistrano を使っていたので、感覚的にはすんなり使えた印象です。

ひとつ、特に印象的なのは、fab でタスクが完了したら、リモートサーバで立ち上げたプロセスがしんでしまったやつかな。これは、sudo 関数の pty オプションを False にして対処してた模様。細かい話はあまり覚えてない。

MongoDB を使う様になってそろそろ一年

MongoDB 関係では細々とパフォーマンス対策をしてた感じでしょうかね。

  • EC2 のインスタンスタイプ変更によるサーバリプレース
  • スロークエリ対策(コード修正、インデックス見直し、など)

十分な設計(特にインデックス)をせずにアプリケーションコードを書くと大火傷するぞ、という事は十二分に学ぶことができたかなと思います。なので、MongoDB を使う場合、アプリケーションコードを書く人は、がっつりと MongoDB について勉強すると良いと思います。公式ドキュメントは充実してるんじゃないかと思います。

とりあえず、一般的なウェブサービスを開発する場合、まずは MySQL を選ぶんじゃないかと思います。

P.S. ログのバッファ役には重宝しそうですよね。