Ansible for Lazy Admins

Jordan Drysdale //

ADVISORY: The techniques and tools referenced within this blog post may be outdated and do not apply to current situations. However, there is still potential for this blog entry to be used as an opportunity to learn and to possibly update or integrate into modern tools and techniques.

For the lazy server and system admins, automating those boring functions of updating packages, finding outdated ones, checking scans, et cetera, Ansible has some very nice features. Here is a quick breakdown on some of those features I have found to be very useful (throughout post, useful stuff in bold).

First, do yourself a favor and run through the best practices – here. This software is useful and amazing for so many reasons, best summarized as agentless ssh key-auth based systems administration. For the security minded admins, review “CSC 2: Inventory of Authorized and Unauthorized Software” – here. Let’s say we approve only a particular kernel version and need to know if we have servers that have fallen out of line with approvals:

system:~$ ansible droplets -m shell -a "dpkg -l |grep linux-image*" -u ansible -K
ii  linux-image-3.13.0-85-generic           3.13.0-85.129                amd64                              
Linux kernel image for version 3.13.0 on 64 bit x86 SMP
ii  linux-image-extra-3.13.0-85-generic 3.13.0-85.129                amd64                            
Linux kernel extra modules for version 3.13.0 on 64 bit x86 SMP

Or, if we have a version of everyone’s second favorite text editor that has been identified in a recent CVE:

system:~$ ansible droplets -m shell -a "dpkg -l |grep nano" -u ansible -K
11.22.33.44 | SUCCESS | rc=0 >>
ii  nano    2.2.6-1ubuntu1     amd64    small, friendly text editor inspired by Pico
33.44.55.66 | SUCCESS | rc=0 >>
ii  nano     2.2.6-1ubuntu1    amd64    small, friendly text editor inspired by Pico

The above chunked commands are considered ad-hoc. Let’s take a look at a YAML file to see what a playbook looks like. Playbooks can be run against all hosts, or subsets depending on your hosts.conf file. This is an apt specific .yml file for updating all packages.

It can’t be that easy, can it? I can run this on ALL of my servers at once? Yup, and let’s do it!

system:/etc/ansible$ ansible-playbook -l droplets roles/common/tasks/apt.yml -u ansible -K

PLAY [all] *********************************************************************

TASK [setup] ******************************************************************

ok: [11.22.33.44]
ok: [33.44.55.66]

TASK [Check if there are packages available to be installed/upgraded] **********

changed: [11.22.33.44]    ###changed here means apt-get upgrade
changed: [33.44.55.66]

TASK [Upgrade all packages to the latest version] ******************************

changed: [11.22.33.44]    ###changed here means apt-get upgrade
changed: [33.44.55.66]

TASK [Check if a reboot is required] *******************************************

ok: [11.22.33.44]
ok: [33.44.55.66]

TASK [Reboot the server] *******************************************************

skipping: [11.22.33.44]
skipping: [33.44.55.66]

PLAY RECAP *********************************************************************

11.22.33.44         : ok=4        changed=2        unreachable=0        failed=0  
33.44.55.66         : ok=4        changed=2        unreachable=0        failed=0  

Okay, that’s fine and dandy, how about a reboot and a quick check of uptime? These .yml files are super simple, very useful and extremely handy. Again, if you were lazy and didn’t want to touch much of anything, you might want to start digging in to Ansible’s functionality.

Commands for these:

system:/etc/ansible$ ansible-playbook -l droplets roles/common/tasks/reboot.yml 
-u ansible -K
system:/etc/ansible$ ansible-playbook -l droplets roles/common/tasks/uptime.yml 
-u ansible -K

Next, let’s say I have a very standard way of deploying ssh across my servers and I want it to be super easy. My ssh.yml file will look like this:

- hosts: all
  become: yes
  tasks:
   - name: configure ssh options to system spec
         template: src=/etc/ansible/roles/common/templates/ssh.conf.j2 
         dest=/etc/ssh/sshd_config
         notify:
          - restart ssh
          - force ssh update
         tags: ssh
   - name: be sure ssh is running and restarted
         service: name=ssh state=restarted enabled=yes
         tags: ssh

Note the ssh.conf.J2 – this is a standard for deploying templates with Ansible. I have also invested a fair amount of time in creating similar YAML files for deploying ntp, fail2ban, filebeat, sendmail, nginx and a ton of other packages. My favorite one so far, and that is earning me the best of the laziest title, takes a combination of iptables to filter server access services by whitelist, then deploys fail2ban with a generic jail.local file to block malicious auth attempts against various services. Then lastly, it dumps a fully TLS enabled filebeat logger back through a firewall port into an elk stack!

Iptables:

- hosts: all  
  become: yes
  tasks:
   - name: CAREFUL ## deploy canned iptables ruleset ## CAREFUL
         copy: src=/etc/ansible/roles/common/templates/iptables.rules.j2 
         dest=/etc/iptables.rules
         tags: iptables
   - name: CAREFUL ## deploy firewall u+x file to enable iptables rules ## CAREFUL
         copy: src=/etc/ansible/roles/common/templates/firewall.j2 
         dest=/etc/network/if-up.d/firewall        mode="a+x"
         tags: firewall

Fail2ban:

- hosts: all
  become: yes
  tasks:
      - name: Install fail2ban
          apt: pkg=fail2ban state=installed update-cache=yes
          register: fail2ban_install
          tags: fail2ban
        - name: Install config
          template: src=jail.local.j2 dest=/etc/fail2ban/jail.local
          notify:
            - reload fail2ban

Filebeat:

- hosts: all
  become: yes
  tasks:
   - name: add apt_key for filebeat
         apt_key: url=https://packages.elasticsearch.org/GPG-KEY-elasticsearch 
         state=present
         tags: apt_key filebeat
   - name: add apt_repo for elastic software
         apt_repository:
           repo: "deb https://packages.elastic.co/beats/apt stable main"
         tags: filebeat repo         
   - name: install filebeat
         apt: pkg=filebeat state=installed update_cache=true
         tags: filebeat
   - name: create remote directory structure /etc/pki/tls/certs
         file: path=/etc/pki/tls/certs state=directory mode=0755
         tags: filebeat tls directories 
   - name: copy over the tls verification cert for secure logging
         copy: src=/etc/ansible/roles/common/files/logstash-forwarder.crt 
         dest=/etc/pki/tls/certs mode=0755
         notify:
           - restart filebeat
         tags: filebeat
   - name: deploy standardized internal network config via template
         template: src=/etc/ansible/roles/common/templates/filebeat.conf.j2 
         dest=/etc/filebeat/filebeat.yml
         tags: filebeat config template
   - name: be sure filebeat is running
         service: name=filebeat state=restarted enabled=yes
         tags: filebeat

Basically, from a Linux administration perspective, Ansible is an excellent choice. What we just went through here is basically still entirely ad-hoc – I stand up a server, create a new user called Ansible, add some keys for authentication and start running playbooks and command scripts. Even better, we can use roles to do almost all of this for us and if we are using AWS or Digital Ocean. Even better yet, we can pre-bake keys and users. If you get to this point, take a look at this last hyperlink – it describes how to bake all of your .yml scripts into a single playbook for use as a “bootstrapper.” Build a Linux box, add a key for Ansible to authenticate with, in five minutes I have a system online, fully firewalled, logging enabled, configured to spec, completely patched, booted and contributing services to digital hyperspace.



Want to learn more mad skills from the person who wrote this blog?

Check out this class from Kent and Jordan:

Defending the Enterprise

Available live/virtual and on-demand!