Ansible Batch Processing in Practice¶

Software Introduction¶

Ansible is an automation operations tool developed in Python. It integrates the advantages of many operations tools (puppet, chef, func, fabric), achieving functionalities such as batch system configuration, batch program deployment, and batch command execution.

Features¶

1. Simple deployment; only the Ansible environment needs to be deployed on the control end, while no operation is required on the managed end.
2. Uses SSH protocol by default for device management.
3. Comes with a large number of regular operations modules that can handle most daily tasks.
4. Simple configuration, powerful functions, and highly extensible.
5. Supports APIs and custom modules, which can be easily extended through Python.
6. Customizes powerful configurations and status management via Playbooks.

Basic Architecture¶

• Ansible: The core program of Ansible.
• HostInventory: Records information about hosts managed by Ansible, including ports, passwords, IPs, etc.
• Playbooks: YAML format files where multiple tasks are defined in one file, specifying which modules should be called to complete the required functions.
• CoreModules: Core modules, primarily used to complete management tasks by calling these modules.
• CustomModules: Custom modules, used to achieve functions that cannot be completed by core modules, supporting various languages.
• ConnectionPlugins: Connection plugins used for communication between Ansible and Hosts.

Task Execution¶

The Ansible system divides the operation methods of control host nodes into two types: adhoc and playbook.

• ad-hoc mode (point-to-point mode) Uses a single module and supports batch execution of single commands. Ad-hoc commands are quick-to-enter commands that do not need to be saved, similar to a shell command in bash.

• playbook mode (script mode) The main management method of Ansible and the key to its powerful functionality. Playbooks accomplish a set of functions through a collection of multiple tasks, such as installing and deploying Web services or performing bulk backups of database servers. Playbooks can simply be understood as configuration files that combine multiple ad-hoc operations.

Batch Processing in Practice¶

Environment Preparation¶

Software Installation¶

Log in to ansible01 and execute the installation command

yum install -y ansible

Main Programs:

• /usr/bin/ansible Main Program
• /usr/bin/ansible-doc Configuration Documentation
• /usr/bin/ansible-playbook Customizes automated tasks, orchestration script tool
• /usr/bin/ansible-pull Remote command execution tool
• /usr/bin/ansible-vault File encryption tool

Main Configuration Files:

• /etc/ansible/ansible.cfg Main Configuration File
• /etc/ansible/hosts Host List (place managed hosts here)
• /etc/ansible/roles/ Directory for storing roles

Passwordless Login¶

Log in to ansible01 and generate a key, default path is /root/.ssh/id_rsa, /root/.ssh/id_rsa.pub

ssh-keygen

Distribute the key to the nodes that need to be managed

ssh-copy-id root@10.0.0.66
ssh-copy-id root@10.0.0.67

Modify the host list file /etc/ansible/hosts, add group name and host IPs

[guance]
10.0.0.67
10.0.0.66

Verify connectivity

ansible guance -m ping

Common Modules¶

Shell Module¶

The Shell module can invoke the shell interpreter on remote hosts to run commands, supporting various shell features like pipes.

• View current user id

ansible guance -m shell -a 'id'

• View users currently logged into the system

ansible guance -m shell -a 'who'

Copy Module¶

This module copies files to remote hosts and also supports generating files from given content and modifying permissions.

• Copy ansible.cfg file to remote host and specify permission as "read-write" -rw-rw-rw-

ansible guance -m copy -a 'src=/etc/ansible/ansible.cfg dest=/tmp/ansible.cfg mode=666'

View ansible.cfg file on remote host

ansible guance -m shell -a 'ls -l /tmp/ansible.cfg'

• Specify content and generate a file

ansible guance -m copy -a 'content="hello world" dest=/tmp/hello mode=666'

View file on remote host

ansible guance -m shell -a 'cat /tmp/hello'

File Module¶

This module sets file attributes, such as creating files, creating link files, deleting files, etc.

• Create app directory under /tmp

ansible guance -m file -a 'path=/tmp/app state=directory'

View /tmp directory

ansible guance -m shell -a 'ls -l /tmp'

• Delete ansible.cfg file previously copied from ansible01

ansible guance -m file -a 'path=/tmp/ansible.cfg state=absent'

Fetch Module¶

This module retrieves (copies) files from remote hosts to the local machine.

• Pull /tmp/hello file from remote host to /root directory

ansible guance -m fetch -a 'src=/tmp/hello dest=/root'

In /root directory, you can see two new directories (remote host IPs are used as directory names)

yum -y install tree
tree /root

Guance Application¶

Bulk Installation¶

Use Shell module to install DataKit (note to modify the corresponding token)

ansible guance -m shell -a 'DK_DATAWAY="https://openway.guance.com?token=token" bash -c "$(curl -L https://static.guance.com/datakit/install.sh)"'

Check if the process has started

ansible guance -m shell -a 'ps -ef|grep datakit|grep -v grep'

Bulk Configuration¶

• Enable netstat plugin

Use shell module to copy file netstat.conf.sample as netstat.conf

ansible guance -m shell -a 'cp /usr/local/datakit/conf.d/host/netstat.conf.sample /usr/local/datakit/conf.d/host/netstat.conf'

Bulk restart DataKit

ansible guance -m shell -a 'systemctl restart datakit'

Bulk Upgrade¶

Create a DataKit upgrade yaml file, /etc/ansible/dk_upgrade.yaml

- hosts: guance
  remote_user: root
  tasks:
    - name: dk version check
      shell: datakit --version|grep -i upgrade|wc -l
      register: version
    - name: dk upgrade
      when: version.stdout > "0"
      shell: DK_UPGRADE=1 bash -c "$(curl -L https://static.guance.com/datakit/install.sh)"

Run playbook

ansible-playbook /etc/ansible/dk_upgrade.yaml

Check if DataKit version is updated

ansible guance -m shell -a 'datakit --version'

Add scheduled task crontab -e (execute bulk upgrade at 02:02 AM every day)

02 02 * * * ansible-playbook /etc/ansible/dk_upgrade.yaml