Speed Up Ansible Playbook

In large network while running Ansible playbook on multiple servers may be a time consuming activity. So there are following two methods by using you can boost a speed of execution your playbook.

1. Pipelining

2. Mitogen Plugin (Third Party)

1. Pipelining

Pipelining, if supported by the connection plugin, reduces the number of network operations required to execute a module on the remote server, by executing many Ansible modules without actual file transfer. This can result in a very significant performance improvement when enabled. However this conflicts with privilege escalation. (Source)

So Pipelining reduces the number of SSH operations required to execute a module on the remote server.

How to enable pipelining in ansible ?

[root@siddhesh ~]# egrep 'ssh_conn|^pipelin' /etc/ansible/ansible.cfg
[ssh_connection]
pipelining = True
[root@siddhesh ~]#

By default pipelining is default. You can enable this by setting value to True

2. Mitogen Plugin (Third Party)

Mitogen for Ansibleis a completely redesigned UNIX connection layer and module runtime forAnsible. Requiring minimal configuration changes, it updates Ansible’s slow and wasteful shell-centic implementation with pure-Python equivalents, invoked via highly efficient remote procedure calls to persistent interpreters tunnelled over SSH. No changes are required to target hosts. (Source)

As per survey done by Mitogen, It gives1.25x - 7x speedup and a CPU usage reduction of at least 2x while performing ansible operation.

How to enable Mitogen Plugin on Ansible master server ?

Step 1. Download Mitogen plugin from their site as mentioned below.

[root@siddhesh ~]# wget https://networkgenomics.com/try/mitogen-0.2.9.tar.gz
--2020-05-24 19:27:05--  https://networkgenomics.com/try/mitogen-0.2.9.tar.gz
Resolving networkgenomics.com (networkgenomics.com)... 91.121.165.123
Connecting to networkgenomics.com (networkgenomics.com)|91.121.165.123|:443... connected.
HTTP request sent, awaiting response... 302 Found
Location: https://files.pythonhosted.org/packages/source/m/mitogen/mitogen-0.2.9.tar.gz 
--2020-05-24 19:27:06--  https://files.pythonhosted.org/packages/67/65/612aa75b9b16ef4d81d1f840ce59f01a7b348ee54df8966be0aeea7e7848/mitogen-0.2.9.tar.gz
Reusing existing connection to files.pythonhosted.org:443.
HTTP request sent, awaiting response... 200 OK
Length: 210868 (206K) [binary/octet-stream]
Saving to: ‘mitogen-0.2.9.tar.gz’

100%[===========================================================================================================================>] 210,868     --.-K/s   in 0.05s
2020-05-24 19:27:06 (4.31 MB/s) - ‘mitogen-0.2.9.tar.gz’ saved [210868/210868]
[root@siddhesh ~]#

Step 2. Extract plugin on local server.

[root@siddhesh ~]# tar -xvpf mitogen-0.2.9.tar.gz

Step 3. Copy plugin path and plugin name as mentioned below under [defaults] section.

[root@siddhesh ~]# egrep 'defaults|^strategy_plugins|^strategy' /etc/ansible/ansible.cfg
[defaults]
strategy_plugins   =  /usr/share/ansible/plugins/mitogen-0.2.9/ansible_mitogen/plugins/strategy
strategy = mitogen_linear
[root@siddhesh ~]#

You can run ansible playbook on bunch of nodes post setting above configuration on Ansible Tower. In next tutorial we'll see how to measure total time spend on each ansible task.