Chapter 5 RH294 RHEL Automation with Ansible

ONLY FOR SELF STUDY, NO COMMERCIAL USAGE!!!

---

Chapter 5. Deploying Files to Managed Hosts

Modifying and Copying Files to Hosts

Describing File Modules

Most of the commonly used modules related to Linux file management are provided with ansible-core in the ansible.builtin collection. They perform tasks such as creating, copying, editing, and modifying permissions and other attributes of files. The following table provides a list of frequently used file management modules:

Table 5.1. Commonly Used File Modules in ansible.builtin

Module name	Module description
blockinfile	Insert, update, or remove a block of multiline text surrounded by customizable marker lines.
copy	Copy a file from the local or remote machine to a location on a managed host. Similar to the file module, the copy module can also set file attributes, including SELinux context.
fetch	This module works like the copy module, but in reverse. This module is used for fetching files from remote machines to the control node and storing them in a file tree, organized by host name.
file	Set attributes such as permissions, ownership, SELinux contexts, and time stamps of regular files, symlinks, hard links, and directories. This module can also create or remove regular files, symlinks, hard links, and directories. A number of other file-related modules support the same options to set attributes as the file module, including the copy module.
lineinfile	Ensure that a particular line is in a file, or replace an existing line using a back-reference regular expression. This module is primarily useful when you want to change a single line in a file.
stat	Retrieve status information for a file, similar to the Linux stat command.

In addition, the ansible.posix collection, which is included in the default automation execution environment, provides some additional modules that are useful for file management:

Table 5.2. Commonly Used File Modules in ansible.posix

Module name	Module description
patch	Apply patches to files by using GNU patch.
synchronize	A wrapper around the rsync command to simplify common tasks. The synchronize module is not intended to provide access to the full power of the rsync command, but does make the most common invocations easier to implement. You might still need to call the rsync command directly via the command module depending on your use case.

Automation Examples with Files Modules

The following examples show ways that you can use these modules to automate common file management tasks.

Ensuring a File Exists on Managed Hosts

Use the ansible.builtin.file module to touch a file on managed hosts. This works like the touch command, creating an empty file if it does not exist, and updating its modification time if it does exist. In this example, in addition to touching the file, Ansible ensures that the owning user, group, and permissions of the file are set to specific values.

- name: Touch a file and set permissions
  ansible.builtin.file:
    path: /path/to/file
    owner: user1
    group: group1
    mode: 0640
    state: touch

Example outcome:

[user@host ~]$ ls -l file
-rw-r-----.  user1 group1 0 Nov 25 08:00 file

Modifying File Attributes

You can use the ansible.builtin.file module to ensure that a new or existing file has the correct permissions or SELinux type as well.

For example, the following file has retained the default SELinux context relative to a user's home directory, which is not the desired context.

[user@host ~]$ ls -Z samba_file
-rw-r--r--.  owner group unconfined_u:object_r:user_home_t:s0 samba_file

The following task ensures that the SELinux context type attribute of the samba_file file is the desired samba_share_t type. This behavior is similar to the Linux chcon command.

- name: SELinux type is set to samba_share_t
  ansible.builtin.file:
    path: /path/to/samba_file
    setype: samba_share_t

Example outcome:

[user@host ~]$ ls -Z samba_file
-rw-r--r--.  owner group unconfined_u:object_r:samba_share_t:s0 samba_file

File attribute parameters are available in multiple file management modules. Use the ansible-navigator doc command for additional information, providing the ansible.builtin.file or ansible.builtin.copy module as an argument.

Note

To set SELinux file contexts persistently in the policy, some options include:

• If you know how to use Ansible roles, you can use the supported redhat.rhel_system_roles.selinux role. That is covered in Chapter 7 of the Red Hat Enterprise Linux Automation with Ansible (RH294) training course.
• You can use the module community.general.sefcontext in the community-supported community.general Ansible Content Collection.

Copying and Editing Files on Managed Hosts

In this example, the ansible.builtin.copy module is used to copy a file located in the Ansible working directory on the control node to selected managed hosts.

By default, this module assumes that force: true is set. That forces the module to overwrite the remote file if it exists but has different contents to the file being copied. If force: false is set, then it only copies the file to the managed host if it does not already exist.

- name: Copy a file to managed hosts
  ansible.builtin.copy:
    src: file
    dest: /path/to/file

To retrieve files from managed hosts use the ansible.builtin.fetch module. This could be used to retrieve a file such as an SSH public key from a reference system before distributing it to other managed hosts.

- name: Retrieve SSH key from reference host
  ansible.builtin.fetch:
    src: "/home/{{ user }}/.ssh/id_rsa.pub
    dest: "files/keys/{{ user }}.pub"

To ensure a specific single line of text exists in an existing file, use the lineinfile module:

- name: Add a line of text to a file
  ansible.builtin.lineinfile:
    path: /path/to/file
    line: 'Add this line to the file'
    state: present

To add a block of text to an existing file, use the ansible.builtin.blockinfile module:

- name: Add additional lines to a file
  ansible.builtin.blockinfile:
    path: /path/to/file
    block: |
      First line in the additional block of text
      Second line in the additional block of text
    state: present

Note

When using the ansible.builtin.blockinfile module, commented block markers are inserted at the beginning and end of the block to ensure idempotency.

# BEGIN ANSIBLE MANAGED BLOCK
First line in the additional block of text
Second line in the additional block of text
# END ANSIBLE MANAGED BLOCK

You can use the marker parameter to the module to help ensure that the right comment character or text is being used for the file in question.

Removing a File from Managed Hosts

A basic example to remove a file from managed hosts is to use the ansible.builtin.file module with the state: absent parameter. The state parameter is optional to many modules. You should always make your intentions clear whether you want state: present or state: absent for several reasons. Some modules support other options as well. It is possible that the default could change at some point, but perhaps most importantly, it makes it easier to understand the state the system should be in based on your task.

- name: Make sure a file does not exist on managed hosts
  ansible.builtin.file:
    dest: /path/to/file
    state: absent

Retrieving the Status of a File on Managed Hosts

The ansible.builtin.stat module retrieves facts for a file, similar to the Linux stat command. Parameters provide the functionality to retrieve file attributes, determine the checksum of a file, and more.

The ansible.builtin.stat module returns a dictionary of values containing the file status data, which allows you to refer to individual pieces of information using separate variables.

The following example registers the results of a ansible.builtin.stat module task and then prints the MD5 checksum of the file that it checked. (The more modern SHA256 algorithm is also available; MD5 is being used here for legibility.)

- name: Verify the checksum of a file
  ansible.builtin.stat:
    path: /path/to/file
    checksum_algorithm: md5
  register: result

- ansible.builtin.debug
    msg: "The checksum of the file is {{ result.stat.checksum }}"

The outcome should be similar to the following:

TASK [Get md5 checksum of a file] *****************************************
ok: [hostname]

TASK [debug] **************************************************************
ok: [hostname] => {
    "msg": "The checksum of the file is 5f76590425303022e933c43a7f2092a3"
}

Information about the values returned by the ansible.builtin.stat module are documented in ansible-navigator doc ansible.builtin.stat, or you can register a variable and display its contents to see what is available:

- name: Examine all stat output of /etc/passwd
  hosts: workstation

  tasks:
    - name: stat /etc/passwd
      ansible.builtin.stat:
        path: /etc/passwd
      register: results

    - name: Display stat results
      debug:
        var: results

Synchronizing Files Between the Control Node and Managed Hosts

The ansible.posix.synchronize module is a wrapper around the rsync tool, which simplifies common file management tasks in your playbooks. The rsync tool must be installed on both the local and remote host. By default, when using the ansible.posix.synchronize module, the "local host" is the host that the ansible.posix.synchronize task originates on (usually the control node), and the "destination host" is the host that ansible.posix.synchronize connects to.

The following example synchronizes a file located in the Ansible working directory to the managed hosts:

- name: synchronize local file to remote files
  ansible.posix.synchronize:
    src: file
    dest: /path/to/file

You can use the ansible.posix.synchronize module and its many parameters in many different ways, including synchronizing directories. Run the ansible-navigator doc ansible.posix.synchronize command for additional parameters and playbook examples.

References

chmod(1), chown(1), rsync(1), stat(1) and touch(1) man pages

ansible-navigator doc command

Ansible documentation --- Index of all Modules - ansible.builtin

Example

---
- name: File management test
  hosts: servers
  tasks:
    - name: Retrieve secure logs
      ansible.builtin.fetch:  # get files from remote host
        src: /var/log/secure
        dest: secure-backups

    - name: Create files dir and set SE linux context
      ansible.builtin.file:
        path: /home/devops/files
        state: directory   # create the directory 'files'
        owner: devops
        group: devops
        mode: 0755
     # setype: samba_share_t   # change the SE context
        setype: _default

    - name: adding message to text
      ansible.builtin.lineinfile:
        path: /home/devops/files/users.txt
        line: "This line was addes by the lineinfile module"
        state: present
        create: true   # create the file if not exists
        owner: devops
        group: devops
        mode: 0664 

    - name: Copying files from here to remote
      ansible.builtin.copy:
        src: system
        dest: /home/devops/files/
        mode: 0664
        owner: devops
        group: devops

    - name: Adding message with blockinfile
      ansible.builtin.blockinfile:
        path: /home/devops/files/users.txt
        block: |
          this block of text consists of two lines.
          They have been added by the blockinfile module.

output:

[student@workstation secure-backups]$ ssh devops@servera 'ls -Z'
unconfined_u:object_r:samba_share_t:s0 files


[student@workstation ~]$ [student@workstation file-manage]$ tree -F secure-backups/
secure-backups/
├── servera.lab.example.com/
│   └── var/
│       └── log/
│           └── secure
└── serverb.lab.example.com/
    └── var/
        └── log/
            └── secure


[student@workstation file-manage]$ ssh devops@servera 'cat files/users.txt'
This line was addes by the lineinfile module
# BEGIN ANSIBLE MANAGED BLOCK
this block of text consists of two lines.
They have been added by the blockinfile module.
# END ANSIBLE MANAGED BLOCK

Deploying Custom Files with Jinja2 Templates

Templating Files

The ansible.builtin Ansible Content Collection provides a number of modules that can be used to modify existing files. These include lineinfile and blockinfile, among others. However, they are not always easy to use effectively and correctly.

A much more powerful way to manage files is to template them. With this method, you can write a template configuration file that is automatically customized for the managed host when the file is deployed, using Ansible variables and facts. This can be easier to control and is less error-prone.

Introduction to Jinja2

Ansible uses the Jinja2 templating system for template files. Ansible also uses Jinja2 syntax to reference variables in playbooks, so you already know a little about how to use it.

Using Delimiters

Variables and logic expressions are placed between tags, or delimiters . When a Jinja2 template is evaluated, the expression {``{ *EXPR* }} is replaced with the results of that expression or variable. Jinja2 templates can also use {% *EXPR* %} for special control structures or logic that loops over Jinja2 code or perform tests. You can use the {# *COMMENT* #} syntax to enclose comments that should not appear in the final file.

In the following example of a Jinja2 template file, the first line includes a comment that is not included in the final file. The variable references in the second line are replaced with the values of the system facts being referenced.

{# /etc/hosts line #}
{{ ansible_facts['default_ipv4']['address'] }}    {{ ansible_facts['hostname'] }}

Building a Jinja2 Template

A Jinja2 template is composed of multiple elements: data, variables, and expressions. Those variables and expressions are replaced with their values when the Jinja2 template is rendered. The variables used in the template can be specified in the vars section of the playbook. It is possible to use the managed hosts' facts as variables in a template.

Template files are most commonly kept in the templates directory of the project for your playbook, and typically are assigned a .j2 file extension to make it clear that they are Jinja2 template files.

Note:

A file containing a Jinja2 template does not need to have any specific file extension (for example, .j2). However, providing such a file extension might make it easier for you to remember that it is a template file.

The following example shows how to create a template for /etc/ssh/sshd_config with variables and facts retrieved by Ansible from managed hosts. When the template is deployed by a play, any facts are replaced by their values for the managed host being configured.

# {{ ansible_managed }}
# DO NOT MAKE LOCAL MODIFICATIONS TO THIS FILE BECAUSE THEY WILL BE LOST

Port {{ ssh_port }}
ListenAddress {{ ansible_facts['default_ipv4']['address'] }}

HostKey /etc/ssh/ssh_host_rsa_key
HostKey /etc/ssh/ssh_host_ecdsa_key
HostKey /etc/ssh/ssh_host_ed25519_key

SyslogFacility AUTHPRIV

PermitRootLogin {{ root_allowed }}
AllowGroups {{ groups_allowed }}

AuthorizedKeysFile /etc/.rht_authorized_keys .ssh/authorized_keys

PasswordAuthentication {{ passwords_allowed }}

ChallengeResponseAuthentication no

GSSAPIAuthentication yes
GSSAPICleanupCredentials no

UsePAM yes

X11Forwarding yes
UsePrivilegeSeparation sandbox

AcceptEnv LANG LC_CTYPE LC_NUMERIC LC_TIME LC_COLLATE LC_MONETARY LC_MESSAGES
AcceptEnv LC_PAPER LC_NAME LC_ADDRESS LC_TELEPHONE LC_MEASUREMENT
AcceptEnv LC_IDENTIFICATION LC_ALL LANGUAGE
AcceptEnv XMODIFIERS

Subsystem sftp	/usr/libexec/openssh/sftp-server

Deploying Jinja2 Templates

Jinja2 templates are a powerful tool that you can use to customize configuration files to be deployed on managed hosts. When the Jinja2 template for a configuration file has been created, it can be deployed to managed hosts by using the ansible.builtin.template module, which supports the transfer of a local file on the control node to the managed hosts.

To use the ansible.builtin.template module, use the following syntax. The value associated with the src key specifies the source Jinja2 template, and the value associated with the dest key specifies the file to be created on the destination hosts.

tasks:
  - name: template render
    ansible.builtin.template:
      src: /tmp/j2-template.j2
      dest: /tmp/dest-config-file.txt

Note:

The ansible.builtin.template module also allows you to specify the owner (the user that owns the file), group, permissions, and SELinux context of the deployed file, just like the ansible.builtin.file module. It can also take a validate option to run an arbitrary command (such as visudo -c) to check the syntax of a file for correctness before templating it into place.

For more details, see ansible-navigator doc ansible.builtin.template.

Managing Templated Files

To avoid having other system administrators modify files that are managed by Ansible, it is a good practice to include a comment at the top of the template to indicate that the file should not be manually edited.

One way to do this is to use the "Ansible managed" string set by the ansible_managed directive. This is not a normal variable but can be used as one in a template. You can set the value for ansible_managed in an ansible.cfg file:

ansible_managed = Ansible managed

To include the ansible_managed string inside a Jinja2 template, use the following syntax:

{{ ansible_managed }}

Control Structures

You can use Jinja2 control structures in template files to reduce repetitive typing, to enter entries for each host in a play dynamically, or conditionally insert text into a file.

Using Loops

Jinja2 uses the for statement to provide looping functionality. In the following example, the users variable has a list of values. The user variable is replaced with all the values in the users variable, one value per line.

{% for user in users %}
      {{ user }}
{% endfor %}

The following example template uses a for statement and a conditional to run through all the values in the users variable, replacing myuser with each value, unless the value is root.

{# for statement #}
{% for myuser in users if not myuser == "root" %}
User number {{ loop.index }} - {{ myuser }}
{% endfor %}

The loop.index variable expands to the index number that the loop is currently on. It has a value of 1 the first time the loop executes, and it increments by 1 through each iteration.

As another example, this template also uses a for statement. It assumes a myhosts variable that contains a list of hosts to be managed has been defined by the inventory being used. If you put the following for statement in a Jinja2 template, all hosts in the myhosts group from the inventory would be listed in the resulting file.

{% for myhost in groups['myhosts'] %}
{{ myhost }}
{% endfor %}

For a more practical example, you can use this example to generate an /etc/hosts file from host facts dynamically. Assume that you have the following playbook:

- name: /etc/hosts is up to date
  hosts: all
  gather_facts: true
  tasks:
    - name: Deploy /etc/hosts
      ansible.builtin.template:
        src: templates/hosts.j2
        dest: /etc/hosts

The following three-line templates/hosts.j2 template constructs the file from all hosts in the group all. (The middle line is extremely long in the template due to the length of the variable names.) It iterates over each host in the group to get three facts for the /etc/hosts file.

{% for host in groups['all'] %}
{{ hostvars[host]['ansible_facts']['default_ipv4']['address'] }} {{ hostvars[host]['ansible_facts']['fqdn'] }} {{ hostvars[host]['ansible_facts']['hostname'] }}
{% endfor %}

Using Conditionals

Jinja2 uses the if statement to provide conditional control. This allows you to put a line in a deployed file if certain conditions are met.

In the following example, the value of the result variable is placed in the deployed file only if the value of the finished variable is True.

{% if finished %}
{{ result }}
{% endif %}

Variable Filters

Jinja2 provides filters which change the output format for template expressions, essentially converting the data in a variable to some other format in the file that results from the template.

For example, filters are available for languages such as YAML and JSON. The to_json filter formats the expression output using JSON, and the to_yaml filter formats the expression output using YAML.

{{ output | to_json }}
{{ output | to_yaml }}

Additional filters are available, such as the to_nice_json and to_nice_yaml filters, which format the expression output in either JSON or YAML human-readable format.

{{ output | to_nice_json }}
{{ output | to_nice_yaml }}

Both the from_json and from_yaml filters expect strings in either JSON or YAML format, respectively.

{{ output | from_json }}
{{ output | from_yaml }}

For more information you can also review "Using filters to manipulate data" in the Ansible User Guide.

References

ansible.builtin.template module - Template a file out to a target host --- Ansible Documentation

Using filters to manipulate data --- Ansible Documentation

Example

# Creating a motd.j2 template file
This is the system {{ ansible_facts['fqdn'] }}

This is a {{ ansible_facts['distribution'] }} version {{ ansible_facts['distribution_version'] }}

{% if ansible_facts['fqdn'] in groups['workstations'] %}
As a workstation user, you need to submit a ticket to receive help with any issues.
{% elif ansible_facts['fqdn'] in groups['webservers'] %}
Please report issues to: {{ system_owner }}.
{% endif  %}

---
- name: Managing Files temaplate test
  hosts: all
  vars: 
    - system_owner: peanutfish@abc.com
  tasks:
    - name: moving motd files
      ansible.builtin.template:
        src: motd.j2
        dest: /etc/motd
        owner: root
        group: root
        mode: 0644

output:

[student@workstation file-template]$ ssh devops@servera 'cat /etc/motd'
This is the system servera.lab.example.com

This is a RedHat version 9.0

Please report issues to: peanutfish@abc.com.



[student@workstation file-template]$ ssh devops@workstation 'cat /etc/motd'
This is the system workstation.lab.example.com

This is a RedHat version 9.0

As a workstation user, you need to submit a ticket to receive help with any issues.

Chapter 5 TEST

In the /home/student/file-review directory, create a playbook file called motd.yml that contains a new play that runs on all hosts in the inventory. It must log in using the devops user on the remote host, and use become to enable privilege escalation for the whole play.

The play must have a task that uses the ansible.builtin.template module to deploy the motd.j2 Jinja2 template file to the file /etc/motd on the managed hosts. The resulting file must have the root user as its owning user and group, and its permissions must be 0644.

Add an additional task that uses the ansible.builtin.stat module to verify that /etc/motd exists on the managed hosts and registers its results in a variable. That task must be followed by a task that uses ansible.builtin.debug to display the information in that registered variable.

Add a task that uses the ansible.builtin.copy module to place files/issue into the /etc/ directory on the managed host, use the same ownership and permissions as /etc/motd.

Finally, add a task that uses the ansible.builtin.file module to ensure that /etc/issue.net is a symbolic link to /etc/issue on the managed host.

# Create a template file here, folder ./templates/motd.j2

Welcome to the END of THE DAY!

This system {{ansible_facts['fqdn']}} provided {{ ansible_facts['memtotal_mb'] }} mb memory and {{ ansible_facts['processor_count'] }} CPU core.

---
- name: Starting file review session 
  hosts: all
  remote_user: devops
  become: true
  become_user: root

  tasks:
    - name: Check system total memory and number of processors
      ansible.builtin.debug:
        msg: >
          The amount of system momory is {{ ansible_facts['memtotal_mb'] }}mb and
          the number of processors is {{ ansible_facts['processor_count'] }}.
        # var: ansible_facts
    - name: Copying motd file
      ansible.builtin.template:
        src: templates/motd.j2
        dest: /etc/motd
        owner: root
        group: root
        mode: 0644

    - name: Check if motd exists
      ansible.builtin.stat:
        path: /etc/motd
      register: result

    - name: show debug info
      ansible.builtin.debug:
        var: result.stat

    - name: Copying files
      ansible.builtin.copy:
        src: files/issue
        dest: /etc/
        owner: root
        group: root
        mode: 0644

    - name: Verify the symbolic link
      ansible.builtin.file:
        src: /etc/issue
        dest: /etc/issue.net
        state: link
        owner: root
        group: root
        force: true    # Force the creation of the symlinks in two cases: the source file does not exist (but will appear later); the destination exists and is a file (so, we need to unlink the path file and create a symlink to the src file in place of it).

---

TO BE CONTINUED...