rsyslog.service not found by Ansible during preseed late_command

Question

I continually find that certain things "just don't work" when run as preseed/late_command in an Ubuntu Bionic preseed file. (The same applies to autoinstall for Ubuntu Focal.) In this particular case, I'm attempting to run an Ansible Role (specifically, the Canonical Ubuntu 18.04 LTS for Ansible STIG) against the target as a late_command:

d-i    preseed/late_command    string \
    in-target /usr/bin/curl -fsSL -o /opt/stig.zip {{ di_preseed.stig_role_url }}; \
    in-target /usr/bin/unzip -qq /opt/stig.zip -d /opt; \
    in-target /usr/bin/unzip -qq /opt/ubuntu1804STIG-ansible.zip -d /opt; \
    in-target /bin/sh -c -- 'cd /opt && ANSIBLE_LOG_PATH=/var/log/ansible.log /bin/sh enforce.sh'

Here enforce.sh is just a wrapper around ansible-playbook.

This install from ISO on Virtualbox fails with:

Failed to run preseeded command ... <command> finished with exit code 2

I am still able to log in to the box as ubuntu and become root. I see that Ansible was successfully installed (it is originally specified in pkgsel/include).

I then open up /var/log/installer/syslog on the target and find that ansible-playbook failed when it could not find rsyslog.service:

This is confusing as rsyslog.service is most certainly enabled and active after the install, which I can confirm with systemctl (status|is-active) rsyslog.

So what I'm seeking to understand here is:

• Why is Ansible unable to find rsyslog.service during install even though it appears to be enabled?
• What factors are different about the installation that result in things seeming to commonly break or not be available?
• Would I be better of running this as an onboot script under init.rc, then last line it removes itself after done?

---

Related: Certain commands (e.g. modprobe or usermod) fail as late-commands in Ubuntu autoinstall

Answer 1

The key thing to remember is that the in-target commands are executed in a chroot environment. They are not executed in a fully booted system where core processes like systemd are running and available.

Testing

I setup a playbook based on the STIG tasks in the screenshot and ran it from the preseed. I saw the same results as you.

• Playbook

---
- hosts: localhost
  gather_facts: no
  connection: local
tasks:
    - name: check if rsyslog.service is installed
      shell: ! systemctl list-unit-files | grep "^rsyslog.service[ \t]+"
      changed_when: False
      check_mode: no
      register: result
      failed_when: result.rc > 1
- name: stigrule_219160_rsyslog_enable
  service:
    name: rsyslog.service
    enabled: "yes"

• Partial preseed file

d-i pkgsel/include string ansible
d-i preseed/late_command string \
    wget -P /target/ http://REDACTED/my_playbook.yml ; \
    in-target ansible-playbook -i /dev/null -b -v /my_playbook.yml

Bionic packages ansible 2.5.1 and that version of the service module appears to execute systemctl show rsyslog.service. This does not work in a chroot environment. To demonstrate, if I open a terminal in the installer environment and run in-target systemctl show rsyslog.service then the log file will show the output

in-target: Running in chroot, ignoring request: show

Potential Fix

I did find a patch in Ansible 2.3 that addresses the problem that systemctl will ignore commands when running in a chroot environment. This was only applied to the systemd module though and not the service module. I updated my playbook and this does run successfully.

---
- hosts: localhost
  gather_facts: no
  connection: local
tasks:
    - name: check if rsyslog.service is installed
      shell: ! systemctl list-unit-files | grep "^rsyslog.service[ \t]+"
      changed_when: False
      check_mode: no
      register: result
      failed_when: result.rc > 1
- name: stigrule_219160_rsyslog_enable fixed
  systemd:
    name: rsyslog.service
    enabled: "yes"

Therefore, you may be able to progress further by modifying the STIG task(s) from using the service module to using the systemd module.