% and $ need to escape as %% and $$:
ExecStart=/bin/bash -c '/home/{{ shopify_api_user }}/shopify-api/getorders.pl --filter created_at_min=$$(date +%%Y-%%m-%%d --date="2 days ago")'
$ systemctl list-units
Filter by type:
$ systemctl list-units --type=socket
Filter by pattern:
$ systemctl list-units sympa*
Failed units:
$ systemctl --failed
It is recommended to examine why these units failed, but in some case these are old or irrelevant errors.
You can also filter the lists of failed units:
$ systemctl list-units --failed 'check_mk*'
You can remove them for the list individually:
$ systemctl reset-failed interchange
Or all of them:
$ systemctl reset-failed
As there might be more than file involved, the following command is handy:
$ systemctl cat sympa
Unit files can be edited through systemctl commands.
$ systemctl edit mybackup.service $systemctl edit --full mybackup.service systemctl edit --full --force mybackup.service
This shows the properties of a currently running unit:
$ systemctl show ssh
You can query a specific property:
$ systemctl show --property=EnvironmentFiles ssh EnvironmentFiles=/etc/default/ssh (ignore_errors=yes)
This relates to the following line in the unit file:
EnvironmentFile=-/etc/default/ssh
List all drop-ins:
~# systemd-delta --type=extended [EXTENDED] /etc/systemd/system/check_mk.socket → /etc/systemd/system/check_mk.socket.d/10-ipacl.conf [EXTENDED] /usr/lib/systemd/system/rc-local.service → /usr/lib/systemd/system/rc-local.service.d/debian.conf [EXTENDED] /usr/lib/systemd/system/systemd-resolved.service → /usr/lib/systemd/system/systemd-resolved.service.d/resolvconf.conf [EXTENDED] /usr/lib/systemd/system/systemd-timesyncd.service → /usr/lib/systemd/system/systemd-timesyncd.service.d/disable-with-time-daemon.conf
See also: Using systemd drop-in units
Commands to execute before the main process started.
This can be used to test the configuration:
ExecStartPre=/usr/sbin/nginx -t -q -g 'daemon on; master_process on;'
Adding a plus sign in front of the command indicates that the root user is executing this command:
ExecStartPre=+/bin/mkdir -p /run/sympa ExecStartPre=+/bin/chown sympa:sympa /run/sympa
User group used when running the service.
Group sympa
This directive is deprecated. It was used to run directives like ExecStartPre as privileged (root) user when your service is running as a non privileged user.
Instead of using this directive you can prepend a plus sign to the command, which indicates that the root user is executing this command.
ExecStartPre=+/bin/mkdir -p /run/sympa ExecStartPre=+/bin/chown sympa:sympa /run/sympa
Restart policy for the unit.
For debugging a failed unit it might make sense to turn off restarting:
Restart: no
This directive creates a directory in the default location (usually /run):
RuntimeDirectory sympa
If multiple related services are using the same directory it is important that only one service declares the runtime directory.
The other service units should declare Requires and After on the "base" service.
User running the service.
User sympa
This configures the working directory for the service, equivalent to cd in
a shell.
WorkingDirectory=/home/sympa
Timers are the equivalents of cron jobs. They consist of a service unit which defines the command to be executed and a timer unit which defines the schedule.
Advantages of using timers instead of cron jobs:
failed timers can be listed and monitored
output is automatically logged
timer is not executed if an instance is already running
easier to locate compared to the various files and directories with cron jobs
run job out of schedule without remembering the exact commandline
memory and CPU limits
precision is one second
Disadvantages:
takes more time to set them up
less control for users' jobs
Undecided:
Timers do not send automatically a notification on failed jobs.
With systemd timers ensure that your scripts provide a proper exit code on errors instead of relying on messages going to standard error output.
The timer needs to be enabled and started to be active:
$ systemctl enable certbot.timer $ systemctl start certbot.timer
Display active timers:
$ systemctl list-timers
Display all timers:
$ systemctl list-timers --all
You can always run the job manually independent with of the current schedule with:
$ systemctl start certbot.service
Every 20 minutes:
OnCalendar=*:0/20
Every even hour (0:00, 2:00, ...)
OnCalendar=0/2:00:00
Every odd hour (1:00, 3:00, ...)
OnCalendar=1/2:00:00
Once a day at midnight:
OnCalendar=daily
Once a day at 15:44:
OnCalendar=15:44
Monday noon:
OnCalendar=Mon 12:00
Start corresponding service immediately if last time was missed (e.g. system was down):
Persistent=true
Certbot service unit:
$ systemctl cat certbot.service # /lib/systemd/system/certbot.service [Unit] Description=Certbot Documentation=file:///usr/share/doc/python-certbot-doc/html/index.html Documentation=https://letsencrypt.readthedocs.io/en/latest/ [Service] Type=oneshot ExecStart=/usr/bin/certbot -q renew PrivateTmp=true
Certbot timer unit:
$ systemctl cat certbot.timer # /lib/systemd/system/certbot.timer [Unit] Description=Run certbot twice daily [Timer] OnCalendar=*-*-* 00,12:00:00 RandomizedDelaySec=43200 Persistent=true [Install] WantedBy=timers.target
Certbot timer status:
$ systemctl status certbot.timer ● certbot.timer - Run certbot twice daily Loaded: loaded (/lib/systemd/system/certbot.timer; enabled; vendor preset: enabled) Active: active (waiting) since Wed 2019-04-03 08:00:27 CEST; 2 months 13 days ago Trigger: Mon 2019-06-17 10:36:39 CEST; 18h left
Testing OnCalendar settings:
$ systemd-analyze calendar $(systemctl cat myservice.timer | sed -n 's/^OnCalendar=//p')
Original form: *:15:44
Normalized form: *-*-* *:15:44
Next elapse: Sun 2020-01-05 18:15:44 CET
(in UTC): Sun 2020-01-05 17:15:44 UTC
From now: 45min left
Set hostname:
$ hostnamectl set-hostname buster-test-box
This is not persistent on Ubuntu. In order to change this, you need to edit
/etc/cloud/cloud.cfg:
preserve_hostname: true
Show dependencies for a target:
$ systemctl list-dependencies network-online.target network-online.target ● ├─networking.service ● └─NetworkManager-wait-online.service
Show status of DNS resolver:
$ sudo resolvectl status
Configuration file: /etc/systemd/timesyncd.conf
Specific NTP servers (e.g. Active Domain Controllers) can be added in the NTP= line, separated by whitespace. Fallback NTP servers are configured in the FallbackNTP= line, for example
[Time] NTP= FallbackNTP=0.debian.pool.ntp.org 1.debian.pool.ntp.org 2.debian.pool.ntp.org 3.debian.pool.ntp.org
Restart systemd-timesyncd service after changing the configuration:
$ sudo systemctl restart systemd-timesyncd
Show time and date details:
$ timedatectl ...
Change timezone:
$ timedatectl set-timezone Europe/Berlin
This feature was introduced with systemd 235, which is available in recent Linux distributions like Debian Buster, Ubuntu Bionic and CentOS 8.
The following example comes from a check_mk monitoring agent which is listening on port 6556. To restrict access to the monitoring server we add the following drop-in (e.g. etc/systemd/system/check_mk.socket.d/10-ipacl.conf):
[Socket] IPAddressDeny=any IPAddressAllow=203.0.113.114
IP Accounting and Access Lists with systemd
$ sudo journalctl -b
This equivalent to dmesg -T.
$ sudo journalctl -k
Long log lines are not wrapped as with less, which makes it hard to see the complete message on the console.
Example to fix that:
$ systemctl status fail2ban.service --no-pager --full
You can change this behaviour with the SYSTEMD_PAGER environment variable:
$ export SYSTEMD_PAGER="less -r"
Retain only the past four days:
$ sudo journalctl --vacuum-time=4d
Retain only the past 750 MB:
$ sudo journalctl --vacuum-size=750M
Example: /lib/systemd/system/postgresql@.service on Debian for PostgreSQL cluster.
# prevent OOM killer from choosing the postmaster (individual backends will # reset the score to 0) OOMScoreAdjust=-900
● mariadb.service - MariaDB 10.3.22 database server
Loaded: loaded (/lib/systemd/system/mariadb.service; enabled; vendor preset: enabled)
Active: failed (Result: exit-code) since Wed 2020-03-25 11:32:09 CET; 1h 1min ago
Docs: man:mysqld(8)
https://mariadb.com/kb/en/library/systemd/
Process: 638935 ExecStartPre=/usr/bin/install -m 755 -o mysql -g root -d /var/run/mysqld (code=exited, status=217/USER)
Mar 25 11:32:09 belukha systemd[1]: Starting MariaDB 10.3.22 database server...
Mar 25 11:32:09 belukha systemd[638935]: mariadb.service: Failed to determine user credentials: No such process
Mar 25 11:32:09 belukha systemd[638935]: mariadb.service: Failed at step USER spawning /usr/bin/install: No such process
Mar 25 11:32:09 belukha systemd[1]: mariadb.service: Control process exited, code=exited, status=217/USER
Mar 25 11:32:09 belukha systemd[1]: mariadb.service: Failed with result 'exit-code'.
Mar 25 11:32:09 belukha systemd[1]: Failed to start MariaDB 10.3.22 database server.
Fixed by:
systemctl daemon-reexec systemctl start mariadb
Run them without systemd stepping on its toes:
export _SYSTEMCTL_SKIP_REDIRECT=1 /etc/init.d/sympa start