My scheduled job runs, but the script does nothing. When I start it myself
in the terminal it works fine. Find out why, look at the environment the job
runs in and at the exit code. Do not change anything yet.
This is one of those errors that costs you an hour and then never again. You have a script that works. You put it in a timer or in a cron job. It runs, it ends, and nothing happens. Start it yourself in the terminal and it works perfectly.
Nine times out of ten the log holds one line: node: command not found. And that line is correct, however strange that looks.
Why there is a difference
An interactive shell loads your profile. That profile holds lines which add extra directories to your PATH: the directory of a version manager for Node, your own bin directory, a directory with tools. That is why your terminal finds node and npm.
A scheduled job does not load that profile. A systemd unit and a cron job start your script in a bare environment, with a short PATH and without your settings. Everything you arranged in your profile does not exist there.
If you use Node through a version manager, the real node sits somewhere under your own home directory, in a folder with the version number in it. That folder is only in your PATH because your profile puts it there on every login. In a timer that does not happen, so node simply does not exist there.
The same goes for a PHP version you picked yourself, for wp and for every helper you once put in place by hand.
Two ways to do it properly
An absolute path in the job. The simplest and the most predictable. In the unit or in the cron line you put the full path to the executable, not just the name. Then it does not matter what the PATH does.
The downside: after a new Node version you have a path to a version that no longer exists. That is not a disaster, because you get a clear error, but you do have to remember it.
Look the path up inside the script. Slightly more work, and considerably more durable. You let the script work out which node to use: first check whether node is in the PATH, and if not, take the highest version in your version manager's directory:
NODE=$(command -v node || ls -1d "$HOME"/.nvm/versions/node/*/bin/node 2>/dev/null | sort -V | tail -1)
[ -x "$NODE" ] || { echo "no node found" >&2; exit 1; }
"$NODE" my-script.js
That last line with sort -V sorts by version number instead of as text, so version 20 comes after version 9 and not before it. And that check with -x matters more than it looks. Without that line the script carries on with an empty variable, and then you get an error that has nothing to do with Node.
The same approach works for every program that comes from your profile.
The error behind the error: failing silently
This is the lesson that cost us more time than the PATH itself. We had a check that ran every hour, with a wrapper around it that judged the result. That wrapper looked at the exit codes it knew: zero was fine, one was a warning, two was a problem.
When the script died on node: command not found, it returned exit code 127. That was not in the list. The wrapper fell through to the "nothing wrong" case and reported nothing. The check sat idle for days without anyone noticing.
The rule we took from that: every unexpected exit code has to raise an alert of its own. Not only the codes you thought of.
"$NODE" my-script.js
rc=$?
case $rc in
0) ;;
1) notify "warning" ;;
2) notify "problem" ;;
*) notify "unexpected exit code $rc, the check did NOT run" ;;
esac
A check that can fail without saying anything is not a check. It is a feeling of safety.
How to find it in two minutes
Let the job run once and look at three things: the exit code, the PATH it ran with and the last lines of the log. Code 127 almost always means "program not found". Code 126 means "found, but not executable", often a permissions problem.
The comparison that proves it: have your script print the PATH and the node it found, once when you start it and once when the job starts it. If the second output shows a short PATH without your own directories, you know.
This is exactly the kind of work you can hand over instead of digging yourself:
Run this scheduled job once now, show the exit code, the PATH it ran with
and the last twenty lines of output. Compare that with what I get in the
terminal and tell me which difference causes the problem.
Common mistakes
Extending the PATH in the unit or in the cron line. That works, but you are pasting a piece of your profile into your job configuration. At the next version switch it is wrong again. An absolute path or a resolve inside the script stays usable.
Loading your profile inside the job. It is tempting to start your script with a login shell so everything is there. You do get your PATH, but also everything else your profile does, and your job starts depending on a file you edit for interactive use. Keep a job's environment small and explicit.
Testing by starting the script yourself. That tests exactly the one situation that always worked. Test by starting the job, not the script.
Assuming a job that ran is a job that succeeded. A job that starts and stops neatly every time can die on the same line every time. Look at the exit code and at what the script produced, not at whether it ran.
Handling only the known exit codes. See the paragraph above. This is the mistake you discover only once you needed it to work.