How Do You Troubleshoot a cronjob That Indirectly Uses Linux Programs (e.g., Executables)?

Problem scenario
You have a Python script that runs successfully when your user runs it.  You have a *nix crontab that calls this Python script at a scheduled time.  The script mostly works, but one part fails when the crontab launches it (automatically).  This problem does not happen when it has been run manually.

The part that fails involves a Linux command via a Python subprocess.check_output() invocation in the program.  That is, the part of the Python program that fails is the portion that invokes a Bash command.  You have tested the crontab to determine what user runs the jobs (by using "* * * * * /usr/bin/whoami > /tmp/log.txt").  You have found that the user is the same as the user who can run the script manually.  When that user runs it manually, there are no problems.  Why will the crontab not correctly run your Python script when it appears to be run in the context as the user who runs it manually and successfully?

Possible Solution #1 (For Maximum Learning if You Are Interested)
Find the line that does not work.  Examine if an executable file (or relevant file) relies on an environmental variable for PATH.  

For Detailed Analysis
One way to see the difference between the environment variables of the crontab execution (to fully appreciate the context of cronjobs) is to set up a cron job like this:

* * * * * /usr/bin/env > /tmp/logenvironment.txt

Compare /tmp/logenvironment.txt to the output displayed by your interactive execution of env (from the Linux command prompt).  The "PATH" variable may be different.  You may need to rewrite the Python script to call the full directory path of the executable (assuming calling an executable is the part where the problem is).  This should not be a problem when the user manually runs it interactively.  This should solve the problem of having different environment variables when the crontab runs the Python program contrasted from a manual run of the same program.

Possible Solution #2 (The Quick and Dirty Solution)
To fix the problem you may want to do the following "quick and dirty" solution. This is not a recommended practice as it may add excessive extra environment variables in a script, but it will resolve the problem. (It is fast because it can be difficult to ascertain which environment variable is necessary for the Python script to run successfully.) Another downside to this solution is that it makes the cronjob slightly more complex to maintain because you will have a Bash script to maintain just to run a Python program.

1. Identify the user account that can run the Python script interactively (or manually) without a problem.
2. Then have write a Bash script to run the Python program. You may want to write it like this:

env > pythonkicker.sh

3. Edit pythonkicker.sh to place at the very bottom a stanza like this:

/usr/bin/python yourprogram.py

This way you are sure that the environment variables are set whenever pythonkicker.sh is run.

4. Now have the cronjob call pythonkicker.sh instead of the Python script directly.

Leave a comment

Your email address will not be published. Required fields are marked *