Mysterious Processes and Frustrations

Linux Chronicles, Chapter IV: Mysterious Processes and Frustrations...

Well, I want to run something called a cron daemon, which is a Linux way to run certain programs, scripts, or commands once per day at an appointed time. Or once per week, or once per hour, etc. My desire is to run five Python scripts (don't worry, they were not written by me, I'm much too brain dead ;-)). The five scripts should, when run properly, automatically search the web for new articles on the Python programming language, and then FTP the new file to my web site each day, giving a new list each day of new web articles about Python. Cool huh? I thought so too... So I set my crontab file, by creating and editing a my_cron_file and typing crontab my_cron_file; that sets up the cron daemon, just like all the books say to do, or just like the man pages say. (By the way, typing man to get useful help that is scrollable on the BASH shell screen is the most useful and easy online help system I have ever seen). But my cron daemon doesn't seem to work properly.

I am embarrassed to be seeking help on this subject. I have used crontab to set up a my_cron_file that runs five Python scripts to automatically run mygale.py and associated programs to update the output. file listing new web articles about Python once per day.

My crontab file reads as such:

30 4 * * * /usr/bin/python /home/ron/mygale.py get all
30 5 * * * /usr/bin/python /home/ron/urls.py
30 6 * * * /usr/bin/python /home/ron/urldb.py
00 7 * * * /usr/bin/python /home/ron/mygale.py -n 200 display all
30 7 * * * /usr/bin/python /home/ron/mygale-uploader.py

crond is running , as checked using
/etc/rc.d/init.d/crond status which shows a running cron process with a pid number.

Also, ps aux | grep cron shows a /usr/sbin/cron process.

Finally, the cron daemons log check /var/log/cron shows all 5 Python scripts properly called at their appointed times each morning. No errors are at all are recorded in the cron error log files.

Now, when I run these scripts exactly as typed above, from the Bash shell,
everything works as planned and that's how I have been updating the
online page that shows the Pythonic articles on my web site.

The mystery is that, even though the log files show the Python scripts called each day, no update occurs on my web site . In deed, when I check the output. file in my /home/ron directory, it is not updated at all by these cron processes. Even though the log file shows the scripts as being ran, it is as if they did not run at all, in that no output occurs.

Yet when I run the same scripts manually, all is updated.

So this morning at 4:30 AM I sat up and ran a "top" process from the Bash shell, that shows all running processes. Sure enough, at exactly 4:30 AM, a new Python process was spawned. The new Python process was matched with a new entry in the cron log file of the mygale.py program being called.

I watched carefully, and the mygale.py process lasted for about 30 seconds and then disappeared from the "top" screen. I assume it terminated normally. Except that when I run the mygale.py get all script manually in the BASH shell, it takes about two minutes to complete. This is the only sign of any problem, but still no error messages are generated anywhere I can find.

Well, it seems to me that a series of processes on Linux or any operating system should be deterministic. That is, things should make sense, when the exact same steps are taken, the result should always be the same.

This procedure I am trying to do does not seem to make sense to me, it seems mysterious. Linux should not be mysterious.

Can anyone help me?

By the way, shouldn't it be possible to cut and paste from the BASH shell to a text editor? It would save me a lot of typing ;-))))

The clipboard in general is a frustration; I can't yet make it work properly in many situations. Is it me or does the KDE clipboard lack a lot of functionality compared to Windows clipboard? I can't even copy and paste from K-EDIT, nor K-WORD, to Netscape Composer. Oh, well.

One last frustration: Linux does hang, at least to the point that this newbie can't recover from it. My box hung when I tried to open a web presentation by Guido van Rossum about Python Conference 10 using K Presenter. I could not even open a console nor close the offending KDE shell to get back to a command line. The box also hangs when I try to print using cups on my HP T45 printer. (I wish I could use my printer.) My downloaded Open Office also hangs my computer just by trying to open it.

Nonetheless, these hang-ups and crashes are not of pivotal importance ot me. But mysterious behavior that not does not allow me to run daily scripts to update my web site does bother me. Computers shooed not be mysterious. They should be deterministic. Do a, then b, then c, get X for output.

Help ;-))))

Ron Stephens
February 10, 2002