0

I am running a cron job everyday at 21:00 UTC which downloads few csv files from sftp server, renames the downloaded files and then loads the files in database(mysql) and updates the time in database (mysql).

ps -ef | grep cron | grep -v grep
root      2656     1  0 May28 ?        00:00:13 crond

cron jobs 

37 21 * * * cd /opt/currentdata/ && ./dbupdate.sh >> /opt/scriptlogs/db1.log 2>&1
30 21 * * * cd /opt/currentdata/ && ./dwnldsftp.sh >> /opt/scriptlogs/db2.log 2>&1
35 21 * * * cd /opt/currentdata/ && /usr/bin/python /opt/currentdata/files2.py >> /opt/scriptlogs/filerename.log 2>&1
40 21 * * * cd /opt/currentdata/ && /usr/bin/python /opt/currentdata/tymup.py >> /opt/scriptlogs/timeupd.log 2>&1

cron jobs are setup as root user and I have provided the permissions to these files as below

-rwxrwxrwx 1 root     root  246 Jul 27 19:27 dwnldsftp.sh
-rwxrwxrwx 1 root     root 3095 Jul 28 04:20 dbupdate.sh
-rwxrwxrwx 1 root     root 3069 Jun 12 19:07 files2.py
-rwxrwxrwx 1 root     root  512 Jul 28 09:02 tymup.py

once the csv files are downloaded I have described in dwnldsftp.sh script to make those files as executable read write. To the whole folder I have kept the scripts I have made them as read write execcutable.

Problem : The cron job runs everyday but it doesnt update the database, I have tried keeping the path of mysql in script top as well , it dowloads the file from SFTP server and renames as well the script for time update in database also runs but loading of csv files in database does not happen.

SHELL=/bin/bash
PATH=/sbin:/bin:/usr/sbin:/usr/bin

But when I manually execute this scripts after doing SSH to my instance and become root user it works absolutely fine.

There are no errors logged in cron logs

CROND[14202]: (root) CMD (cd /opt/currentdata/ && ./dwnldsftp.sh >> /opt/scriptlogs/db2.log 2>&1)
CROND[14213]: (root) CMD (cd /opt/currentdata/ && /usr/bin/python /opt/currentdata/files2.py >> /opt/scriptlogs/filerename.log 2>&1)
CROND[14216]: (root) CMD (cd /opt/currentdata/ && ./dbupdate.sh >> /opt/scriptlogs/db1.log 2>&1)
CROND[14225]: (root) CMD (cd /opt/currentdata/ && /usr/bin/python /opt/currentdata/tymup.py >> /opt/scriptlogs/timeupd.log 2>&1)

Can anyone help me here what I am missing here ?

EDIT :

Conditions when cron works :

I SSH to instance become root user , then execute those above 4 files manually(SFTP file download, rename files, update database, update time) so data is updated in database successfully now I set the cron job to run just next 15 minutes after the time I executed the scripts manually for whole flow (SFTP file download, rename files, update database, update time) and logout the SSH session. cron works successfully and data in database is updated.

When cron doesn't works :

I SSH into instance execute command crontab -u root -e then I set the cron jobs for whole flow (SFTP file download, rename files, update database, update time) to run at 21:00 UTC as described above, then logout from SSH session. I check next day the data is updated or not in database I find that data is not updated , I check cron logs where I see that cron executed successfully.

Roy
  • 1
  • 1
  • @GeraldSchneider I have gone through all this posts and debugged it :-) Nothing I can find , I dont have "syslog" but "messages" but no response of error in those as well for cron – Roy Jul 31 '19 at 06:12
  • 4
    cron errors are usually sent via local mail. The most common problems are missing environment variables and a different home directory. All this is mentioned in the linked question and its answers, yet I see nothing about it in your question. – Gerald Schneider Jul 31 '19 at 06:29
  • @GeraldSchneider Where should I see the error ? I do not know about environment variable error where can I see this? what email ID should i provide ? and for home directory I have already changed this to root as I mentioned. Can you help to understand what information you need more here ? I can provide – Roy Jul 31 '19 at 06:41
  • Also I dont understand when I login manually and execute it works and after that if I set the cron job to run in next 15 minutes it works as well ...? – Roy Jul 31 '19 at 06:44
  • Hold on... @Roy, am I reading that last comment correctly, and the crontab in the question _sometimes_ does work from `cron` (even though it usually doesn't)? If that's correct, then it seems like very important information and could rule out the usual cron problems of having different environment settings, so I would recommend editing the question to 1) state that it sometimes works and 2) describe as fully as you're able the circumstances under which it works. – Dave Sherohman Jul 31 '19 at 07:42
  • 1
    @DaveSherohman Hello I have updated the cases :-) can you have a look and advice – Roy Jul 31 '19 at 10:36
  • 1
    What do the logs created by the cron job say? – Jenny D Jul 31 '19 at 19:54
  • @JennyD they have no error, everything executes well ...all my print statements work after a function. – Roy Aug 01 '19 at 03:43
  • How can I check if there is env variable issue – Roy Aug 01 '19 at 08:41

0 Answers0