How can we get cron results?

[abtlang]

Hi, I've seen quite a few questions about a cron log, but I don't think I've ever seen any solution as to how to get the results of the cron job set up in version 2.0.1.

I have no problems with running the cron (check.php) and I've seen that it does indeed recognize broken links and reciprocal links that have been removed.

However, I've also seen that there doesn't seem to be any way to get a report on which links it has found to be broken, and on which reciprocal links have been removed.

Since I'm just setting up my directory, at the moment, with just 10 or so links, that's no problem, because I can go into the Admin Panel and see which link is now indicated as being not reciprocal. But, when I have hundreds of links, some of them reciprocal, some of them non-reciprocal, it's going to take me a long time to work out which links, if any, have changed status as a result of a cron job (I notice that if the cron job finds that a reciprocal link no longer exists, it simply marks the corresponding listing as non-reciprocal - but it doesn't tell you what it's done, and in fact it doesn't tell you whether it's done anything or not!).

Is there any way of having the cron send a report of the results? Or of having a log on the server that can be consulted?

Sorry if this is answered somewhere else - I've certainly spent a lot of time looking and I haven't found it!

Thanks in advance,

Terry.

[John Turner]

Hi abtlang and welcome.

I have implemented modification
After cronjob is finished
Email would be sent with results

open your '../cron/check.php'

***find following code

$query .= "OR (`last_check_date` IS NULL) ";
$query .= "ORDER BY `id` ASC LIMIT {$num_links}";
$links =& $gDb->getAll($query);

***add after it following code

$count_broken=0;
$count_reciprocal=0;
$count_pagerank=0;
$count_all=0;

***find following code

			if (empty($headers))
			{
				$link_header = 1;
			}
			else
			{
				$link_header = (int)$headers['Status'];

***add after it following code

				$count_broken++;

***find following code

					$content = get_page_content($url);
					if (!empty($content))
					{
						$recip_valid = (int)has_url($content, $settings['reciprocal_text']);

***add after it following code

						$count_reciprocal++;

***find following code

					$pr_content = trim(get_page_content($pr_url));
					if (preg_match('/^Rank_\d{1}:\d{1}:(\d+)$/', $pr_content, $matches))
					{
						$pagerank = (int)$matches[1];

***add after it following code

						$count_pagerank++;

***find following code

		$query .= '`last_check_date` = NOW() ';
		$query .= "WHERE `id` = {$link['id']}";
		$gDb->query($query);
	}
}
?>

***replace it with

		$query .= '`last_check_date` = NOW() ';
		$query .= "WHERE `id` = {$link['id']}";
		$gDb->query($query);
		$count_all++;
	}

$query = "SELECT `value` FROM `{$gDb->mPrefix}config` WHERE `name`='site_email'";
$admin_email =& $gDb->getOne($query);

$date_check = date("F j, Y, g:i a");
$subject="Cron report [{$date_check}]";
$body = "{$count_all} links checked\n
(broken links checked - {$count_broken})\n 
(reciprocal links checked - {$count_reciprocal})\n
(pagerank links checked - {$count_pagerank})";

mail($admin_email, $subject, $body, "From: {$admin_email}\r\n"."Reply-To: {$admin_email}\r\n");
}
?>

***Save and close it

Hope it helps.

[abtlang]

Thanks, John.

I'll try this and let you know (I imagine it will take a time so don't worry if you don't hear anything immediately).

Terry.

[John Turner]

You are welcome Terry.
Always at your services.

I'll be waiting for your results.

[Mark Brookes]

When developing the crons for ver 1.2 Vincent wrote some very good results logging procedures. I felt it was a shame that they were removed once the development phase was done.

In order to satisfy my host, and to not overload their server, I have had to tell check.php to only check 5 links at a time, so I could end up with 80+ emails telling me the results for each ckeck-run before the whole database of links had been checked.

What I would really like is something which said .... End of current major cycle, so send a email report of all results within that cycle- and highlight changes (eg new broken links, new non-reciprocal).

In fact, somewhere on the forum (or perhaps on the old forum) there is a discussion between Vincent & me of possible "follow-up actions" that the cron checks could/should do.

e.g if link is newly. broken, send link partner an email. If link has checked as broken twice, send link Partner an email saying that failure to correct may cause 'suspension' of link. If link has checked as broken 3 times send email & suspend link.
etc... etc.

(hint) I hope that ideas like these can be introduced in some future version


Regards
MArk

[abtlang]

Hi, John,

I've tested your solution for a few days, and I've seen that the only thing it really does is count the number of links each cron job checks.

What I'm really looking for is a way of knowing which are the broken links the cron job has found, and which faulty reciprocal links the job has found, that are no longer on the page where they should be.

Of the two, the broken links problem is not so bad, because there shouldn't be any broken links, so as soon as I see a link in the broken link list, then I know the cron job has found that one.

But the reciprocal links are a different problem. If some links are supposed to be reciprocal and others non-reciprocal, then if the cron job just changes the faulty ones it finds from "reciprocal links" to "non-reciprocal links", without notifying me which faulty link it found, then I'm not likely to spot it.

I can see two solutions:
1. A log sent to the directory administrator at the end of the cron job indicating which broken links were found and which faulty reciprocal links were found (not how many, but which ones).
2. If that is not possible, it might be feasible to create a new category of links, named "temp non-reciprocal links", and get the cron job to move any faulty reciprocal links it finds to there, rather than directly to "non-reciprocal links". (I seem to remember this solution from an earlier thread, but it seems to have disappeared - perhaps it's what Mark was talking about in his post).

Is there any chance of either of these two solutions being implemented?

Many thanks, as always,

Terry.

[John Turner]

Hi abtlang,

Sorry for delay.
In compliance with your requirements, here the solution:

It is necessary to hack your ../cron/check.php
***find following code

$query .= "OR (`last_check_date` IS NULL) ";
$query .= "ORDER BY `id` ASC LIMIT {$num_links}";
$links =& $gDb->getAll($query);

***add after it following code

$broken = '';
$nonreciprocal = '';

***find following code

			if (empty($headers))
			{
				$link_header = 1;
			}

***replace it with following code

			if (empty($headers))
			{
				$link_header = 1;
				$broken .= $link['url'].'\n';
			}

***find following code

					$content = get_page_content($url);
					if (!empty($content))
					{
						$recip_valid = (int)has_url($content, $settings['reciprocal_text']);
					}

***replace it with following code

					$content = get_page_content($url);
					if (!empty($content))
					{
						$recip_valid = (int)has_url($content, $settings['reciprocal_text']);
						if($recip_valid == 0)
						{
							$nonreciprocal .= $link['url'].'\n';
						}
					}

*** find following code

$date_check = date("F j, Y, g:i a");
$subject="Cron report [{$date_check}]";
$body = "{$count_all} links checked\n
(broken links checked - {$count_broken})\n 
(reciprocal links checked - {$count_reciprocal})\n
(pagerank links checked - {$count_pagerank})";

*** replace it with following code

$date_check = date("F j, Y, g:i a");
$subject="Cron report [{$date_check}]";
$body = "{$count_all} links checked\n
(broken links checked - {$count_broken})\n 
(reciprocal links checked - {$count_reciprocal})\n
(pagerank links checked - {$count_pagerank})";
if($broken)
{
$body .= 'Broken \n'.$broken.'\n';
}
if($nonreciprocal)
{
$body .= 'Non reciprocal\n'.$nonreciprocal.'\n';
}

*** SAVE and close it.

You'll receive email with urls for broken and nonreciprocal links, if cron-verification will determine this.

Hope it helps.

[abtlang]

Thanks again, John.

As before, I'll try it out and let you know how it goes.

Best regards,

Terry.

[Mark Brookes]

Hello John,

An interesting development ... I have some questions

[1] My cron job is limited to 5 checks at a time to avoid overloading my host's server - It takes about 80 cron-jobs to complete checking ALL my links - is there any way to send me a summary email at the end of ALL the checks?

[2]
Please can you explain the numbers reported as

(broken links checked -)
(reciprocal links checked -)
(pagerank links checked -)

as they dont seen to tie up with the results which appear in the _links Table.

please see my attachments for a specific example

I think the numbes are about 'successful' results,
in which case I think
Broken Links Checked - Results OK = 4 (agreed)
Reciprocal Links Checked - Results OK = 3 (not 4 as reported in email)
Pagerank Links Checked - Results OK = 4 (agreed)

About the Warning reports:-

Warning these links are Broken- (ok)
* Please can it start each url on a new line

Warning these links fail Reciprocal Test:- (ok)
* Please can it start each url on a new line

*Please can there be report like....
Warning Partner homepage Pagerank < = 0



Regards
Mark

[abtlang]

Hello again, John.

First, thanks very much for the work done so far - now it is just what I wanted.

However, I have a slight problem - the email report doesn't include the broken links it finds. I notice in the attachment sent by Mark that his report does include a report on broken links found, so I assume I've made a mistake in my file. I attach a copy of my cron/check.php file after the changes I made - I wonder if you'd mind having a look at it to see if there's anything wrong.

And if you could answer Mark's request about how to get the broken links and reciprocal links reports onto separate lines, that would be the icing on the cake!

Thanks once more,

Terry.