Method for viewing search engine spider visit records in WordPress blogs

Nov 17, 2019#WordPress26

AI Translation

This post is translated from Chinese into English through AI.View Original

AI-generated summary

This article explains a method to view the visit records of search engine spiders on a WordPress blog. Instead of using plugins, the method involves adding code to the functions.php file in the theme directory. The code identifies popular search engine spiders and logs their visit records in a robotslogs.txt file. By accessing this file, users can see detailed information about the spiders' visits.

Note: Many people want to know which spiders frequently visit their blog websites and how many times they visit. For this question, plugins are usually used to solve it. In fact, besides plugins, we can also view spider visiting records by using code. Here, I will explain the method. The code in this article was found online, and I added a few mainstream search engine spiders myself.

Method#

First, put the following code into the functions.php file in the theme directory.

// Count spiders
function get_naps_bot(){
$useragent = strtolower($_SERVER['HTTP_USER_AGENT']);
if (strpos($useragent, 'Googlebot') !== false){
return 'Googlebot';
}
if (strpos($useragent, 'msnbot') !== false){
return 'MSNbot';
}
if (strpos($useragent, 'slurp') !== false){
return 'Yahoobot';
}
if (strpos($useragent, 'Baiduspider') !== false){
return 'Baiduspider';
}
if (strpos($useragent, 'sohu-search') !== false){
return 'Sohubot';
}
if (strpos($useragent, '360Spider') !== false){
return '360Spider';
}
if (strpos($useragent, 'Sosospider') !== false){
return 'Sosospider';
}
if (strpos($useragent, 'bingbot') !== false){
return 'bingbot';
}
if (strpos($useragent, 'Sogouspider') !== false){
return 'Sogouspider';
}
return false;
}
function nowtime(){
date_default_timezone_set('Asia/Shanghai');
$date=date("Y-m-d.G:i:s");
return $date;
}
$searchbot = get_naps_bot();
if ($searchbot) {
$tlc_thispage = addslashes($_SERVER['HTTP_USER_AGENT']);
$url=$_SERVER['HTTP_REFERER'];
$file="robotslogs.txt";
$time=nowtime();
$data=fopen($file,"a");
$PR="$_SERVER[REQUEST_URI]";
fwrite($data,"Time:$time robot:$searchbot URL:$tlc_thispage\n page:$PR\r\n");
fclose($data);
}

Then, create a txt file named robotslogs.txt in the root directory and set its permissions to 777. After that, visit http://your_domain/robotslogs.txt to see detailed spider visiting records.