I’ve noticed that Image drop shadow wordpress plugin did not work properly with the large images. I’ve fixed this bug for images up to 800×600 pixels size and I’ve added a donation url. So if you like this plugin you can make a donation to it.

Download Image Drop Shadow 1.1 now (zip, 26 kb)

Donate to this plugin

Related posts

• Image Drop Shadow plugin for wordpress
• All in One SEO Pack + runPHP = short meta descriptions

If you have a number of feeds from multiple sources (blogs, news sites, etc.) and want to merge them to one feed I have ready solution for you. It can be done very easy and quickly with Yahoo Pipes. Just create “Fetch Feed” module in new just created pipe and add your feeds to it like on the image below.

If you have a lot of feeds you may separate them into a few “Fetch Feed” modules and then use “Union” module like on the image below.

Finally you can sort the items of result feed by publish date using “Sort” module like on the image below.

As a result we got the feed containing the items of multiple feeds and the items are sorted by publish date. You can find this pipe here.

Most Commented Posts

• PageRank Spider
• Finally, Mass PageRank Checker tool!
• Mass PageRank Checker
• PHP Adsense monitor update
• Mass PageRank Checker updated!

As you know 1.5 month ago I’ve started PHP AdSense account library project. The current version of the library is 0.2, which includes new features and improvements. Here is the small stats for the project:

• 291 Visits
• 127 Visitors
• 86 downloads

Visitors by country:

1. Russia
2. Ukraine
3. United States
4. Germany
5. Czech Republic
6. India
7. United Kingdom
8. Denmark
9. France

3 issues were posted and 1 bug was fixed thanks to one of the users.

I think the project is on the right way and it’s useful for people.

Related posts

• PHP account adsense monitor
• Backing up MySQL tables under extremal conditions
• Add Perl and PHP search engine plugins to your browser
• How old is your site?
• How to check Google PageRank in PHP

I’ve started PHP AdSense account library project on Google Code. There are the sources from my previous blog post now which you can download as the version 0.1. I’ve choose to release the library under GNU GPL 3 license.

I’m planning to continue the development of this library, add new features, write the tutorial and you can help me to do it :)

The following tasks are in my nearest plans:

• Write the tutorial and user manual
• Add AdSense for Search support
• Develop flexible methods for report retrieving

Visit PHP AdSense account library project

Most Commented Posts

• PageRank Spider
• Finally, Mass PageRank Checker tool!
• Mass PageRank Checker
• PHP Adsense monitor update
• Mass PageRank Checker updated!

One of my readers (Thanks Herick!) has noticed that Adsense monitor became unable to login to adsense account. Google has made small changes in authorization, but I have fixed the script and now it works perfect.

And one thing that I have not mentioned in previous post – you must create ‘tmp’ folder in the script’s folder and add write permissions to it. The script will use this folder for temporary cookies files.

Download Adsense monitor script

Most Commented Posts

• PageRank Spider
• Finally, Mass PageRank Checker tool!
• Mass PageRank Checker
• PHP Adsense monitor update
• Mass PageRank Checker updated!

Paul Maunders said me about cool scraping and crawling framework based on python – Scrapy. I’ve tried it and developed about 20 spiders. It’s really nice stuff but it would be great if scrapy’s developers will add some more features:

1. Rules for form submitting
2. Proxy server support

Visit scrapy website

Most Commented Posts

• PageRank Spider
• Finally, Mass PageRank Checker tool!
• Mass PageRank Checker
• PHP Adsense monitor update
• Mass PageRank Checker updated!

There is a nice PHP class for the retrieving the data from Google Adsense account like impressions, clicks, ctr, ecpm and earnings – PHP account adsense monitor. But it does not work now, seems Google updated the authorization system for Adsense. I’ve fixed it a bit and now you can use it in your web applications.

A small example:

<?php
include('adsense.php');

$adsense = new AdSense();
if ($adsense->connect('username', 'password'))
{
  $data = $adsense->yesterday();
  var_dump($data);
}
else
{
  die('Could not login to AdSense account!');
}
?>

This code will output something like this:

array(5) {
  ["impressions"]=>
  string(2) "49"
  ["clicks"]=>
  string(1) "0"
  ["ctr"]=>
  string(5) "0.00%"
  ["ecpm"]=>
  string(5) "$0.00"
  ["earnings"]=>
  string(5) "$0.00"
}

You can download the script here.

Related posts

• PHP AdSense account library news and stats
• Backing up MySQL tables under extremal conditions
• Add Perl and PHP search engine plugins to your browser
• How old is your site?
• How to check Google PageRank in PHP

You can find the beginning of this post here – How to develop a good scraper on Perl – Lesson 1 and How to develop a good scraper on Perl – Lesson 2

If your scraper works as just one thread it can be very slow to scrape a large site for a short time. Perl has a set of classes that will help you to run as many threads as you want simultaneously:

use threads;

#run 'scrape' function in 10 threads
my @threads;
for (1..10) {
  my $thr = threads->create(\&scrape);
  $thr->detach;
  push @threads, $thr;
}

You can share the variables between threads:

use threads::shared;

my $counter :shared = 0;

sub scrape {
  ...
  {
    #lock 'counter' variable and modify it's value
    lock $counter;
    $counter++;
  }
  #here 'counter' variable will be unlocked
  ...
}

It’s recommended to start one additional thread which will control the other ones:

sub control {
  while (1) {
    for (my $i = 0; $i < 10; $i++) {
      #check if the threads are alive
      if ($threads[$i] eq undef || !$threads[$i]->is_running) {
        #if some threads are dead, run them again
        print "Some of the threads is stopped! Rerunning...\n";
        $threads[$i] = threads->create(\&scrape);
        $threads[$i]->detach unless ($threads[$i] eq undef);
      }
      sleep(1);
    }
  }
}

And the last tasty thing is thread queues. You can just add urls to the queue and your threads will get them from queue or wait if the queue is empty:

use Thread::Queue;

#create queue
my $data_queue = Thread::Queue->new();

sub scrape {
  #get url from the queue or wait if the queue is empty
  while (my $params = $data_queue->dequeue) {
    my $url = $params->[0];
    ...
  }
}

#add url to the queue
$data_queue->enqueue(['http://www.example.com/']);

You can find the documentation for all these classes here: threads class, threads tutorial, threads::shared, Thread::Queue.

The books I recommend:

Related posts

• How to develop a good scraper on Perl – Lesson 2
• How to develop a good scraper on Perl – Lesson 1
• Add Perl and PHP search engine plugins to your browser

You can find the beginning of this post here – How to develop a good scraper on Perl – Lesson 1

2. Treebuilder class allows you to convert html page to the tree structure and perform the operations with the tree nodes like searching, walking through the nodes etc. If you will add XPath support to the Treebuilder class, you will get very powerful tool for html parsing. Look at the example below.

use HTML::TreeBuilder::XPath;

#create treebuilder object and parse the html code from content variable
my $tree = HTML::TreeBuilder::XPath->new_from_content($content);

#find all nodes 'div->h1' in the tree, div node must have 'class' attribute
#that will match the '/details\d+/i' regular expression
if (my @name_nodes = $tree->findnodes('//div[@class=~/details\d+/i]/h1')) {
  #get the trimmed text value from the first result node
  $name = $name_nodes[0]->as_trimmed_text;
}

You can find the full documentation here: a href=”http://search.cpan.org/~petek/HTML-Tree-3.23/lib/HTML/TreeBuilder.pm”>HTML::TreeBuilder, HTML::TreeBuilder::XPath

The books I recommend:

Related posts

• How to develop a good scraper on Perl – Lesson 3
• How to develop a good scraper on Perl – Lesson 1
• Add Perl and PHP search engine plugins to your browser

If you want to develop a good scraper Perl can be very good solution for you. It has all you need for these purposes: mechanize library, treebuilder class and threads support.

1. Mechanize library is a complex library for automating interaction with websites. It completely simulates user’s activity like clicking on links and submitting forms and has a lot of another useful features. Let’s look at the code:

use WWW::Mechanize;

#create mechanize object
my $mech = WWW::Mechanize->new();

#set user agent string
$mech->agent('Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.1');

#go to http://www.example.com/
$mech->get('http://www.example.com/');

#click on link 'Some text'
$mech->follow_link(text => 'Some text');

#fill and submit the form
$mech->submit_form(
  form_name => 'search',
  fields => { query => 'Some text' },
  button => 'Search Now'
);

Of course, there were basic features used in the example above, you can find full documentation here.

The books I recommend:

Related posts

• How to develop a good scraper on Perl – Lesson 3
• How to develop a good scraper on Perl – Lesson 2
• Add Perl and PHP search engine plugins to your browser