Log in

View Full Version : Multi URL Metatag Scraper



young_coder
07-15-2010, 01:10 PM
Dear all,
Can somebody help me to modify this script to scrape more than just one page?
Thank you very much advance.


<?php
$url = "http://www.example.com";
$fp = fopen( $url, ‘r’ );
$content = "";

while( !feof( $fp ) ) {
$buffer = trim( fgets( $fp, 4096 ) );
$content .= $buffer;
}

$start = '<title>';
$end = '<\/title>';

preg_match( "/$start(.*)$end/s", $content, $match );
$title = $match[ 1 ];
$metatagarray = get_meta_tags( $url );
$keywords = $metatagarray[ "keywords" ];
$description = $metatagarray[ "description" ];

echo "<div><strong>URL:</strong> $url</div>\n";
echo "<div><strong>Title:</strong> $title</div>\n";
echo "<div><strong>Description:</strong> $description</div>\n";
echo "<div><strong>Keywords:</strong> $keywords</div>\n";
?>

fileserverdirect
07-15-2010, 04:59 PM
<?php
$urls = array ("http://google.com", "http://yahoo.com", "http://bing.com");
foreach($urls as $url){
$fp = fopen( $url, ‘r’ );
$content = "";

while( !feof( $fp ) ) {
$buffer = trim( fgets( $fp, 4096 ) );
$content .= $buffer;
}

$start = '<title>';
$end = '<\/title>';

preg_match( "/$start(.*)$end/s", $content, $match );
$title = $match[ 1 ];
$metatagarray = get_meta_tags( $url );
$keywords = $metatagarray[ "keywords" ];
$description = $metatagarray[ "description" ];

echo "<div><strong>URL:</strong> $url</div>\n";
echo "<div><strong>Title:</strong> $title</div>\n";
echo "<div><strong>Description:</strong> $description</div>\n";
echo "<div><strong>Keywords:</strong> $keywords</div>\n";
}
?>

Now theoretically this should work, but I could not even get the single url to work. Meta Tags are becoming very rare on websites, you may want to use an if statement to check if the site even has meta tags.