Results 1 to 2 of 2

Thread: Multi URL Metatag Scraper

  1. #1
    Join Date
    Feb 2010
    Posts
    25
    Thanks
    3
    Thanked 0 Times in 0 Posts

    Default Multi URL Metatag Scraper

    Dear all,
    Can somebody help me to modify this script to scrape more than just one page?
    Thank you very much advance.


    <?php
    $url = "http://www.example.com";
    $fp = fopen( $url, ‘r’ );
    $content = "";

    while( !feof( $fp ) ) {
    $buffer = trim( fgets( $fp, 4096 ) );
    $content .= $buffer;
    }

    $start = '<title>';
    $end = '<\/title>';

    preg_match( "/$start(.*)$end/s", $content, $match );
    $title = $match[ 1 ];
    $metatagarray = get_meta_tags( $url );
    $keywords = $metatagarray[ "keywords" ];
    $description = $metatagarray[ "description" ];

    echo "<div><strong>URL:</strong> $url</div>\n";
    echo "<div><strong>Title:</strong> $title</div>\n";
    echo "<div><strong>Description:</strong> $description</div>\n";
    echo "<div><strong>Keywords:</strong> $keywords</div>\n";
    ?>

  2. #2
    Join Date
    Nov 2006
    Location
    Northeast USA
    Posts
    408
    Thanks
    8
    Thanked 30 Times in 28 Posts

    Default

    PHP Code:
    <?php
    $urls 
    = array ("http://google.com""http://yahoo.com""http://bing.com");
    foreach(
    $urls as $url){
    $fp fopen$url, &#8216;r’ );
    $content "";

    while( !
    feof$fp ) ) {
    $buffer trimfgets$fp4096 ) );
    $content .= $buffer;
    }

    $start '<title>';
    $end '<\/title>';

    preg_match"/$start(.*)$end/s"$content$match );
    $title $match];
    $metatagarray get_meta_tags$url );
    $keywords $metatagarray"keywords" ];
    $description $metatagarray"description" ];

    echo 
    "<div><strong>URL:</strong> $url</div>\n";
    echo 
    "<div><strong>Title:</strong> $title</div>\n";
    echo 
    "<div><strong>Description:</strong> $description</div>\n";
    echo 
    "<div><strong>Keywords:</strong> $keywords</div>\n";
    }
    ?>
    Now theoretically this should work, but I could not even get the single url to work. Meta Tags are becoming very rare on websites, you may want to use an if statement to check if the site even has meta tags.
    Last edited by fileserverdirect; 07-15-2010 at 06:58 PM. Reason: fixed an http tag
    -Ben -- THE DYNAMIC DRIVERS
    My Links: My DD Profile||My Youtube Video Tutorials||DD Helping Coders||DD Coders In Training
    I told my client to press F5, the client pressed F, then 5, *facepalm*

  3. The Following User Says Thank You to fileserverdirect For This Useful Post:

    young_coder (07-16-2010)

Bookmarks

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •