Scraping specific text from a webpage using xpath

Question

I've searched and tried multiple ways to get this but I'm not sure why it won't find most of the information on the webpage.

Page to scrape: https://m.safeguardproperties.com/

Info needed: Version number for PhotoDirect for Apple (currently 4.4.0)

Xpath to text needed (I think) : /html/body/div[1]/div[2]/div[1]/div[4]/div[3]/a

Attempts:

<?php

$file = "https://m.safeguardproperties.com/";
$doc = new DOMDocument();
$doc->loadHTMLFile($file);

$xpath = new DOMXpath($doc);

$elements = $xpath->query("/html/body/div[1]/div[2]/div[1]/div[4]/div[3]/a");

echo "<PRE>";

if (!is_null($elements)) {
  foreach ($elements as $element) {
      var_dump ($element);
    echo "<br/>[". $element->nodeName. "]";

    $nodes = $element->childNodes;
    foreach ($nodes as $node) {
      echo $node->nodeValue. "\n";
    }
  }
}

echo "</PRE>";

?>

Second Attempt:

<?PHP
$file = "https://m.safeguardproperties.com/";
$doc = new DOMDocument();
$doc->loadHTMLFile($file);

echo '<pre>';

  // trying to find all links in document to see if I can see the correct one
  $links = [];
  $arr = $doc->getElementsByTagName("a");

  foreach($arr as $item) { 
    $href =  $item->getAttribute("href");
    $text = trim(preg_replace("/[\r\n]+/", " ", $item->nodeValue));
    $links[] = [
      'href' => $href,
      'text' => $text
    ];
  }

var_dump($links);
echo '</pre>';
?>

Show source
| dom   | php   | html   | extract   | xpath   2017-10-04 23:10 1 Answers

Answers to Scraping specific text from a webpage using xpath ( 1 )

  1. 2017-10-05 00:10

    For that particular website, the versions are being loaded from JSON data client side, you won't find them in the base document.

    http://m.safeguardproperties.com/js/photodirect.json

    This was located by comparing the original document source to the finished DOM and inspecting the network activity in the developer console.

    $url = 'https://m.safeguardproperties.com/js/photodirect.json';
    $json = file_get_contents( $url );
    $object = json_decode( $json );
    echo $object->ios->version; //4.4.0
    

    Please respect other websites and cache your GET request.

Leave a reply to - Scraping specific text from a webpage using xpath

◀ Go back