My target is to scrape data according to the following list of attributes:
However, when scraping data from the website, only the first one of a list of URLs can be successfully executed. Even if I use import.io, I find that the search result webpages paginated in accordance with either one of the 5 search result patterns under different parameter settings have the same URLs with parameter equal to Any, Dealer, Private, sold or For Sale.
It seems that real URLs of the subsequent webpages are deliberately hidden or the sever of the websites encrypts the corresponding information so that suggested methods from import i o help article do not work. How to do in order to successfully scrape the data? Where to find the real URLs (or actual URL patterns)?