How Do You Write a Python Program to Find if The “Read more” Preview on a Web Page is Visible for Every Web Page Previewed?

Problem scenario
Sometimes the "Read more" or "Continue reading" hyperlink in the page preview of your website (e.g., when you search for a word), is not visible. You know that some of your web page previews on your website don't have the "Read more" hyperlink at the end. This makes the preview appear to have the entire web page. You need to identify the postings that don't have the "Read more." How do you use Python to find the pages that have previews that have no "Read more" hyperlink?

Solution

# Change the "http://www.domainname.com/miscellaneous/page/" to the URL that will have the format http://www.domainname.com/miscellaneous/page/25 
# where 25 is the number of pages to iterate through.
# Change the "25" to be the page number associated with the highest url in this format: http://www.domainname.com/miscellaneous/page/25
# If you have a different number of page previews per page of search results than 10, change the 10 to the number your website uses.


import re
import requests
listofurls = []

tempstring = "http://www.domainname.com/miscellaneous/page/"
for i in range(25): 
  url = tempstring + str(i)
  listofurls.append(url)

searchterm = "Read more"

def finder(yoururl, searchterm):
    r = requests.get(yoururl)
    found = str(len(re.findall(searchterm, r.text)))
    if found:
      if found != '10':
        print("The string/pattern '" + searchterm + "' was found " + found + " times when searching " + yoururl)


for x in listofurls:
  finder(x, searchterm)

print("The format of the URL returned in the output will show you 25 (or 10 or whatever number you have) previews.  
You will have to manually find which posting has no "Read more" hyperlink.  We have found making a back up of the posting, deleting it completely, and reposting it as a new page to be the most effective way of making the "Read more" hyperlink appear.")


# You may also want to see "How Do You Get WordPress Preview Postings to Show ">> Read more..."?"

If the page preview hyperlink is not visible and you are using WordPress, see this posting.

Leave a comment

Your email address will not be published. Required fields are marked *