Home  /  Questions  /  Question

50   50
Jun 06, 2011

Design Patterns for Web App Screen Scraping?

Hi All,

I am developing a Windows Form application to screen scrape a complex web application that has Ajax in its pages.

In fact, the web app has multiple pages and I need to navigate & perform click operation many times to get the appropriate page data.
I am thinking about a Design Pattern to implement this. Can anyone guide as to which Pattern or Patterns would be suitable for this?


56   94.9
Jun 07, 2011

Regex is your HTML screen scraping friend. 
No specific Patterns come to mind.

Good luck!
 1 comment
John I appreciate your suggestion & thank you. I am using C# and Web Browser Control to extract the page content. I was success in processing all the ajax requests on the pages & get the data. Now I am planning for developing an API so that other tool/project can use this API & get required data. I thought Singleton pattern but not sure whether this is good decision or not. Any help... Thanks, Neetha --- Neetha Adehalli  Jun 07, 2011

90   96.5
Aug 26, 2011
Building complex objects and exposing as API - Builder pattern is the best ... Please share the details of the expected features...c an try to put in a workable solution..

- Tarriq Ferrose Khan

Aug 30, 2011
Hi Neetha,

What about using visitor pattern ? from what i understand about the context is that you have collection of pages that needs to be visited to collect data so my suggestion is to use visitor pattern where you visit each page to collect the required data.

Thanks & Regards
Pubudu Thilakarathna

0   0
Oct 02, 2017
Screen Scraping is an important part of data migration and integration scenarios. It enables modern apps to talk with legacy apps that do not offer an API and is the complement to the data entry side of automation.Get all latest apps for free on Appvn app store. Appvn app download.