Design Patterns for Web App Screen Scraping?

 
50   50
Jun 06, 2011
 
Hi All,

I am developing a Windows Form application to screen scrape a complex web application that has Ajax in its pages.

In fact, the web app has multiple pages and I need to navigate & perform click operation many times to get the appropriate page data.
I am thinking about a Design Pattern to implement this. Can anyone guide as to which Pattern or Patterns would be suitable for this?

Thanks,
Neetha



56   94.4
Jun 07, 2011
Neetha:

Regex is your HTML screen scraping friend. 
No specific Patterns come to mind.

Good luck!
 1 comment
 
John I appreciate your suggestion & thank you. I am using C# and Web Browser Control to extract the page content. I was success in processing all the ajax requests on the pages & get the data. Now I am planning for developing an API so that other tool/project can use this API & get required data. I thought Singleton pattern but not sure whether this is good decision or not. Any help... Thanks, Neetha --- Neetha Adehalli  Jun 07, 2011

80   96.3
Aug 26, 2011
Building complex objects and exposing as API - Builder pattern is the best ... Please share the details of the expected features...c an try to put in a workable solution..


- Tarriq Ferrose Khan

Aug 30, 2011
Hi Neetha,

What about using visitor pattern ? from what i understand about the context is that you have collection of pages that needs to be visited to collect data so my suggestion is to use visitor pattern where you visit each page to collect the required data.

Thanks & Regards
Pubudu Thilakarathna