Dec 28, 2007 at 3:34 AM
I am trying to create a library nWebDataExtractor that will

1. Allow user to specify URL(s) and the depth of linking
2. Crawl the specified list of url and extract web response
3. Crawl the links found on the specified list of url(s) upto specified depth level
4. Get web response data from those nested urls too
5. convert web response from HTML / XHTML/ XML to standard XML
6. apply a given xslt to extract the data into another xml (which can be consumed by a dataset)
7. the extracted data can be used anywhere user wants