I hired a developer to create a program for me to scrape data from several sites into an Excel spreadsheet. Apparently, he wrote the program in Visual Basic 6 using IE automation. The program is VERY slow. And every time one of the websites makes a minor HTML change, the program needs to be updated.

Upon hearing this, another developer just shook his head and said it should have been writen using Borland Delphi 7, using Internet sockets.

All this is Greek to me.

In simple terms, what program is best at extracting data from websites? What should I be looking for in the development of such a script?

__________________

---

VB application is slower if we compare it to Delphi , but it doesn't make any application which build with Delphi is always fast. It depends on their algorithms.
I think 'the another developer' build their application with Borland Delphi, so they said Delphi is fastest than VB.
I use Internet Sockect for extracting XML from Amazon for my Mp3 tagger (I use Delphi), but i think whatever programming language we are using, we can't increase the extraction time anymore.
I admit, IE Automation is slower than Internet Socket in Delphi, but you can ask your VB developer to change its way in extracting some data from internet, don't use IE automation.
Maybe there's another problems in your VB application, not just because IE automation.

__________________
Powered by Yahoo! Answers

---