Automating Web Navigation with the WebVCR

Recent developments in Web technology such as the inclusion of scripting languages, frames, and the growth of dynamic content, have made the process of retrieving Web content more complicated, and sometimes tedious. For example, Web browsers do not provide a method for a user to bookmark a frame-based Web site once the user navigates within the initial frameset. Also, some sites require users to go through a sequence of steps in order to access their data. These steps cause dynamic pages to be generated, often with session- ids encoded in the URL or embedded in the page (once the session times out, the URL can no longer be used to access the page). These hard-to-reach pages are often the best candidates for a shortcut, because significantly more effort is required to reach them than to reach a standard page which has a well-defined URL.