1. Introduction

Using the Web API made available to them by Google, one of the major web search engines, software developers can now automate web searches to build web monitoring applications. Offered as a free beta service, the Google Web API has several limitations. Each developer has to register to obtain a license key. With that key, you are entitled to a maximum of 1000 automated searches per day. Each search is limited to a maximum of 10 results, and you cannot search beyond the 1000th result.

2. Tools

We will use Visual Foxpro as the development tool and database, together with Microsoft's SOAP toolkit to access Google Web API service using the Simple Object Access Protocol (SOAP). The code listed in this note is limited to the bare minimum necessary to collect and organise the data and should not be used in a production application.

3. Getting results from Google

lcGoogleWSDL = URL of Google API Web Service WSDL file
lcGoogleKey = developer's Google license key
lcQuery = search string
llFilter, llAdultFilter, lcRestrict, lcInputChar, lcOutputChar = additional search parameters

With the Visual Foxpro code below, we get a first set of results

oSoap = CREATEOBJECT("MSSOAP.SoapClient")
oSoap.msSoapInit(lcGoogleWSDL)
oResults = oSoap.doGoogleSearch(lcGoogleKey, ;
    lcQuery,lnStart,lnEnd,llFilter, ;
    lcRestrict,llAdultFilter, ;
    lcLanguage,lcInputChar,lcOutputChar)

4. Exploring the results objects

Using VFP7's IntelliSense, we can explore the oResults objects
The XML data returned by Google have the following structure :

5. Collecting data for web monitoring

As Google API returns results in batches of 10 only, we do a first pass to determine the number of results, then do multiple passes to collect all results in successive batches of 10.

For each URL found by Google, we check if it has been found in a previous search. In that case, the corresponding record is updated, after transferring data from the last search in the "before last" fields. 
Other URLs are added to the Result table with data from the last search.