Wednesday, 5 May 2010

A quick ‘n cheap geocoding process using Talend, Soap and web service …

Hi all,
Today, let’s play with Talend Open Studio (TOS). Here is a quick and cheap way to process ip adress geocoding with Talend, by using web service call. For this purpose, I will call an ip lookup service provided by ippages.com. Using the free version, you will be able to send 20000 ip geocoding queries, that’s good.
First, let’s design our job. I use a tWebServiceInput step, configured like below.
As you can see, we have to tell this step where the WSDL is. In that case, the web service interface is : http://www.ippages.com/soap2008/lookupserver.php?wsdl. Then, don’t forget to call the appropriate method : callshowmyip_lookup. Finally, the web service is waiting for 7 parameters : you put the ip adress on the first one and – for instance – the country code in the fourth one. Make sure you will store the webservice answer into a dedicated row (for me it is named “line” and your tWebServiceInput step is now ready.
The ippage web service will send you back an XML answer. You will have to manage / process this XML.
Now you can export your ip geocoding data into a new xml file, like in my simple example. Or, better, you can use a tExtractXMLField to parse – using XPATH - your XML and distribute the data you need for any further step. Your XPATH code will start with \root\row\line.
The XML answer back from ippages will look like this :
The free usage of ippages has some restrictions : some informations are not available and you have to subscribe to a commercial offer to have them. If you have a look to the XML up there, you will notice that you still have a lot of interesting informations with the free offer.
Happy geocoding.

No comments: