Web Services Aggregator
•
2 likes•545 views
This is a acadamic class project on web services aggreagtion.
![CmpE 275Fall 2009 Project 2 WS - Content Aggregator &Provider(WSCAP) Search and Retrieval Services Specifications Document Team Name:Ants Dhaval Patel,[object Object]](https://image.slidesharecdn.com/webservicesaggregator-12660304736422-phpapp02/85/Web-Services-Aggregator-1-320.jpg)
![For Images: each ID must start with I- e.g. I-1 ;S- for Sites, V- for Videos, D- for Docs.The Ring Site Service WSDL ImplSite Search ProviderJMS QJMS QSite StoreDBCLIENT{OR}GoogleSearchDoc Service WSDL ImplDoc Search ProviderJMS QJMS Q{OR}Doc StoreDBCLIENTVideo Service WSDL ImplVideo Search ProviderJMS QJMS QVideo StoreDBCLIENT{OR}Image Service WSDL ImplImage Search ProviderJMS QJMS QImage StoreExternal ImageServiceDBCLIENT{OR}Doc Ring WSDLSite Ring WSDL Image Ring WSDLVideo Ring WSDL Brief Introduction The Ring consists of the 4 services, Ants Image Service, Ants Video Service, Ants Doc Service and Ants Site service. Each service has two WSDLs, Ants WSDL acts as a client interface and the AntsRing WSDL is used as an inter-service interface. For better understanding we shall be calling the Client interfacing WSDL as the External WSDL and the service interfacing WSDL as the Internal WSDL. As it can be seen in the diagram, the moment client sends a search request to any of the four services, the service implementation sends the request to the listening queue of Provider Search which then performs the task and then it calls the Internal WSDL of the next service and passes its search results as well. This task is extended and implemented across each service and the ring n/w of web services is created. Providers As shown in the diagram above, each of the services has twoWSDL implementations and a ‘provider’ component to it. The client and inter-service interfacing is always done through the WSDL. So, both the WSDL implementationsare hosted within tomcat and this is the point of contact for the client and the fellow service. The Internal WSDL on receiving any request from a client passes on the request to the JMS queue of its provider, which on processing the request,forwards the search result to the next service in the ring by connecting to its Internal WSDL. The Internal WSDL's job if to simply interface with the service contacting it and to pass the request to the Provider IMPL via its listening queue. The Providers can be configured using the file: SERVICEHOME/resources/Provider.cfg.xml Why separate Providers? Provider provides another layer of abstraction, which enables us to take the logic out of the WSDL implementation JMS Each web service uses an JMS queues to communicate. A SearchTransferObject(STO) is created and the Search request, SearchResponse and originator attributes are set in it. This object is then passed from the provider search to the search of External WSDLwhich then removes the search result from the STO and sends it to the Client.Please not that the Queue names have been kept constant for the sake of demonstration only. The queue name in future actualization of the project can be explicitly specified in Provider.cfg.xml file. And also for the sake of demonstration we have used only one JBoss server to host all the queues. We have provided the option of changing the path of JBoss in Provider.cfg.xml, therefore if interested one can run four Jboss servers run the implementation as well. About the Search Results Each provider uses its own database to append its results to the search packets that it gets for processing and also interfaces with external services, wherever applicable, to get results from there and append those results also to its results. Client’s perspective: Finding and Getting The Final SearchResponse External ImageService The Image service also interfaces with the external image service to search images and download them. External Search Engine Search for Sites The Image service also interfaces with the external image service to search images and download them. GoogleSearch The Ants Site service also connects to the Google web services to collect the site results from the Google search engine and appends the results in the Ants service format and sends the result to the Client along with other results. In order to get Google search results we are using the Google Ajax API (REST Based). The URL that is required in order to retrieve the search results is shown on the line below.
http://ajax.googleapis.com/ajax/services/search/web?start=0&rsz=large&v=1.0&q=?
Is the query that needs to be searched. The AJAX Search API URL, that performs the web search Specifies the start of the search result and the size specifies the result limit.In this case start is 0 and large says to bring back 8 results. The following lists down the steps to connect and retrieve results from the Google API. ,[object Object]](data:image/gif;base64,R0lGODlhAQABAIAAAAAAAP///yH5BAEAAAAALAAAAAABAAEAAAIBRAA7)
Recommended
Recommended
More Related Content
Similar to Web Services Aggregator
Similar to Web Services Aggregator (20)
Web Services Aggregator
- 3. Instantiate URLConnection and open connection.
- 4. Add a string referrer as any website name as a request property to the connection.
- 5. Then get the input stream which has the query result.
- 6. The stream that comes back is in JSON, the input stream for the q = spider man would look something like this.
- 8. {"results":[
- 13. "title":"003cb003eSpider003c/b003e-003cb003eMan003c/b003e - Wikipedia, the free encyclopedia","titleNoFormatting":"Spider-Man - Wikipedia, the free encyclopedia",
- 14. "content":"003cb003eSpider003c/b003e-003cb003eMan003c/b003e is a fictional Marvel Comics superhero. The character was created by writer and editor Stan Lee and artist and co-plotter Steve Ditko. 003cb003e...003c/b003e"},
- 15. .
- 16. .
- 17. .
- 20. Above is an example on one result sent by Google service, every call generates eight such results. Every time the client contacts the WS-Ants service, the Site service requests the Google service. From the resulting JSON stream, we parse only the content and the URL and send it to the client.
- 21. When the client requests to download any site content that was sent by Google, our site service tries to crawl that external page and send the HTML content to the user in Base 64 Binary, if the crawling is disabled for that URL requested by the Client, the following HTML page is generated and sent to the client.