Clarification on the federated search algorithm

[jrao1]

Hi,

I took a quick look at the code related to FederatorRunHandler, am I right to assume when performing federated search, the results of each search is mixed together only by 3 methods: sequentially, round robin and randomly? This means the federated search result doesn't at all consider the score or other sorting method when merging the results?

Thanks

[michaelsembwever]

Actually only sequentially is implemented in FederatorRunHandler. see handleRunningQuery(..).

FederatorRunHandlerConfig.Blend should ultimately be service provider interface (spi). The current code using an enum and a switch statement is unfortunate really. It would be nice to see it turned pluggable, replacing the enum with something extendable and some inversion of control in FederatorRunHandler so that it delegates out to the different implementations for blending. Such inversion of control pattern is common throughout the possom architecture.

Maybe you're interested in providing a patch here...?

[jrao1]

Ok, thanks for the clarification. I'm not sure how useful these blends would be helpful for people who actually wants federated search though, as I understand it federated search should merge the results and sort them by some factor, so that the results would look like they're coming from one search engine. Do you know any algorithm or design that can do this? Any open source implementation for them?

[michaelsembwever]

On Fri, 2012-05-11 at 06:20 -0700, raojx wrote:

I'm not sure how useful these blends would be helpful for people who
actually wants federated search though, as I understand it federated
search should merge the results and sort them by some factor, so that
the results would look like they're coming from one search engine. Do
you know any algorithm or design that can do this? Any open source
implementation for them?

The results indeed will look like they came from one search engine.
Merging the results and sorting them by whatever algorithm you want is
just to code away. Sesam has run with complicated federating before as
you allude to, and Possom is the framework to achieve it. Of course the
more complicated the merging and weighting is the longer the result list
from each individual back end search engine will need to be.

~mck

"The Matrix is the world that has been pulled over your eyes to blind
you from the truth."

| http://github.com/finn-no | http://tech.finn.no |