Decentralized load balancing for highly irregular search problems
Di Fatta, G. and Berthold, M.R. (2006) Decentralized load balancing for highly irregular search problems. Microprocessors and Microsystems , 31 (4). pp. 220-226. ISSN 0141-9331
Full text not archived in this repository.
To link to this item DOI: 10.1016/j.micpro.2007.01.004
In this paper, we present a distributed computing framework for problems characterized by a highly irregular search tree, whereby no reliable workload prediction is available. The framework is based on a peer-to-peer computing environment and dynamic load balancing. The system allows for dynamic resource aggregation, does not depend on any specific meta-computing middleware and is suitable for large-scale, multi-domain, heterogeneous environments, such as computational Grids. Dynamic load balancing policies based on global statistics are known to provide optimal load balancing performance, while randomized techniques provide high scalability. The proposed method combines both advantages and adopts distributed job-pools and a randomized polling technique. The framework has been successfully adopted in a parallel search algorithm for subgraph mining and evaluated on a molecular compounds dataset. The parallel application has shown good calability and close-to linear speedup in a distributed network of workstations.