IMP
2.0.0
The Integrative Modeling Platform
|
Grid main functionality: manage services. More...
Inherits Thread.
Public Member Functions | |
def | acquire_service |
For a given service, it finds available server and returns corresponding proxy worker. More... | |
def | add_server |
(for internal use) More... | |
def | copy_files |
(for internal use) More... | |
def | find_server |
(for internal use) More... | |
def | initialise |
Initialise the grid: Create temp paths if needed, either shared or remotely on each host Copy source files to this folder. | |
def | publish |
Publish instance on all the hosts of the grid (specific for the communication method) More... | |
def | release_service |
Used to inform server/proxy that we are done using the service, and it should be released, when the last funciton call of the proxy is finished. More... | |
def | set_loader |
sets the name of the compiled python file of the loader | |
It contains list of servers, which provide services. When instance is published in the grid, corresponding servers object are created on each host and saved as list in the dictionary, with a key corresponding to their service_id.
Servers has part specific for communication method (at the creation stage), thus have to be defined in derived classes.
USAGE:
g = DerivedGrid(..) g.copy_files(isd_src_path, ['*.pyc', '*.so']) some_obj_instance = some_obj_class(..) service_id = g.publish(some_obj_instance) -> a) assigns some_service_id to this instance b) creates servers that manage the execution of this instance in remote hosts: servers[some_service_id] = [server1, server2, ...] proxy = g.acquire_service(service_id) ## returns proxy object proxy.set_parameters(p) # these functions will be performed sequentally proxy.f() # and service won't be freed proxy.g() # unless .release_service() is called g.release_service(proxy)
Definition at line 33 of file AbstractGrid.py.
def IMP.isd.AbstractGrid.AbstractGrid.acquire_service | ( | self, | |
service_id | |||
) |
Definition at line 81 of file AbstractGrid.py.
def IMP.isd.AbstractGrid.AbstractGrid.add_server | ( | self, | |
server | |||
) |
Registers the server within a grid
1) register itself (grid) in the server 2) proxy must have _selfrelease attribute to be able to release service (i.e. put the connected to it server in the .queues[service_id]), once the last function call before invoking .release_service() has finished calculations
self.queues: a dict[DerivedServer.service_id]=Queue(-1) containing the DerivedServers self.servers: dict[DerivedServer.service_id]=[list of DerivedServer instances]
Definition at line 68 of file AbstractGrid.py.
def IMP.isd.AbstractGrid.AbstractGrid.copy_files | ( | self, | |
rootdir, | |||
filelist, | |||
hosts = None , |
|||
exclude = ['cns/' , |
|||
svn | |||
) |
Copies source files to a common folder so that they were accessible to the grid on remote hosts
g = DerivedGrid(...)
src_path = os.path.join(os.environ['ISD_ROOT'],'/src/py')
from compileall import compile_dir compile_dir(src_path, maxlevels=0, force=True, quiet=True)
g.copy_files(src_path, ['*.pyc','*.so'])
The method will replicate the directory structure of items in the filelist, e.g. will create data folder in the destination for
g.copy_files(src_path, ['data/*.txt'])
also, it will copy all subfolders matching the required name, i.e. it will copy all files called foo.txt from all existing subfolders it can find
g.copy_files(src_path, ['Isd/foo.txt']) will copy files like 'Isd/foo.txt', 'Isd/a/foo.txt', 'Isd/a/b/c/foo.txt' etc.
the exclude keyword excludes paths containing the given strings.
Definition at line 122 of file AbstractGrid.py.
def IMP.isd.AbstractGrid.AbstractGrid.find_server | ( | self, | |
service_id | |||
) |
Selects a server that is capable of executing the requested service. If they are all busy, waits until there will be one available.
Definition at line 76 of file AbstractGrid.py.
def IMP.isd.AbstractGrid.AbstractGrid.publish | ( | self, | |
instance | |||
) |
Returns: service_id
Internally:
It creates list of servers performing the job specified by this service_id
Definition at line 52 of file AbstractGrid.py.
def IMP.isd.AbstractGrid.AbstractGrid.release_service | ( | self, | |
proxy | |||
) |
However, it means that someone (the proxy or the grid) will have to call _release_service() itself and also restore ._selfrelease to the previos state (FALSE)
Definition at line 91 of file AbstractGrid.py.