IMP.isd.AbstractGrid.AbstractGrid Class Reference

Grid main functionality: manage services. More...

Inherits Thread.

Public Member Functions
def	acquire_service
	For a given service, it finds available server and returns corresponding proxy worker. More...
def	add_server
	(for internal use) More...
def	copy_files
	(for internal use) More...
def	find_server
	(for internal use) More...
def	initialise
	Initialise the grid: Create temp paths if needed, either shared or remotely on each host Copy source files to this folder.
def	publish
	Publish instance on all the hosts of the grid (specific for the communication method) More...
def	release_service
	Used to inform server/proxy that we are done using the service, and it should be released, when the last funciton call of the proxy is finished. More...
def	set_loader
	sets the name of the compiled python file of the loader

Detailed Description

It contains list of servers, which provide services. When instance is published in the grid, corresponding servers object are created on each host and saved as list in the dictionary, with a key corresponding to their service_id.

Servers has part specific for communication method (at the creation stage), thus have to be defined in derived classes.

USAGE:

g = DerivedGrid(..)
g.copy_files(isd_src_path, ['*.pyc', '*.so'])

some_obj_instance = some_obj_class(..)

service_id = g.publish(some_obj_instance)

-> a) assigns some_service_id to this instance
   b) creates servers that manage the execution of this instance
      in remote hosts: servers[some_service_id] = [server1, server2, ...]

proxy = g.acquire_service(service_id) ## returns proxy object

proxy.set_parameters(p)  # these functions will be performed sequentally
proxy.f()                # and service won't be freed
proxy.g()                # unless .release_service() is called

g.release_service(proxy)

Note

This class is only available in Python.

Definition at line 33 of file AbstractGrid.py.

Member Function Documentation

def IMP.isd.AbstractGrid.AbstractGrid.acquire_service	(		self,
			service_id
	)

Definition at line 81 of file AbstractGrid.py.

def IMP.isd.AbstractGrid.AbstractGrid.add_server	(		self,
			server
	)

Registers the server within a grid

1) register itself (grid) in the server 2) proxy must have _selfrelease attribute to be able to release service (i.e. put the connected to it server in the .queues[service_id]), once the last function call before invoking .release_service() has finished calculations

self.queues: a dict[DerivedServer.service_id]=Queue(-1) containing the DerivedServers self.servers: dict[DerivedServer.service_id]=[list of DerivedServer instances]

Definition at line 68 of file AbstractGrid.py.

def IMP.isd.AbstractGrid.AbstractGrid.copy_files	(		self,
			rootdir,
			filelist,
			hosts = None,
			exclude = ['cns/',
			svn
	)

Copies source files to a common folder so that they were accessible to the grid on remote hosts

g = DerivedGrid(...)

src_path = os.path.join(os.environ['ISD_ROOT'],'/src/py')

from compileall import compile_dir compile_dir(src_path, maxlevels=0, force=True, quiet=True)

g.copy_files(src_path, ['*.pyc','*.so'])

The method will replicate the directory structure of items in the filelist, e.g. will create data folder in the destination for

g.copy_files(src_path, ['data/*.txt'])

also, it will copy all subfolders matching the required name, i.e. it will copy all files called foo.txt from all existing subfolders it can find

g.copy_files(src_path, ['Isd/foo.txt']) will copy files like 'Isd/foo.txt', 'Isd/a/foo.txt', 'Isd/a/b/c/foo.txt' etc.

the exclude keyword excludes paths containing the given strings.

Definition at line 122 of file AbstractGrid.py.

Here is the call graph for this function:

def IMP.isd.AbstractGrid.AbstractGrid.find_server	(		self,
			service_id
	)

Selects a server that is capable of executing the requested service. If they are all busy, waits until there will be one available.

Definition at line 76 of file AbstractGrid.py.

def IMP.isd.AbstractGrid.AbstractGrid.publish	(		self,
			instance
	)

Returns: service_id

Internally:

It creates list of servers performing the job specified by this service_id

Definition at line 52 of file AbstractGrid.py.

def IMP.isd.AbstractGrid.AbstractGrid.release_service	(		self,
			proxy
	)

However, it means that someone (the proxy or the grid) will have to call _release_service() itself and also restore ._selfrelease to the previos state (FALSE)

Definition at line 91 of file AbstractGrid.py.

Here is the call graph for this function:

---

The documentation for this class was generated from the following file:

• AbstractGrid.py