The two choices that I see which satisfy these criteria are:
- embed all data into the data section of the libraries.
I don't like this option so much, since it makes it harder to modify
the
libraries (you'd have to rebuild IMP each time). Users that get
prebuilt
binaries would never be able to inspect these data files.
I would hope they would never need to inspect them :-) If so,
something is poorly designed/documented. The bigger issue is that such
a plan is that binary data support is a pain (although I have run into
standard ways of doing this, especially for the common case of bitmap
images).
Plus, all of
the libraries would get read in to memory whether you need them or
not,
increasing the load time and the memory footprint.
Anything really large can easily go in a separate library. And I
wouldn't imagine that unused data in a library is ever read off disk
(that is what memory mapped io is for). But I haven't specifically
looked into its handling on our various platforms.
(and for those it may
make more sense to store them as static structs anyway).
This is just a mechanism for doing that which can be reused for
parsing user data.
- IMP is built with the path to the data stored internally and it
must
be installed before use.
That's the usual Unix way of doing things, so I think we should go for
that. Each module can have a data directory at the same level as the
existing src, include, pyext, etc. directories, and at install time
these files can be placed in /usr/share/imp/modulename/ or similar.
Again, it is pretty important for debugging that we support
installation into the build directory or somewhere which doesn't
involve copying headers. I don't think that involves too much messing
with the install scripts.
You'd still need it - or something equivalent - for the Python path,
and
on platforms other than Linux and Mac where rpath doesn't work.
Typically you install things somewhere where those things are already
set up (otherwise it is a useless installation). Plus, no one uses
anything other than those platforms, so not requiring it for them
would be a nice step forward.