[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: score function weights

I think the three alternatives for weights are 
1) Hide it in the restraint class. To make this clean, we should hide the the ModelData* and mirror its functions in the Restraint base class (strictly speaking we only need to mirror add_to_deriv, but for symmetry, we should do all if we do one). For the return value, I would suggest adding a function weighted_eval which is called by the Model or RestraintSet instead of eval and calls eval internally. Then, anyone who wants to override that directly will know they need to deal with the weights themselves.  Note that this proposal breaks if we move to a "Particles as Objects" framework since adding to the deriv involves an arbitrary function call. 

2) Keep it external to the restraint. To do this we would give each restraint a separate ModelData which has a weight in it and scales add_to_deriv calls accordingly. The external user would be responsible to for weighting the output. I don't like this as much as 1.

3) Store the weight in the Restraint base and hope the Restraints scale everything themselves. This is pretty easy to write a test that checks if a restraint is doing the right thing and such a test could be put in the model and run on each added restraint if debugging checks are enabled. This is the only approach of the three that will work if we go to particles as objects. We can provide macros that make the code a bit simpler too. 

If we don't change to the new particle design, I think the first is the best. If we do, we are stuck with the third (and had better write a bunch of macros and test code). 

As for changing the distance on a distance restraint, it seems like you should just add code to set the constant in the appropriate restraint class. Seems like a useful thing to have. 

On Nov 5, 2007, at 1:36 PM, Friedrich Foerster wrote:

hi all,

i think we urgently need a solution for integrating different weights of terms into the scoring function to get real applications going.

Schedules in optimization

Old fashioned - but working - optimization strategies in modeling often use variable target functions. That means the scoring function is gradually changed in the course of optimization. In MODELLER, Andrej probably spent significant time of his thesis figuring out a reasonable way of varying the different terms of the scoring function to eventually obtain the best (=lowest scoring) results. Although the eventual relative weights of all terms is 1, finding the global minimum can be greatly facilitated by 'guiding' the optimization by removing high barriers that make the optimization stuck in early stages of the optimization.

In particular, it is beneficial to vary the term penalizing steric clashes S_vol (soft-sphere overlap) during optimization, ie have it practically zero in the beginning of the optimization to allow the particles to rearrange drastically and then gradually increase the weight for this term. In MODELLER a 'schedule' was used for this purpose. A schedule was a list of relative weights of the relative weights of the scoring terms.

IMP currently does not have any way of easily adjusting the different weights of the terms constituting the score. The only way of changing the restraints is to generate new ones, eg for volume exclusion one would need to regenerate these restraints each time specifying a different standard deviation every time, which is awkward in my eyes.

I would be eager to have some kind of schedule in IMP, ie an easy way to alter the relative contribution of terms to the score. Specifically, one could envision the following things useful during an optimization run for the volume exclusion restraint:

  • scale S_vol differently at different stages of optimization
  • vary radii (=minimum distance in restraint) at different stages of optimization

Technically, the weight could be either the same for a given kind of restraint, or possibly even a property of a restraint. In the latter case, specific atom pairs could be penalized less than others for clashes, which might make sense if the accuracy of the representation (eg a rigid body) is lower/higher for specific particles.

I do not favor any particular way of integrating a weight. One option - and currently a parameter in the EM score - is to call the restraint with a given weight. The term (and derivative) will be added to the overall functions using this weight.

Any ideas/suggestions/preferences?


also found at:

% Friedrich Foerster                                            %
% Andrej Sali Lab                                               %
% University of California at San Francisco                     %
# MC 2552                                                       %
% Byers Hall Room 501                                           %
% 1700 4th Street                                               %
% San Francisco, CA 94158-2330, USA                             %
%                                                               %
% phone:  +1 (415) 514-4258                                     %
% fax:    +1 (415) 514-4231                                     %
%                                                               %
%   www.salilab.org/~frido                                      %
%                                                               %