Re: [IMP-dev] change of optimizer performance from v7018 to v9277

[Daniel Russel <>]

It appears that Frido's tests were passing (and probably still are), so they don't seem to have been sensitive to the magnitude of the derivative being off.

On May 18, 2011, at 10:44 AM, Keren Lasker wrote:

> i think the test is broken.
> i worked on that a while back with daniel (ie daniel's comment on the magnitude) but i did not manage to resolve the problem.
> i will look at that again, clearly it is very important.
> will report back to the list once resolved.
> keren.
> On May 18, 2011, at 10:39 AM, Daniel Russel wrote:
> 
>> On May 18, 2011, at 9:05 AM, Friedrich Foerster wrote:
>> 
>>> I remember painfully writing a test case for the em restraint and the derivatives when I implemented the em score. Is that test not functional anymore?
>> They are still there, but I'm not sure if they pass. It also isn't clear to me what exactly they test as they use some modeller functionality that I am not familiar with to do the actual checking. (Reason I'm not sure about their status: I broke the build last night on all platforms except windows, and for some reason the windows test run didn't terminate. And the em tests have been failing in general for rather a while, and it is not something I've looked in to. And I don't have modeller installed on any of my machines, so that is not a test I run myself).
>> 
>> Keren, do you have more sense of the status of those tests and, if they are broken, when they broke?
>> 
>> Anyway, there is a test I added a bit back to test that the magnitudes of the derivatives make sense, since they didn't (eg that they roughly match the number one gets for a numeric approximation of the derivative). So there are definitely problems that need to be fixed :-)
>> 
>>> 
>>> Cheers
>>> 
>>> Friedrich
>>> 
>>> Dr. Friedrich Foerster
>>> Max-Planck Institute of Biochemistry
>>> Am Klopferspitz 18
>>> D-81252 Martinsried
>>> 
>>> www.tomotronic.org
>>> 
>>> 
>>> On 18.05.2011, at 16:55, Keren Lasker <> wrote:
>>> 
>>>> Major changes to the code from my part were in verion 7212 in class CoarseCC, which mainly dealt with rigid bodies support.
>>>> I am not sure if that is the reason for Pia's problem, but as she needs to use EM based optimization relatively soon, i will take a look again.
>>>> In any case - Pia - it would be useful if you can write a test case that I can check things against.
>>>> keren.
>>>> On May 18, 2011, at 7:43 AM, Daniel Russel wrote:
>>>> 
>>>>> I suspect it is an issue with em::FitRestraint as the derivatives returned by that restraint are currently broken (the direction is fine, but the magnitude off by quite a bit). As a result derivative-using optimizers don't perform well with it.
>>>>> 
>>>>> Keren, Ben, is there a plan for fixing it?
>>>>>        --Daniel
>>>>> 
>>>>> On May 18, 2011, at 6:31 AM, Pia Unverdorben wrote:
>>>>> 
>>>>>> Dear all,
>>>>>> 
>>>>>> is it possible, that something fundamental changed in the optimizers, steepest descent as well as conjugate gradients? 
>>>>>> I try to fit a protein to an em density, simulated from another conformational state using the em restraint and plenty of harmonic restraints. With an older version (revision 7018) it moves towards the density, whereas with the revision 9277 the score and the positions barely change, irrespective of the number of steps, threshold, stepsize and different weights of restraints.
>>>>>> More importantly, i did compare the optimization results of v 7018 and 9277 of conjugate gradient optimization and steepest descent from identical starting positions with identical restraints and optimizer settings. the initial scores are identical, but then the problem starts:
>>>>>> For example, with v7018 my model improves from a score of 1835 to 1101 after 100 steps of steepest descent optimization. However, with v9277 my score increases (!) to 1836. behavior of conjugate gradients similarly deteriorates for v9277, albeit i did not observe increases of scores.  nevertheless, improvements in score were disappointingly low.
>>>>>> Does someone know, why the optimizers behave so differently, for my purposes worse?
>>>>>> 
>>>>>> Thanks a lot!
>>>>>> 
>>>>>> Regards,
>>>>>> 
>>>>>> Pia 
>>>>>> _______________________________________________
>>>>>> IMP-dev mailing list
>>>>>> 
>>>>>> https://salilab.org/mailman/listinfo/imp-dev
>>>>> 
>>>> 
>>>> 
>>>> _______________________________________________
>>>> IMP-dev mailing list
>>>> 
>>>> https://salilab.org/mailman/listinfo/imp-dev
>>>> 
>> 
>> 
>> _______________________________________________
>> IMP-dev mailing list
>> 
>> https://salilab.org/mailman/listinfo/imp-dev
> 
> 
> _______________________________________________
> IMP-dev mailing list
> 
> https://salilab.org/mailman/listinfo/imp-dev