There is a strange behavior in regard to synaptic adaptation module. In fact, this module behaves strangely. It trains very fast, but then drops to some level (in loss) and then just does not do anything and is even worse than model without it. The idea might be that we are using too high learning rate for this or that the implementation may not be correct.
There is a strange behavior in regard to synaptic adaptation module. In fact, this module behaves strangely. It trains very fast, but then drops to some level (in loss) and then just does not do anything and is even worse than model without it. The idea might be that we are using too high learning rate for this or that the implementation may not be correct.