Adaptive Cutoff Threshold (ACT)#
(hivenas.core.nas.act)
Calculates a cutoff performance threshold, below which a model stops training.
- class TerminateOnThreshold(monitor='val_sparse_categorical_accuracy', threshold_multiplier=0.25, diminishing_factor=0.25, n_classes=None)[source]#
Bases:
CallbackAdaptive Cutoff Threshold (ACT)
Keras Callback that terminates training if a given
val_sparse_categorical_accuracydynamic threshold is not reached after \(\epsilon\) epochs. The termination threshold has a logarithmic nature where the threshold increases by a decaying factor.- beta#
threshold coefficient (captures the leniency of the calculated threshold)
- Type
float
- monitor#
the optimizer metric type to monitor and calculate ACT on
- Type
str
- n_classes#
number of classes
- Type
int
- zeta#
diminishing factor; a positive, non-zero factor that controls how steeply the function horizontally asymptotes at \(y = 1.0\) (i.e 100% accuracy)
- Type
float
- get_threshold(epoch)[source]#
Calculates the termination threshold given the current epoch
\[ΔThreshold = ß(1 - \frac{1}{n})\]\[ \begin{align}\begin{aligned}Threshold_{base} = \frac{1}{n} + ΔThreshold &= \frac{1}{n} + ß(1 - \frac{1}{n}) \\\ &= \frac{(1 + ßn - ß)}{n}\end{aligned}\end{align} \]\(Threshold_{base} \Rightarrow (\frac{1}{n},\: 1) \;\) ; horizontal asymptote at \(\; Threshold_{base} = 1\)
\(ΔThreshold\) decays as the number of classes decreases.
To account for the expected increase in accuracy over the number of epochs \(ε\) , a growth factor \(g\) is added to the base threshold:
\[g = (1 - Threshold_{base}) - \frac{1}{\frac{1}{1-Threshold_{base}} + ζ(ε - 1)}\]\[Threshold_{adaptive} = Threshold_{base} + g\]\[g \Rightarrow [Threshold_{base}, 1) \; ; \text{horizontal asymptote at} \; g = 1\]- Parameters
epoch (int) – current epoch
- Returns
calculated cutoff threshold
- Return type
float