runtime: remove float64 multiplication in heap trigger compute path

As of the last CL, the heap trigger is computed as-needed. This means
that some of the niceties we assumed (that the float64 computations
don't matter because we're doing this rarely anyway) are no longer true.
While we're not exactly on a hot path right now, the trigger check still
happens often enough that it's a little too hot for comfort.

This change optimizes the computation by replacing the float64
multiplication with a shift and a constant integer multiplication.

I ran an allocation microbenchmark for an allocation size that would hit
this path often. CPU profiles seem to indicate this path was ~0.1% of
cycles (dwarfed by other costs, e.g. zeroing memory) even if all we're
doing is allocating, so the "optimization" here isn't particularly
important. However, since the code here is executed significantly more
frequently, and this change isn't particularly complicated, let's err
on the size of efficiency if we can help it.

Note that because of the way the constants are represented now, they're
ever so slightly different from before, so this change technically isn't
a total no-op. In practice however, it should be. These constants are
fuzzy and hand-picked anyway, so having them shift a little is unlikely
to make a significant change to the behavior of the GC.

For #48409.

Change-Id: Iabb2385920f7d891b25040226f35a3f31b7bf844
Reviewed-on: https://go-review.googlesource.com/c/go/+/397015
Run-TryBot: Michael Knyszek <mknyszek@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Michael Pratt <mpratt@google.com>

diff --git a/src/runtime/mgcpacer.go b/src/runtime/mgcpacer.go
index 44e45f2d09870732539c12609ea623c286a50604..ad3712595c7983f5b46f86f74255712f9ee41e0d 100644 (file)
--- a/src/runtime/mgcpacer.go
+++ b/src/runtime/mgcpacer.go
@@ -981,6 +981,27 @@ func (c *gcControllerState) heapGoalInternal() (goal, minTrigger uint64) {
        return goal, sweepDistTrigger
 }
 
+const (
+       // These constants determine the bounds on the GC trigger as a fraction
+       // of heap bytes allocated between the start of a GC (heapLive == heapMarked)
+       // and the end of a GC (heapLive == heapGoal).
+       //
+       // The constants are obscured in this way for efficiency. The denominator
+       // of the fraction is always a power-of-two for a quick division, so that
+       // the numerator is a single constant integer multiplication.
+       triggerRatioDen = 64
+
+       // The minimum trigger constant was chosen empirically: given a sufficiently
+       // fast/scalable allocator with 48 Ps that could drive the trigger ratio
+       // to <0.05, this constant causes applications to retain the same peak
+       // RSS compared to not having this allocator.
+       minTriggerRatioNum = 45 // ~0.7
+
+       // The maximum trigger constant is chosen somewhat arbitrarily, but the
+       // current constant has served us well over the years.
+       maxTriggerRatioNum = 61 // ~0.95
+)
+
 // trigger returns the current point at which a GC should trigger along with
 // the heap goal.
 //
@@ -1006,25 +1027,21 @@ func (c *gcControllerState) trigger() (uint64, uint64) {
        // increase in RSS. By capping us at a point >0, we're essentially
        // saying that we're OK using more CPU during the GC to prevent
        // this growth in RSS.
-       //
-       // The current constant was chosen empirically: given a sufficiently
-       // fast/scalable allocator with 48 Ps that could drive the trigger ratio
-       // to <0.05, this constant causes applications to retain the same peak
-       // RSS compared to not having this allocator.
-       if triggerBound := uint64(0.7*float64(goal-c.heapMarked)) + c.heapMarked; minTrigger < triggerBound {
-               minTrigger = triggerBound
+       triggerLowerBound := uint64(((goal-c.heapMarked)/triggerRatioDen)*minTriggerRatioNum) + c.heapMarked
+       if minTrigger < triggerLowerBound {
+               minTrigger = triggerLowerBound
        }
 
-       // For small heaps, set the max trigger point at 95% of the way from the
-       // live heap to the heap goal. This ensures we always have *some* headroom
-       // when the GC actually starts. For larger heaps, set the max trigger point
-       // at the goal, minus the minimum heap size.
+       // For small heaps, set the max trigger point at maxTriggerRatio of the way
+       // from the live heap to the heap goal. This ensures we always have *some*
+       // headroom when the GC actually starts. For larger heaps, set the max trigger
+       // point at the goal, minus the minimum heap size.
        //
        // This choice follows from the fact that the minimum heap size is chosen
        // to reflect the costs of a GC with no work to do. With a large heap but
        // very little scan work to perform, this gives us exactly as much runway
        // as we would need, in the worst case.
-       maxTrigger := uint64(0.95*float64(goal-c.heapMarked)) + c.heapMarked
+       maxTrigger := uint64(((goal-c.heapMarked)/triggerRatioDen)*maxTriggerRatioNum) + c.heapMarked
        if goal > defaultHeapMinimum && goal-defaultHeapMinimum > maxTrigger {
                maxTrigger = goal - defaultHeapMinimum
        }