criterion performance measurements

overview

want to understand this report?

100% dups/sort . nub 1000

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 8.709139516281723e-5 8.729125230127311e-5 8.751431291191765e-5
Standard deviation 6.191549929632812e-7 7.159766004562372e-7 8.589027844424205e-7

Outlying measurements have no (8.263888888888918e-3%) effect on estimated standard deviation.

100% dups/nub . sort 1000

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 3.263574015783523e-4 3.267099464089787e-4 3.2752596820612606e-4
Standard deviation 8.716668340132161e-7 1.697343770018259e-6 3.407946085219074e-6

Outlying measurements have slight (1.0637067869117743e-2%) effect on estimated standard deviation.

99% dups/sort . nub 1000

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 1.3705507189064083e-3 1.3726924776604778e-3 1.3745178825551702e-3
Standard deviation 5.191575924082461e-6 6.680036930299542e-6 8.85715910224543e-6

Outlying measurements have slight (1.514792899408277e-2%) effect on estimated standard deviation.

99% dups/nub . sort 1000

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 6.802636417650225e-4 6.809330684748058e-4 6.816657668295868e-4
Standard deviation 1.9003928784104976e-6 2.3649162388213397e-6 3.105098999311087e-6

Outlying measurements have slight (1.2656147271531887e-2%) effect on estimated standard deviation.

95% dups/sort . nub 1000

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 1.993573700632426e-3 1.9976638486346415e-3 2.003494900874798e-3
Standard deviation 1.3908732116922468e-5 1.6633438099974404e-5 2.1106067280788563e-5

Outlying measurements have slight (1.7236072637734686e-2%) effect on estimated standard deviation.

95% dups/nub . sort 1000

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 1.00600928935038e-3 1.007987820391971e-3 1.0104075527514996e-3
Standard deviation 5.883162938995957e-6 7.652647779656247e-6 1.155766071430393e-5

Outlying measurements have slight (1.388613370363026e-2%) effect on estimated standard deviation.

90% dups/sort . nub 1000

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 2.4893080949112355e-3 2.495645307301316e-3 2.5034699112563654e-3
Standard deviation 1.7355593284829975e-5 2.2299769202917542e-5 2.9924230828005707e-5

Outlying measurements have slight (1.851192595229619e-2%) effect on estimated standard deviation.

90% dups/nub . sort 1000

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 1.3655017603591798e-3 1.3678778930299265e-3 1.3727914266647133e-3
Standard deviation 6.991679686691142e-6 1.077612190540742e-5 1.924305310326381e-5

Outlying measurements have slight (1.5147928994082762e-2%) effect on estimated standard deviation.

80% dups/sort . nub 1000

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 3.1468249524360024e-3 3.1626973369541954e-3 3.184557404474011e-3
Standard deviation 4.777268347332688e-5 6.024101532493979e-5 7.28867826720394e-5

Outlying measurements have slight (7.577555536185404e-2%) effect on estimated standard deviation.

80% dups/nub . sort 1000

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 1.9406746515519466e-3 1.944258117180299e-3 1.9494064654815059e-3
Standard deviation 1.1225835747214432e-5 1.4383816950362008e-5 2.0197076566044144e-5

Outlying measurements have slight (1.694411414982164e-2%) effect on estimated standard deviation.

75% dups/sort . nub 1000

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 3.399403947868971e-3 3.40713043364518e-3 3.4142406534391203e-3
Standard deviation 2.1351947434558788e-5 2.5558035553542293e-5 3.236145419028371e-5

Outlying measurements have slight (2.039930555555542e-2%) effect on estimated standard deviation.

75% dups/nub . sort 1000

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 2.1671755169907687e-3 2.1728454346559532e-3 2.1806955526681945e-3
Standard deviation 1.867061554890569e-5 2.3496975869824836e-5 3.07629351163998e-5

Outlying measurements have slight (1.753826530612234e-2%) effect on estimated standard deviation.

70% dups/sort . nub 1000

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 3.643154262667756e-3 3.657026035925171e-3 3.671900341103555e-3
Standard deviation 3.890602283062321e-5 4.546136938645117e-5 5.527311104774123e-5

Outlying measurements have slight (2.1266540642721984e-2%) effect on estimated standard deviation.

70% dups/nub . sort 1000

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 2.373877198542995e-3 2.3789333147411227e-3 2.3838507816065163e-3
Standard deviation 1.3423157329520634e-5 1.6513805120842472e-5 2.0683859571818507e-5

Outlying measurements have slight (1.8175582990397805e-2%) effect on estimated standard deviation.

60% dups/sort . nub 1000

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 3.92876622221414e-3 3.948350416100217e-3 3.965031941491767e-3
Standard deviation 4.028329031708148e-5 5.53772325408171e-5 8.42144257288484e-5

Outlying measurements have slight (2.172839506172867e-2%) effect on estimated standard deviation.

60% dups/nub . sort 1000

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 2.804571926284837e-3 2.812220594300142e-3 2.819423502883415e-3
Standard deviation 2.1264469243237123e-5 2.5274756242123484e-5 3.09167919216234e-5

Outlying measurements have slight (1.9223375624759707e-2%) effect on estimated standard deviation.

50% dups/sort . nub 1000

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 4.5744068097482975e-3 4.593672220439813e-3 4.621053696647679e-3
Standard deviation 5.853029304975827e-5 7.340667275041105e-5 9.256448940588686e-5

Outlying measurements have slight (2.3242630385487555e-2%) effect on estimated standard deviation.

50% dups/nub . sort 1000

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 3.25054749444719e-3 3.261807403550279e-3 3.2732683903103047e-3
Standard deviation 2.929742630169291e-5 3.685184486259769e-5 4.8469500991866813e-5

Outlying measurements have slight (2.039930555555548e-2%) effect on estimated standard deviation.

40% dups/sort . nub 1000

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 4.995636169936953e-3 5.033094375689797e-3 5.089904633406109e-3
Standard deviation 1.0516204410242017e-4 1.3874987560804996e-4 2.0965893211065406e-4

Outlying measurements have moderate (0.11449337357524733%) effect on estimated standard deviation.

40% dups/nub . sort 1000

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 4.0041484175140675e-3 4.022406510063196e-3 4.044168458789761e-3
Standard deviation 4.9883363859874885e-5 6.289999724504343e-5 8.380443752898207e-5

Outlying measurements have slight (2.1728395061728568e-2%) effect on estimated standard deviation.

30% dups/sort . nub 1000

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 5.3117552210513825e-3 5.335187758340923e-3 5.3638959477850575e-3
Standard deviation 7.019360782240822e-5 8.280935610574041e-5 1.0418406980504766e-4

Outlying measurements have slight (2.4375000000000185e-2%) effect on estimated standard deviation.

30% dups/nub . sort 1000

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 4.588190735207057e-3 4.612134105505917e-3 4.632909674284537e-3
Standard deviation 5.566936461390218e-5 6.721211071414107e-5 8.085673413803723e-5

Outlying measurements have slight (2.324263038548764e-2%) effect on estimated standard deviation.

25% dups/sort . nub 1000

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 5.589697219721926e-3 5.612125092782699e-3 5.631035085581516e-3
Standard deviation 5.166328951069497e-5 6.360985470631922e-5 9.050896257493368e-5

Outlying measurements have slight (2.498356344510188e-2%) effect on estimated standard deviation.

25% dups/nub . sort 1000

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 4.803047841585502e-3 4.8181216421449595e-3 4.837416434096096e-3
Standard deviation 3.7905212994883074e-5 5.423952389139938e-5 7.477541404298587e-5

Outlying measurements have slight (2.3242630385487382e-2%) effect on estimated standard deviation.

20% dups/sort . nub 1000

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 5.675610454672006e-3 5.715542399953325e-3 5.756187947545737e-3
Standard deviation 1.0420237339004716e-4 1.2406854872505432e-4 1.4992214537078998e-4

Outlying measurements have slight (7.189200863962177e-2%) effect on estimated standard deviation.

20% dups/nub . sort 1000

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 5.015593606304137e-3 5.037534953023713e-3 5.069989141156842e-3
Standard deviation 6.0493280840393474e-5 7.928520709023255e-5 1.1763562757029197e-4

Outlying measurements have slight (2.3795359904818607e-2%) effect on estimated standard deviation.

10% dups/sort . nub 1000

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 6.1396399058308414e-3 6.191311439561789e-3 6.2685711497890454e-3
Standard deviation 1.3602329582507655e-4 1.844123706880661e-4 2.5523997585085925e-4

Outlying measurements have moderate (0.12355488224925985%) effect on estimated standard deviation.

10% dups/nub . sort 1000

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 5.772645274941049e-3 5.801073458931043e-3 5.839448900667828e-3
Standard deviation 7.878261893706728e-5 9.870981253725712e-5 1.224250609594203e-4

Outlying measurements have slight (2.4983563445101952e-2%) effect on estimated standard deviation.

5% dups/sort . nub 1000

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 6.559219365940528e-3 6.59591642472227e-3 6.623842805485694e-3
Standard deviation 6.344937169201397e-5 8.94225864284955e-5 1.2753620284150253e-4

Outlying measurements have slight (2.700617283950606e-2%) effect on estimated standard deviation.

5% dups/nub . sort 1000

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 5.956689120446498e-3 6.014319430605044e-3 6.108256416775385e-3
Standard deviation 1.3867984281991407e-4 2.1454638977711442e-4 3.2915895754668373e-4

Outlying measurements have moderate (0.14983905517610793%) effect on estimated standard deviation.

1% dups/sort . nub 1000

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 6.5756405903126544e-3 6.6054883708679315e-3 6.641706870405213e-3
Standard deviation 7.049949998263882e-5 9.595726092631596e-5 1.44383946371016e-4

Outlying measurements have slight (2.7006172839506293e-2%) effect on estimated standard deviation.

1% dups/nub . sort 1000

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 6.16338010301321e-3 6.188515540625973e-3 6.225016901723535e-3
Standard deviation 7.141527042532965e-5 9.356189002082396e-5 1.423280686541034e-4

Outlying measurements have slight (2.5623268698060857e-2%) effect on estimated standard deviation.

0% dups/sort . nub 1000

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 6.4801950030019305e-3 6.514590416431889e-3 6.5561556282223935e-3
Standard deviation 8.446987681995031e-5 1.1546760484648343e-4 1.5253300957243431e-4

Outlying measurements have slight (2.6296566837107162e-2%) effect on estimated standard deviation.

0% dups/nub . sort 1000

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 6.171925962116612e-3 6.208315484090736e-3 6.269910154603805e-3
Standard deviation 8.364973804137252e-5 1.3873332998967416e-4 2.1651249123221276e-4

Outlying measurements have slight (7.381523804758039e-2%) effect on estimated standard deviation.

understanding this report

In this report, each function benchmarked by criterion is assigned a section of its own. The charts in each section are active; if you hover your mouse over data points and annotations, you will see more details.

Under the charts is a small table. The first two rows are the results of a linear regression run on the measurements displayed in the right-hand chart.

We use a statistical technique called the bootstrap to provide confidence intervals on our estimates. The bootstrap-derived upper and lower bounds on estimates let you see how accurate we believe those estimates to be. (Hover the mouse over the table headers to see the confidence levels.)

A noisy benchmarking environment can cause some or many measurements to fall far from the mean. These outlying measurements can have a significant inflationary effect on the estimate of the standard deviation. We calculate and display an estimate of the extent to which the standard deviation has been inflated by outliers.