criterion performance measurements

overview

want to understand this report?

100% dups/sort . nub 1000

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 5.1846673488369543e-5 5.1955115406381845e-5 5.2061557747401875e-5
Standard deviation 2.9991646178475403e-7 3.635531429170844e-7 4.5156132657263837e-7

Outlying measurements have no (7.633136094674724e-3%) effect on estimated standard deviation.

100% dups/nub . sort 1000

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 2.735847659301642e-4 2.741124947548288e-4 2.7485082556186213e-4
Standard deviation 1.5630867788484867e-6 2.05575317071832e-6 2.809239498459923e-6

Outlying measurements have slight (1.0308159722222137e-2%) effect on estimated standard deviation.

99% dups/sort . nub 1000

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 8.221053157340888e-4 8.229932032071981e-4 8.241047823357662e-4
Standard deviation 2.709666115196165e-6 3.3934741781392894e-6 4.948032906156776e-6

Outlying measurements have slight (1.3330898466033587e-2%) effect on estimated standard deviation.

99% dups/nub . sort 1000

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 5.483815404350628e-4 5.492934302618322e-4 5.511655277403761e-4
Standard deviation 2.576192772261684e-6 4.3826023710605345e-6 8.775043996370364e-6

Outlying measurements have slight (1.190303382203518e-2%) effect on estimated standard deviation.

95% dups/sort . nub 1000

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 1.265097962399076e-3 1.2666050328703552e-3 1.2712154905793547e-3
Standard deviation 3.2499741088292884e-6 8.004516947554153e-6 1.69030967128391e-5

Outlying measurements have slight (1.492194674012856e-2%) effect on estimated standard deviation.

95% dups/nub . sort 1000

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 8.076586785526663e-4 8.086150926952698e-4 8.097737258425246e-4
Standard deviation 2.776971546148492e-6 3.6257027519365476e-6 5.267778685133948e-6

Outlying measurements have slight (1.3155555555555556e-2%) effect on estimated standard deviation.

90% dups/sort . nub 1000

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 1.4682554683186561e-3 1.4700822797642436e-3 1.4723261466917836e-3
Standard deviation 5.77990536001612e-6 6.8783873394752736e-6 8.311105257778285e-6

Outlying measurements have slight (1.5621063240110858e-2%) effect on estimated standard deviation.

90% dups/nub . sort 1000

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 1.0333705097576409e-3 1.0351571692264105e-3 1.037046561454947e-3
Standard deviation 5.138623238063956e-6 6.27518557306157e-6 8.388027836503652e-6

Outlying measurements have slight (1.4081632653061225e-2%) effect on estimated standard deviation.

80% dups/sort . nub 1000

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 1.9544593511227446e-3 1.9595253810031866e-3 1.986685125954167e-3
Standard deviation 7.109095469721591e-6 2.983272952259414e-5 7.062377613972369e-5

Outlying measurements have slight (1.69441141498216e-2%) effect on estimated standard deviation.

80% dups/nub . sort 1000

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 1.4790030960061978e-3 1.480393513537747e-3 1.4819395021824098e-3
Standard deviation 4.459559095072825e-6 5.261108102306972e-6 6.973737405607548e-6

Outlying measurements have slight (1.562106324011086e-2%) effect on estimated standard deviation.

75% dups/sort . nub 1000

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 2.1063843706450197e-3 2.1083250051752386e-3 2.1110608138761484e-3
Standard deviation 5.696363254675608e-6 7.6060600368838986e-6 1.0427657823120928e-5

Outlying measurements have slight (1.7538265306122326e-2%) effect on estimated standard deviation.

75% dups/nub . sort 1000

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 1.6511121935169843e-3 1.65411403820713e-3 1.6570184445031228e-3
Standard deviation 8.625934653609475e-6 1.0570432209394402e-5 1.373408102989234e-5

Outlying measurements have slight (1.6124697661918742e-2%) effect on estimated standard deviation.

70% dups/sort . nub 1000

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 2.2438410798641803e-3 2.2460240746936264e-3 2.248264935319031e-3
Standard deviation 6.273409242398431e-6 7.419349457822353e-6 9.099219592297674e-6

Outlying measurements have slight (1.7851239669421204e-2%) effect on estimated standard deviation.

70% dups/nub . sort 1000

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 1.916391455640179e-3 1.9209731590028095e-3 1.936004191896445e-3
Standard deviation 9.113359011517176e-6 2.6450358886811063e-5 5.692768776700677e-5

Outlying measurements have slight (1.694411414982156e-2%) effect on estimated standard deviation.

60% dups/sort . nub 1000

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 2.6496771949495687e-3 2.654350724027507e-3 2.6593615679438008e-3
Standard deviation 1.2582704020098308e-5 1.5466980100738843e-5 1.9726492666175793e-5

Outlying measurements have slight (1.8860946745561893e-2%) effect on estimated standard deviation.

60% dups/nub . sort 1000

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 2.404851774771879e-3 2.4086140132885438e-3 2.4141733236255703e-3
Standard deviation 1.1276294217368505e-5 1.5594148431212945e-5 2.5129508969452253e-5

Outlying measurements have slight (1.8175582990397805e-2%) effect on estimated standard deviation.

50% dups/sort . nub 1000

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 2.8863710327238293e-3 2.8928746951275078e-3 2.9058772958997186e-3
Standard deviation 1.699685920877944e-5 2.8789145783205682e-5 5.586070589376644e-5

Outlying measurements have slight (1.9223375624759672e-2%) effect on estimated standard deviation.

50% dups/nub . sort 1000

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 2.659323261328539e-3 2.6625385686578377e-3 2.66594393437302e-3
Standard deviation 9.463685116360663e-6 1.0759834287911765e-5 1.2479412664345792e-5

Outlying measurements have slight (1.88609467455621e-2%) effect on estimated standard deviation.

40% dups/sort . nub 1000

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 3.1817050730097675e-3 3.1854407333431435e-3 3.1903494160690957e-3
Standard deviation 1.034678073701202e-5 1.4112316108874426e-5 2.2589230427616685e-5

Outlying measurements have slight (1.999167013744273e-2%) effect on estimated standard deviation.

40% dups/nub . sort 1000

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 3.0637214724737053e-3 3.0675555143490035e-3 3.0719003220668873e-3
Standard deviation 1.0939933526637702e-5 1.3306846078209808e-5 1.726837798554499e-5

Outlying measurements have slight (1.96e-2%) effect on estimated standard deviation.

30% dups/sort . nub 1000

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 3.457674540300125e-3 3.4608865313036554e-3 3.4653749279933677e-3
Standard deviation 8.985612512429599e-6 1.1796571781776954e-5 1.6926604468035688e-5

Outlying measurements have slight (2.082390221819815e-2%) effect on estimated standard deviation.

30% dups/nub . sort 1000

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 3.3838091246934064e-3 3.388678579219499e-3 3.394343382581759e-3
Standard deviation 1.425664590269864e-5 1.7061860599475802e-5 2.1000617200617626e-5

Outlying measurements have slight (2.039930555555545e-2%) effect on estimated standard deviation.

25% dups/sort . nub 1000

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 3.549652870446891e-3 3.5543325426216133e-3 3.5599672847065524e-3
Standard deviation 1.4546726061048817e-5 1.6934148030916654e-5 1.9738096968267915e-5

Outlying measurements have slight (2.082390221819828e-2%) effect on estimated standard deviation.

25% dups/nub . sort 1000

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 3.5906538095023225e-3 3.595388325778948e-3 3.600954460164912e-3
Standard deviation 1.3467713736532694e-5 1.655312342769315e-5 2.1830656822502982e-5

Outlying measurements have slight (2.082390221819821e-2%) effect on estimated standard deviation.

20% dups/sort . nub 1000

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 3.781480557741861e-3 3.7875125865391287e-3 3.794857417447447e-3
Standard deviation 1.7320335852762688e-5 2.130075322494026e-5 2.733279404292193e-5

Outlying measurements have slight (2.126654064272212e-2%) effect on estimated standard deviation.

20% dups/nub . sort 1000

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 3.838468399138971e-3 3.849484691029673e-3 3.889281972377442e-3
Standard deviation 9.941648543071004e-6 6.145465075381796e-5 1.287694427306962e-4

Outlying measurements have slight (2.1728395061728495e-2%) effect on estimated standard deviation.

10% dups/sort . nub 1000

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 4.005359222565637e-3 4.00933476115889e-3 4.0129778631927794e-3
Standard deviation 9.886946042525709e-6 1.2233188768643487e-5 1.5369839719679826e-5

Outlying measurements have slight (2.1728395061728394e-2%) effect on estimated standard deviation.

10% dups/nub . sort 1000

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 4.341311098785385e-3 4.347202324816069e-3 4.356769118538992e-3
Standard deviation 1.3633725972350999e-5 2.222168478250711e-5 4.236232346337532e-5

Outlying measurements have slight (2.271498107084893e-2%) effect on estimated standard deviation.

5% dups/sort . nub 1000

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 4.146066768347288e-3 4.155627971191964e-3 4.165566397896197e-3
Standard deviation 2.333558400115933e-5 2.9079549308113702e-5 3.837910171238348e-5

Outlying measurements have slight (2.221074380165287e-2%) effect on estimated standard deviation.

5% dups/nub . sort 1000

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 4.578229207676874e-3 4.585055320851664e-3 4.590635252015487e-3
Standard deviation 1.6285736114734832e-5 2.0195932478197822e-5 2.536945097063175e-5

Outlying measurements have slight (2.324263038548753e-2%) effect on estimated standard deviation.

1% dups/sort . nub 1000

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 4.277758566303519e-3 4.283669312447701e-3 4.294588333199768e-3
Standard deviation 1.4989398678068132e-5 2.569594450217295e-5 4.642911343402163e-5

Outlying measurements have slight (2.2210743801652895e-2%) effect on estimated standard deviation.

1% dups/nub . sort 1000

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 4.7353511544705506e-3 4.740556652365754e-3 4.7462434209791845e-3
Standard deviation 1.3909762874366092e-5 1.6896987906393625e-5 2.0988614633287635e-5

Outlying measurements have slight (2.3242630385487476e-2%) effect on estimated standard deviation.

0% dups/sort . nub 1000

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 4.2809165409568245e-3 4.287199949121727e-3 4.293040353936015e-3
Standard deviation 1.6908425317947705e-5 2.0016405346159845e-5 2.423586485475622e-5

Outlying measurements have slight (2.221074380165289e-2%) effect on estimated standard deviation.

0% dups/nub . sort 1000

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 4.771318508254847e-3 4.779051052224051e-3 4.786624511413614e-3
Standard deviation 1.985040139737209e-5 2.3982163982577597e-5 3.134066794356606e-5

Outlying measurements have slight (2.3242630385487476e-2%) effect on estimated standard deviation.

understanding this report

In this report, each function benchmarked by criterion is assigned a section of its own. The charts in each section are active; if you hover your mouse over data points and annotations, you will see more details.

Under the charts is a small table. The first two rows are the results of a linear regression run on the measurements displayed in the right-hand chart.

We use a statistical technique called the bootstrap to provide confidence intervals on our estimates. The bootstrap-derived upper and lower bounds on estimates let you see how accurate we believe those estimates to be. (Hover the mouse over the table headers to see the confidence levels.)

A noisy benchmarking environment can cause some or many measurements to fall far from the mean. These outlying measurements can have a significant inflationary effect on the estimate of the standard deviation. We calculate and display an estimate of the extent to which the standard deviation has been inflated by outliers.