2 replies to this topic

[ZenLulz]

BenchShark is our new product allowing the developers to measure the speed of a set of functions in order to make sure they aren't and won't become a bottleneck for their .NET applications. The project initially started when we wanted to optimize the library MemorySharp. There is a lot of tools on the market, from open source to very expensive solutions. We wanted to focus on a lightweight product that allows to make automatic checking in order to avoid regressions. Also, benchmarking in .NET must follow several best practices to provide valuable results. They are notably defined by Eric Lippert, a person who contributed to the C# compiler at Microsoft.

 

 

Here is a very quick overview of BenchShark. Basically, your application contains a function that performs some operations and takes some time to be executed.

public static double Power(int x, int y) { return Math.Pow(x, y); }

The function can be benchmarked with a number of iterations to measure its performance.

var result = new BenchShark().EvaluateTask(() => Power(5, 5), 100);

Finally, to ensure that the function won't become a bottleneck, the result can be unit tested.

Assert.IsTrue(result.AverageExecutionTime < TimeSpan.FromSeconds(1));

I (ZenLulz) recently decided to start a blog to talk about C# optimization, tips & tricks. My first post will be dedicated to this library and write a summary of the best practices of the benchmarking using a managed language in general. A section in the forum was created for questions, feedbacks and ideas concerning BenchShark. Do not hesitate to post your message over there.

 

In addition, a new section was created in the forum. This new section is about extensions made for MemorySharp. If you built a nice piece of code that extends the library and you think it can be useful for other developers, you can advertise your development by creating your own thread there.

 

To conclude, as some of you already noticed, we changed the font of the website. From Helvetica Neue, we chose to use Open Sans that brings a sweet rendering and more pleasant to read.
We also wanted to thanks the members on the board to have submitted a lot of bugs in our libraries !

Click here to view the article

[dzmitry.lahoda]

Does library extracts IL of ()=> and generates custom loop in runtime? 

If not then next is true:

()=> is not free in C#.  Test in article adds overhead of ()=> 100 times. This overhead is big for small times, I think Math.Pow will be influenced.

 

Would be great to have lib with ()=> but without overhead of ()=>, e.g. via parsing IL in time and generating for loop with inlining content of ()=>.

 

You can look into example at https://github.com/a...izzo/NStopwatch to see what I mean. In the end of README there are links to several libs similar.

[ZenLulz]

Hi Dzmitry,
 
Thanks for your input.
 
The creation of the lambda expression (() => ...) is performed once when the function EvaluateTask is called, therefore this time is not measured by the benchmark. The time measured is the invocation of the lambda expression. Nevertheless, there is not a real difference between calling directly a function and invoking a lambda expression that embeds the said function.
 
Take a simplistic example. Let's compare the time between these two cases.

// Declare the stopwatch
var sw = new Stopwatch();

// Warm up
Power(5, 5);

// Perform the benchmark
sw.Start();
for (var i = 0; i < 1000000; i++)
{
   Power(5, 5);
}
sw.Stop();

Console.WriteLine("Direct invocation: {0} ticks; {1} ms ", sw.ElapsedTicks, sw.Elapsed.Milliseconds);

This code evaluates the total execution time for calling the function Power directly.

// Delcare the lambda expression
Action a = () => Power(5, 5);

// Warm up
a.Invoke();

// Perform the benchmark
sw.Reset();
sw.Start();
for (var i = 0; i < 1000000; i++)
{
   a.Invoke();
}
sw.Stop();

Console.WriteLine("Lambda invocation: {0} ticks; {1} ms ", sw.ElapsedTicks, sw.Elapsed.Milliseconds);

This code evaluates the total execution time by invoking the lambda expression that contains the function Power.
 
NB: The function Power is the function defined in the news.
 
The results are pretty much the same.
 

Direct invocation: 895223 ticks; 62 ms Lambda invocation: 907676 ticks; 63 ms


Direct invocation: 909727 ticks; 63 ms Lambda invocation: 948255 ticks; 66 ms

Direct invocation: 966336 ticks; 67 ms Lambda invocation: 926177 ticks; 64 ms

Direct invocation: 914914 ticks; 63 ms Lambda invocation: 912548 ticks; 63 ms

Direct invocation: 888370 ticks; 62 ms Lambda invocation: 907102 ticks; 63 ms

Direct invocation: 877695 ticks; 61 ms Lambda invocation: 904316 ticks; 63 ms

Direct invocation: 881524 ticks; 61 ms Lambda invocation: 902048 ticks; 63 ms

Direct invocation: 882329 ticks; 61 ms Lambda invocation: 927395 ticks; 64 ms

Direct invocation: 959154 ticks; 66 ms Lambda invocation: 905338 ticks; 63 ms

Direct invocation: 931322 ticks; 65 ms Lambda invocation: 903518 ticks; 63 ms

 
Let me know if I missed the point.
 
Cheers
ZenLulz