Categories See All →
1
Let BenchShark bite your applications !
Nov 15 2013 10:59 PM |
ZenLulz
in BenchShark
BenchShark is our new product allowing the developers to measure the speed of a set of functions in order to make sure they aren't and won't become a bottleneck for their .NET applications. The project initially started when we wanted to optimize the library MemorySharp. There is a lot of tools on the market, from open source to very expensive solutions. We wanted to focus on a lightweight product that allows to make automatic checking in order to avoid regressions. Also, benchmarking in .NET must follow several best practices to provide valuable results. They are notably defined by Eric Lippert, a person who contributed to the C# compiler at Microsoft.Here is a very quick overview of BenchShark. Basically, your application contains a function that performs some operations and takes some time to be executed.
public static double Power(int x, int y) { return Math.Pow(x, y); }
The function can be benchmarked with a number of iterations to measure its performance.
var result = new BenchShark().EvaluateTask(() => Power(5, 5), 100);
Finally, to ensure that the function won't become a bottleneck, the result can be unit tested.
Assert.IsTrue(result.AverageExecutionTime < TimeSpan.FromSeconds(1));
I (ZenLulz) recently decided to start a blog to talk about C# optimization, tips & tricks. My first post will be dedicated to this library and write a summary of the best practices of the benchmarking using a managed language in general. A section in the forum was created for questions, feedbacks and ideas concerning BenchShark. Do not hesitate to post your message over there.In addition, a new section was created in the forum. This new section is about extensions made for MemorySharp. If you built a nice piece of code that extends the library and you think it can be useful for other developers, you can advertise your development by creating your own thread there.
To conclude, as some of you already noticed, we changed the font of the website. From Helvetica Neue, we chose to use Open Sans that brings a sweet rendering and more pleasant to read.
We also wanted to thanks the members on the board to have submitted a lot of bugs in our libraries !
2 Comments
Does library extracts IL of ()=> and generates custom loop in runtime?
If not then next is true:
()=> is not free in C#. Test in article adds overhead of ()=> 100 times. This overhead is big for small times, I think Math.Pow will be influenced.
Would be great to have lib with ()=> but without overhead of ()=>, e.g. via parsing IL in time and generating for loop with inlining content of ()=>.
You can look into example at https://github.com/a...izzo/NStopwatch to see what I mean. In the end of README there are links to several libs similar.
Thanks for your input.
The creation of the lambda expression (() => ...) is performed once when the function EvaluateTask is called, therefore this time is not measured by the benchmark. The time measured is the invocation of the lambda expression. Nevertheless, there is not a real difference between calling directly a function and invoking a lambda expression that embeds the said function.
Take a simplistic example. Let's compare the time between these two cases.
// Declare the stopwatch var sw = new Stopwatch(); // Warm up Power(5, 5); // Perform the benchmark sw.Start(); for (var i = 0; i < 1000000; i++) { Power(5, 5); } sw.Stop(); Console.WriteLine("Direct invocation: {0} ticks; {1} ms ", sw.ElapsedTicks, sw.Elapsed.Milliseconds);This code evaluates the total execution time for calling the function Power directly.// Delcare the lambda expression Action a = () => Power(5, 5); // Warm up a.Invoke(); // Perform the benchmark sw.Reset(); sw.Start(); for (var i = 0; i < 1000000; i++) { a.Invoke(); } sw.Stop(); Console.WriteLine("Lambda invocation: {0} ticks; {1} ms ", sw.ElapsedTicks, sw.Elapsed.Milliseconds);This code evaluates the total execution time by invoking the lambda expression that contains the function Power.NB: The function Power is the function defined in the news.
The results are pretty much the same.
Let me know if I missed the point.
Cheers
ZenLulz