La Vita è Bella

Tuesday, February 28, 2017

Comparing []byte's in Go

As I'm new to Go, I think it's quite understandable that I'm not familiar with the standard library yet, and reinvented the wheel sometimes.

One day I need to compare two []byte's. Since I'm not aware of any implementation in standard library, and this is simple enough, I wrote my own comparing function. It's something like:

func equal(b1, b2 []bytebool {
  if len(b1) != len(b2) {
    return false
  }
  for i, b := range b1 {
    if b != b2[i] {
      return false
    }
  }
  return true
}

A few days later, I saw my colleague used reflect.DeepEqual in our codebase. I thought "Oh that's cool! No need to reinvent the wheel!" So I removed my own implementation and used reflect.DeepEqual instead.

A few more days later, I accidentally discovered the existence of bytes.Equal. This is obviously faster than reflect.DeepEqual (because its feature is much simpler), so I'm going to replace them. But before that, I decided to do a simple benchmark test to see how much faster.

(In case you are not familiar with Go or Go's testing package, Go provided a very easy to use benchmark framework in it. Some might think the lack of features like Assume makes the testing code mouthful. I kind of agree with that, but I'm also mostly OK with that.)

The full benchmark test code can be found in this public gist. I'll just post the result here

BenchmarkEqual/reflect.DeepEqual-1024-4         	   20000	     91193 ns/op
BenchmarkEqual/equal-1024-4                     	 3000000	       544 ns/op
BenchmarkEqual/bytes.Equal-1024-4               	50000000	        22.6 ns/op
BenchmarkEqual/reflect.DeepEqual-1048576-4      	      20	  89556304 ns/op
BenchmarkEqual/equal-1048576-4                  	    3000	    536891 ns/op
BenchmarkEqual/bytes.Equal-1048576-4            	   30000	     44613 ns/op
BenchmarkEqual/reflect.DeepEqual-67108864-4     	       1	5801186044 ns/op
BenchmarkEqual/equal-67108864-4                 	      30	  37011544 ns/op
BenchmarkEqual/bytes.Equal-67108864-4           	     200	   8574768 ns/op
BenchmarkNonEqual/reflect.DeepEqual-1024-4      	 5000000	       280 ns/op
BenchmarkNonEqual/equal-1024-4                  	500000000	         3.46 ns/op
BenchmarkNonEqual/bytes.Equal-1024-4            	300000000	         4.56 ns/op
BenchmarkNonEqual/reflect.DeepEqual-1048576-4   	 5000000	       272 ns/op
BenchmarkNonEqual/equal-1048576-4               	500000000	         3.44 ns/op
BenchmarkNonEqual/bytes.Equal-1048576-4         	300000000	         4.52 ns/op
BenchmarkNonEqual/reflect.DeepEqual-67108864-4  	 5000000	       269 ns/op
BenchmarkNonEqual/equal-67108864-4              	500000000	         3.42 ns/op
BenchmarkNonEqual/bytes.Equal-67108864-4        	300000000	         4.50 ns/op

In short, bytes.Equal is usually 3 orders of magnitude faster than reflect.DeepEqual when the result is true, and 2 orders of magnitude faster when the result is false. (Also, my own implementation is only 1 order of magnitude slower than bytes.Equal when the result is true, and slightly faster when the result is false :)

And of course I replaced reflect.DeepEqual with bytes.Equal in our code base as appropriate.



tags: , , , ,

23:33:17 by fishy - dev - Permanent Link

Revision: 1.4/1.4, last modified on 2017-02-28 @ 23:49.

Karma: 5 (61.90% out of 21 were positive) [+/-]

You can subscribe to RSS 2.0 feed for comments and trackbacks

Trackbacks:
There are currently no trackbacks for this item.
Use this TrackBack url to ping this item (right-click, copy link target). If your blog does not support Trackbacks you can manually add your trackback by using this form.

No comments yet

Add Comment

 

May the Force be with you. RAmen