string: memchr_inv() speed improvements
authorAkinobu Mita <akinobu.mita@gmail.com>
Fri, 23 Mar 2012 22:02:14 +0000 (15:02 -0700)
committerLinus Torvalds <torvalds@linux-foundation.org>
Fri, 23 Mar 2012 23:58:35 +0000 (16:58 -0700)
commitf43804bf5f9ae1e60077c5f22aee5fdfe4f09837
tree61e2dcaef0ab774d5cd68a89859f569d54fa7f78
parenta403d930c58eb8448f81fa90c125ac36dc8ef89d
string: memchr_inv() speed improvements

- Generate a 64-bit pattern more efficiently

memchr_inv needs to generate a 64-bit pattern filled with a target
character.  The operation can be done by more efficient way.

- Don't call the slow check_bytes() if the memory area is 64-bit aligned

memchr_inv compares contiguous 64-bit words with the 64-bit pattern as
much as possible.  The outside of the region is checked by check_bytes()
that scans for each byte.  Unfortunately, the first 64-bit word is
unexpectedly scanned by check_bytes() even if the memory area is aligned
to a 64-bit boundary.

Both changes were originally suggested by Eric Dumazet.

Signed-off-by: Akinobu Mita <akinobu.mita@gmail.com>
Suggested-by: Eric Dumazet <eric.dumazet@gmail.com>
Cc: Brian Norris <computersforpeace@gmail.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
lib/string.c