The other week I was optimizing some text processing code. The first thing I did was optimize the Perl regexps (this code was written in Perl), and eventually I decided to do a complete rewrite in C. And so a few hours later, the C code was ready to run, but it was so much slower than the Perl, and then I thought C is faster than Perl, but what is going wrong? My first thought was, 'Maybe I need assembly?' But then I thought that won't be very portable, and so I decided to use glibc's functions like strcmp and strchr to do things like find the next space. This made the C code considerably faster than the Perl.
Now, you may be wondering: how are glibc's string functions (like strchr or memchr) faster than a simple while (*ptr != '\0' && *ptr != ' ') { ...; ptr++; }?
The thing is your CPU's "wide" SIMD (Single Instruction Multiple Data) instructions (like SSE2 or AVX2) enable it to compare 16 or 32 bytes respectively with a single instruction. Think about it: comparing 16 or 32 bytes with a single instruction, is practically guaranteed to be faster.
Compare that to what our naive C loop does at the assembly level for every single byte:
1. Check if *ptr is '\0'
2. Check if *ptr is ' '
3. Increment the pointer
4. Jump back to the top of the while loop
Wash, rinse, and repeat--plus the CPU has to predict those branches (jumps) for every byte. Using glibc's hand-tuned (yes, really hand-tuned) assembly to do the scanning avoids all that overhead, while keeping the C code portable.
The results (measured in output speed, since a lot of input data gets discarded):
Perl: 4.6 MB/s
Naive C: 1.2 MB/s
C with glibc: 26.2 MB/s
Threat: Ask for permission before republishing full text copies of this blog, or I will hunt you down.
Copyright © 2026 ~cartwright