Skip to content

cmp: count lines only when they are printed, and count them faster - #427

Open
KrushnXGIT wants to merge 1 commit into
uutils:mainfrom
KrushnXGIT:cmp-count-lines-only-when-needed
Open

KrushnXGIT wants to merge 1 commit into
uutils:mainfrom
KrushnXGIT:cmp-count-lines-only-when-needed

Conversation

@KrushnXGIT

Copy link
Copy Markdown

After each equal chunk, cmp counted the newlines in it one byte at a time, even with -s and -l, which never print a line number. That counting was most of cmp's CPU time.

  • Count lines only in the default mode, the only one that prints them.
  • Count them in 255-byte chunks summed in a u8, which the compiler vectorizes.

Two 300 MB files that differ at 250 MB, and a 40 MB text file (hyperfine, 10 runs, tmpfs):

GNU cmp 3.10 main this PR
cmp a b 87.8 ms 168.4 ms 78.4 ms
cmp -s a b 88.0 ms 168.5 ms 79.0 ms
cmp -l a b 88.1 ms 171.2 ms 81.8 ms
text, 2M lines 12.9 ms 26.6 ms 11.2 ms

Output, error messages and exit codes match main for -s, -l, -b, -n, -i, EOF and equal files. The GNU testsuite gives the same results before and after, and the cmp fuzzers ran clean for 90 s each. A new test checks the line number after 30,000 lines, for a difference and for EOF.

@codspeed

codspeed Bot commented Nov 20, 2026

Copy link
Copy Markdown
Contributor

Merging this PR will improve performance by ×5.1

3 warnings

⚠️ 24 benchmarks are dominated by system calls

System calls cannot be simulated consistently and are left out of the measure. The reported value understates the real cost. Switch to the Walltime instrument to measure them.

Learn more about system calls

See affected benchmarks

⚠️ 8 benchmarks with a significant change ran on different hardware

Their base and head results were measured on machines with a different CPU, core count or memory, so the changes might not come from the code. Run base and head on the same runner type.

Learn more about Macro Runners

See affected benchmarks

⚠️ 8 benchmarks with a significant change ran in a different software environment

Their base and head results were measured with a different OS or toolchain, such as compiler, runtime or library versions, so the changes might come from this difference rather than from the code. Open an affected benchmark to see what changed.

See affected benchmarks

⚡ 8 improved benchmarks
✅ 18 untouched benchmarks

Performance Changes

Benchmark BASE HEAD Efficiency
⚡ cmp_compare_files_different[25000] 11.7 ms 1 ms ×12
⚡ cmp_compare_files_different[10000] 4,704.3 µs 404.2 µs ×12
⚡ cmp_compare_files_different[1000] 506.7 µs 98.1 µs ×5.2
⚡ cmp_compare_files_equal[25000] 46.7 ms 10 ms ×4.7
⚡ cmp_compare_files_equal[10000] 18.7 ms 4 ms ×4.6
⚡ cmp_compare_files_equal[1000] 1,897.1 µs 428.8 µs ×4.4
⚡ cmp_compare_files_equal[100] 215.1 µs 68.4 µs ×3.1
⚡ cmp_compare_files_different[100] 76.1 µs 35.1 µs ×2.2

Tip

Curious why performance improved? Comment @codspeedbot explain why performance improved on this PR, or directly use the CodSpeed MCP with your agent.


Comparing KrushnXGIT:cmp-count-lines-only-when-needed (f0d5b99) with main (824dfcf)

Open in CodSpeed

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant