Skip to content

diff: count -t columns like a terminal after CR, backspace and controls - #384

Merged
sylvestre merged 7 commits into
uutils:mainfrom
binhvq:expand-tabs-control-chars
Nov 19, 2026
Merged

sylvestre merged 7 commits into
uutils:mainfrom
binhvq:expand-tabs-control-chars

Conversation

@binhvq

@binhvq binhvq commented Nov 11, 2026

Copy link
Copy Markdown
Contributor

Fixes #354.

do_expand_tabs measured the Unicode width of everything between tabs, so control characters threw the padding off. It now tracks the column like GNU diff 3.12 does with -t:

  • \r goes back to column 0. In the normal and context formats the line marker (< , ! , ...) is written again after it when more text follows; unified and ed don't repeat theirs, same as GNU.
  • \b goes one column back. At column 0 it is dropped, which is also what GNU prints.
  • other control characters, C1 included, take no column
  • invalid UTF-8 bytes take one column each, instead of the whole chunk being counted by bytes when one byte is invalid

Text runs between control characters are still measured with UnicodeWidthStr, so the ZWJ emoji test keeps its width. do_write_line takes the marker as a new argument.

Checked against GNU diff in en_US.UTF-8 for the normal, unified, context and ed formats. In the C locale GNU counts every non-ASCII byte as one column; that was already different here and isn't touched.

@github-actions

Copy link
Copy Markdown

GNU diffutils testsuite comparison:

Test results comparison:
  Current:   TOTAL: 33 / PASSED: 20 / FAILED: 9 / SKIPPED: 4
  Reference: TOTAL: 33 / PASSED: 19 / FAILED: 10 / SKIPPED: 4

Changes from main branch:
  TOTAL: +0
  PASSED: +1
  FAILED: -1

Test improvements (1):
  + expand-tabs

Comment thread src/utils.rs
let tab = b'\t';
let ntabs = line.iter().filter(|c| **c == tab).count();
if ntabs == 0 {
///

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

could you please move #[must_use] below the doc comment? right now it splits it in two

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done, it now sits right above the fn.

Comment thread src/context_diff.rs Outdated
DiffLine::Change(e) => {
write!(output, "! ").expect("write to Vec is infallible");
do_write_line(&mut output, &e, params.expand_tabs, params.tabsize)
do_write_line(&mut output, &e, params.expand_tabs, params.tabsize, b"! ")

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the marker is now written twice at every call site, could do_write_line write it itself?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done, do_write_line now writes the marker itself and the call sites no longer do.

…hars

# Conflicts:
#	src/context_diff.rs
#	src/normal_diff.rs
#	src/unified_diff.rs
@sylvestre
sylvestre merged commit 0b3a9a3 into uutils:main Nov 19, 2026
31 checks passed
@sylvestre

Copy link
Copy Markdown
Collaborator

Thanks for your PR

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

diff - -t tab expansion miscounts columns after CR, backspace and control bytes

2 participants