Repository navigation
MINOR: Check buffer bounds in BaseFixedWidthVector.copyFrom - #1505
Merged
Merged
Conversation
Arawoof06
requested review from
jbonofre,
laurentgo,
lidavidm and
wgtmac
as code owners
November 19, 2026 05:16
jbonofre
reviewed
Nov 19, 2026
jbonofre
left a comment
Member
There was a problem hiding this comment.
LGTM. Using ArrowBuf.setBytes checks the source buffer bounds, and setting the validity bit after the copy keeps a failed copy from marking the destination slot valid.
The regression test covers both copyFrom and copyFromSafe with a truncated source buffer.
jbonofre
approved these changes
Nov 19, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What's Changed
BaseFixedWidthVector.copyFromcopiedtypeWidthbytes with a rawMemoryUtil.copyMemory, so the source range was never checked against the source data buffer.loadFieldBuffersdoes not compare the data buffer length with the field node length, and full validation is off by default inArrowReader, so an IPC batch or dictionary that declares 16 Int values with a 4-byte data buffer loads without error.get(8)on that vector throwsIndexOutOfBoundsException, butcopyFrom/copyFromSafe(also reached throughTransferPair.copyValueSafe, e.g. fromDictionaryEncoder.decodeand the struct/list/unioncopyFrompaths) copies whatever native memory follows the buffer into the target vector, with bounds checking enabled.The copy now goes through
ArrowBuf.setBytes(index, src, srcIndex, length), which always checks the source range and checks the destination when bounds checking is on, matching the variable-widthcopyFrom, which already uses the checkedgetBytes. The validity bit is set after the copy, so a rejected copy leaves the target slot null.The new test in
TestCopyFromloads anIntVectorwith a short data buffer and expectsIndexOutOfBoundsExceptionfrom bothcopyFromandcopyFromSafe. Before the change, the same setup copied the0x41414141written just past the slice into the target. Thevectorandalgorithmtest suites pass.