Repository navigation
Description
A Document is never freed by reference counting. Its parts and its package refer to each other, so the document and all of its lxml trees stay in memory until Python's cycle collector runs. In a long-running process like a web worker or a batch job, that looks like a memory leak. I think it explains part of #1364 and #1428, since the trees do get freed by gc.collect().
#1428 also measured RSS after gc.collect(). On glibc, freed memory usually stays with the allocator until malloc_trim(0), so the script below trims before each measurement.
Reproduce
import gc
import io
import sys
import weakref
import docx
def make_document():
document = docx.Document()
for i in range(3000):
document.add_paragraph(f"Paragraph {i}: " + "lorem ipsum dolor sit amet " * 8)
document.save(io.BytesIO())
return document
# The cycle: document part -> package -> package relationships -> document part
document = make_document()
part, package = document.part, document.part.package
print("part._package is package:", part._package is package)
print("package.rels targets the part:", any(r.target_part is part for r in package.rels.values()))
del document, part, package
# Dropping the last reference does not free the document; only the cycle collector does
gc.collect()
gc.disable()
package_ref = weakref.ref(make_document().part.package)
print("package alive after its last reference was dropped:", package_ref() is not None)
print("objects freed by gc.collect():", gc.collect())
print("package alive after gc.collect():", package_ref() is not None)
gc.enable()
if sys.platform == "linux": # how much memory those cycles hold (glibc)
import ctypes
libc = ctypes.CDLL("libc.so.6")
def rss_mib():
libc.malloc_trim(0) # leave out freed memory that glibc can return
with open("/proc/self/status") as status:
return next(int(line.split()[1]) for line in status if line.startswith("VmRSS:")) / 1024
gc.collect()
gc.disable()
print(f"RSS baseline: {rss_mib():.0f} MiB")
for _ in range(10):
make_document()
print(f"RSS after 10 documents, none referenced: {rss_mib():.0f} MiB")
gc.collect()
print(f"RSS after gc.collect(): {rss_mib():.0f} MiB")
gc.enable()Output with python-docx 1.2.0 and lxml 6.1.3 on CPython 3.14.10 (Linux, glibc 2.41):
part._package is package: True
package.rels targets the part: True
package alive after its last reference was dropped: True
objects freed by gc.collect(): 109
package alive after gc.collect(): False
RSS baseline: 28 MiB
RSS after 10 documents, none referenced: 106 MiB
RSS after gc.collect(): 28 MiB
On macOS with CPython 3.14.8 and 3.12.2, the first five lines match (3.12.2 frees 111 objects instead of 109). The RSS part only runs on Linux.
Why
Part.__init__ keeps self._package = package, and the package gets back to its parts through OpcPackage.rels → _Relationship._target. Parts also point at each other through Part.rels. So every part, along with the lxml tree it owns, is in a reference cycle. master has the same code.
The cycle collector is triggered by counts of tracked Python objects, not by memory use. A document leaves about 110 tracked objects behind, but its XML trees hold several MiB of native memory (about 8 MiB per document in the example above). Documents can pile up between collections, and a process that goes idle after building documents keeps them until some unrelated allocation triggers a collection.
Possible improvement
An opt-in way to release a document would help, for example a Document.close() (plus __enter__/__exit__) that breaks the part ↔ package references, so the XML trees are freed as soon as the caller is done with them. Weak back-references would remove the cycle entirely, but today a caller can hold on to a part without its Document, so that would change behavior.
In the meantime, a note in the docs saying that long-running processes should call gc.collect() after building or loading documents would save people from chasing an lxml leak.
If a Document.close() (or something like it) would be welcome, I can open a PR with tests.
Activity