Crashed in a multi-threaded environment

[wciq1208]

pypdfium2==4.30.0
marker-pdf==0.2.13

I call the code as follows:

    self.model = load_all_models()
# other code
            with self._chat_lock:
                full_text, images, out_meta = convert_single_pdf(pdf_file.name, self.model, max_pages=max_pages, langs=langs, batch_multiplier=batch_multiplier, start_page=start_page)

An error occurred when triggering Python's GC:

#0  __pthread_kill_implementation (no_tid=0, signo=6, threadid=46945239307840) at ./nptl/pthread_kill.c:44
#1  __pthread_kill_internal (signo=6, threadid=46945239307840) at ./nptl/pthread_kill.c:78
#2  __GI___pthread_kill (threadid=46945239307840, signo=signo@entry=6) at ./nptl/pthread_kill.c:89
#3  0x00002aaaaac31476 in __GI_raise (sig=sig@entry=6) at ../sysdeps/posix/raise.c:26
#4  0x00002aaaaac177f3 in __GI_abort () at ./stdlib/abort.c:79
#5  0x00002aaaaac78676 in __libc_message (action=action@entry=do_abort, fmt=fmt@entry=0x2aaaaadcab77 "%s\n") at ../sysdeps/posix/libc_fatal.c:155
#6  0x00002aaaaac8fcfc in malloc_printerr (str=str@entry=0x2aaaaadc870e "corrupted double-linked list") at ./malloc/malloc.c:5664
#7  0x00002aaaaac907cc in unlink_chunk (p=<optimized out>, av=0x2ab754000030) at ./malloc/malloc.c:1635
#8  0x00002aaaaac90969 in malloc_consolidate (av=av@entry=0x2ab754000030) at ./malloc/malloc.c:4780
#9  0x00002aaaaac91ea0 in _int_free (av=0x2ab754000030, p=0x2ab7546bce60, have_lock=<optimized out>) at ./malloc/malloc.c:4674
#10 0x00002aaaaac94453 in __GI___libc_free (mem=<optimized out>) at ./malloc/malloc.c:3391
#11 0x00002aab5f946488 in std::__Cr::deque<std::__Cr::unique_ptr<CPDF_ObjectWalker::SubobjectIterator, std::__Cr::default_delete<CPDF_ObjectWalker::SubobjectIterator> >, std::__Cr::allocator<std::__Cr::unique_ptr<CPDF_ObjectWalker::SubobjectIterator, 
`
std::__Cr::default_delete<CPDF_ObjectWalker::SubobjectIterator> > > >::~deque() () from /opt/conda/lib/python3.10/site-packages/pypdfium2_raw/libpdfium.so
#12 0x00002aab5f946236 in CPDF_PageObjectHolder::~CPDF_PageObjectHolder() () from /opt/conda/lib/python3.10/site-packages/pypdfium2_raw/libpdfium.so
#13 0x00002aab5f942d1e in CPDF_Page::~CPDF_Page() () from /opt/conda/lib/python3.10/site-packages/pypdfium2_raw/libpdfium.so
`
#14 0x00002aaaab31d052 in ffi_call_unix64 () from /opt/conda/lib/python3.10/lib-dynload/../../libffi.so.8
#15 0x00002aaaab31b925 in ffi_call_int () from /opt/conda/lib/python3.10/lib-dynload/../../libffi.so.8
#16 0x00002aaaab31c06e in ffi_call () from /opt/conda/lib/python3.10/lib-dynload/../../libffi.so.8
#17 0x00002aaaab2fc1e7 in _call_function_pointer (argtypecount=<optimized out>, argcount=1, resmem=0x2ab24a4feb90, restype=<optimized out>, atypes=<optimized out>, avalues=<optimized out>, pProc=0x2aab5fa2c3b0 <FPDF_ClosePage>, flags=4353)
    at /usr/local/src/conda/python-3.10.14/Modules/_ctypes/callproc.c:916
#18 _ctypes_callproc (pProc=0x2aab5fa2c3b0 <FPDF_ClosePage>, argtuple=0x2ab24b34f340, flags=4353, argtypes=<optimized out>, restype=0x747720 <_Py_NoneStruct>, checker=0x0) at /usr/local/src/conda/python-3.10.14/Modules/_ctypes/callproc.c:1262
#19 0x00002aaaab30523e in PyCFuncPtr_call (self=<optimized out>, inargs=<optimized out>, kwds=0x0) at /usr/local/src/conda/python-3.10.14/Modules/_ctypes/_ctypes.c:4221
#20 0x00000000004f705b in _PyObject_MakeTpCall (tstate=0x2ab73c0608c0, callable=0x2aab5f6213c0, args=<optimized out>, nargs=<optimized out>, keywords=0x0) at /usr/local/src/conda/python-3.10.14/Objects/call.c:215
#21 0x00000000004f3106 in _PyObject_VectorcallTstate (kwnames=0x0, nargsf=<optimized out>, args=0x2aaf5e17fb20, callable=0x2aab5f6213c0, tstate=<optimized out>) at /usr/local/src/conda/python-3.10.14/Include/cpython/abstract.h:112
#22 _PyObject_VectorcallTstate (kwnames=0x0, nargsf=<optimized out>, args=0x2aaf5e17fb20, callable=0x2aab5f6213c0, tstate=<optimized out>) at /usr/local/src/conda/python-3.10.14/Include/cpython/abstract.h:99
#23 PyObject_Vectorcall (kwnames=0x0, nargsf=<optimized out>, args=0x2aaf5e17fb20, callable=0x2aab5f6213c0) at /usr/local/src/conda/python-3.10.14/Include/cpython/abstract.h:123
#24 call_function (kwnames=0x0, oparg=<optimized out>, pp_stack=<synthetic pointer>, trace_info=0x2ab24a4fee80, tstate=<optimized out>) at /usr/local/src/conda/python-3.10.14/Python/ceval.c:5893
#25 _PyEval_EvalFrameDefault (tstate=<optimized out>, f=0x2aaf5e17f9a0, throwflag=<optimized out>) at /usr/local/src/conda/python-3.10.14/Python/ceval.c:4181
#26 0x00000000004fdd4f in _PyEval_EvalFrame (throwflag=0, f=0x2aaf5e17f9a0, tstate=0x2ab73c0608c0) at /usr/local/src/conda/python-3.10.14/Include/internal/pycore_ceval.h:46
#27 _PyEval_Vector (kwnames=<optimized out>, argcount=<optimized out>, args=<optimized out>, locals=0x0, con=0x2aab5fbb2720, tstate=0x2ab73c0608c0) at /usr/local/src/conda/python-3.10.14/Python/ceval.c:5067
#28 _PyFunction_Vectorcall (func=0x2aab5fbb2710, stack=<optimized out>, nargsf=<optimized out>, kwnames=<optimized out>) at /usr/local/src/conda/python-3.10.14/Objects/call.c:342
#29 0x00000000004f08a9 in do_call_core (kwdict=0x2ab50b9f0640, callargs=0x2ab24b6895c0, func=0x2aab5fbb2710, trace_info=0x2ab24a4ff040, tstate=<optimized out>) at /usr/local/src/conda/python-3.10.14/Python/ceval.c:5945
#30 _PyEval_EvalFrameDefault (tstate=<optimized out>, f=0x2aaf52726440, throwflag=<optimized out>) at /usr/local/src/conda/python-3.10.14/Python/ceval.c:4277
#31 0x00000000004fdd4f in _PyEval_EvalFrame (throwflag=0, f=0x2aaf52726440, tstate=0x2ab73c0608c0) at /usr/local/src/conda/python-3.10.14/Include/internal/pycore_ceval.h:46
#32 _PyEval_Vector (kwnames=<optimized out>, argcount=<optimized out>, args=<optimized out>, locals=0x0, con=0x2aab5f630dd0, tstate=0x2ab73c0608c0) at /usr/local/src/conda/python-3.10.14/Python/ceval.c:5067
#33 _PyFunction_Vectorcall (func=0x2aab5f630dc0, stack=<optimized out>, nargsf=<optimized out>, kwnames=<optimized out>) at /usr/local/src/conda/python-3.10.14/Objects/call.c:342
#34 0x00000000004f08a9 in do_call_core (kwdict=0x2ab50b9f0a00, callargs=0x2ab24a9475e0, func=0x2aab5f630dc0, trace_info=0x2ab24a4ff200, tstate=<optimized out>) at /usr/local/src/conda/python-3.10.14/Python/ceval.c:5945
#35 _PyEval_EvalFrameDefault (tstate=<optimized out>, f=0x2aabc3a928c0, throwflag=<optimized out>) at /usr/local/src/conda/python-3.10.14/Python/ceval.c:4277
#36 0x00000000004f63ad in _PyEval_EvalFrame (throwflag=0, f=0x2aabc3a928c0, tstate=0x2ab73c0608c0) at /usr/local/src/conda/python-3.10.14/Include/internal/pycore_ceval.h:46
#37 _PyEval_Vector (kwnames=0x0, argcount=<optimized out>, args=<optimized out>, locals=0x0, con=0x2aaaab149d90, tstate=0x2ab73c0608c0) at /usr/local/src/conda/python-3.10.14/Python/ceval.c:5067
#38 _PyFunction_Vectorcall (kwnames=0x0, nargsf=<optimized out>, stack=<optimized out>, func=0x2aaaab149d80) at /usr/local/src/conda/python-3.10.14/Objects/call.c:342
#39 _PyObject_FastCallDictTstate (tstate=0x2ab73c0608c0, callable=0x2aaaab149d80, args=<optimized out>, nargsf=<optimized out>, kwargs=<optimized out>) at /usr/local/src/conda/python-3.10.14/Objects/call.c:142
#40 0x0000000000507b36 in _PyObject_Call_Prepend (tstate=0x2ab73c0608c0, callable=0x2aaaab149d80, obj=0x2ab48fe2ca80, args=<optimized out>, kwargs=0x0) at /usr/local/src/conda/python-3.10.14/Objects/call.c:431
#41 0x00000000005cf913 in slot_tp_call (self=0x2ab48fe2ca80, args=0x2ab24b44d6c0, kwds=0x0) at /usr/local/src/conda/python-3.10.14/Objects/typeobject.c:7494
#42 0x00000000004f705b in _PyObject_MakeTpCall (tstate=0x2ab73c0608c0, callable=0x2ab48fe2ca80, args=<optimized out>, nargs=<optimized out>, keywords=0x0) at /usr/local/src/conda/python-3.10.14/Objects/call.c:215
#43 0x000000000059860a in _PyObject_VectorcallTstate (kwnames=0x0, nargsf=9223372036854775809, args=0x2ab24a4ff458, callable=0x2ab48fe2ca80, tstate=0x2ab73c0608c0) at /usr/local/src/conda/python-3.10.14/Include/cpython/abstract.h:112
#44 PyObject_CallOneArg (func=0x2ab48fe2ca80, arg=<optimized out>) at /usr/local/src/conda/python-3.10.14/Include/cpython/abstract.h:184
#45 0x00000000004e19c9 in handle_weakrefs (old=0x75b6d0, unreachable=0x2ab24a4ff520) at /usr/local/src/conda/python-3.10.14/Modules/gcmodule.c:887
#46 gc_collect_main (tstate=0x2ab73c0608c0, generation=2, n_collected=0x2ab24a4ff600, n_uncollectable=0x2ab24a4ff5f8, nofail=0) at /usr/local/src/conda/python-3.10.14/Modules/gcmodule.c:1281
#47 0x000000000059168c in gc_collect_with_callback (tstate=tstate@entry=0x2ab73c0608c0, generation=2) at /usr/local/src/conda/python-3.10.14/Modules/gcmodule.c:1413
#48 0x00000000004d789a in gc_collect_generations (tstate=0x2ab73c0608c0) at /usr/local/src/conda/python-3.10.14/Modules/gcmodule.c:1468
#49 _PyObject_GC_Alloc (basicsize=<optimized out>, use_calloc=0) at /usr/local/src/conda/python-3.10.14/Modules/gcmodule.c:2297
#50 _PyObject_GC_Malloc (basicsize=<optimized out>) at /usr/local/src/conda/python-3.10.14/Modules/gcmodule.c:2307
#51 _PyObject_GC_New (tp=0x749da0 <PyDict_Type>) at /usr/local/src/conda/python-3.10.14/Modules/gcmodule.c:2319
#52 0x00000000004d8d5a in new_dict (values=0x759048 <empty_values>, keys=0x749d60 <empty_keys_struct>) at /usr/local/src/conda/python-3.10.14/Objects/dictobject.c:663
#53 PyDict_New () at /usr/local/src/conda/python-3.10.14/Objects/dictobject.c:745
#54 0x00002aab129d56e6 in _parse_object_unicode (next_idx_ptr=<optimized out>, idx=50500, pystr=0x2ab740437170, s=0x2aab5a4088e0) at /usr/local/src/conda/python-3.10.14/Modules/_json.c:704
#55 scan_once_unicode (s=0x2aab5a4088e0, pystr=0x2ab740437170, idx=50499, next_idx_ptr=<optimized out>) at /usr/local/src/conda/python-3.10.14/Modules/_json.c:1064
#56 0x00002aab129d55b9 in _parse_array_unicode (next_idx_ptr=<optimized out>, idx=<optimized out>, pystr=0x2ab740437170, s=0x2aab5a4088e0) at /usr/local/src/conda/python-3.10.14/Modules/_json.c:841
#57 scan_once_unicode (s=0x2aab5a4088e0, pystr=0x2ab740437170, idx=<optimized out>, next_idx_ptr=<optimized out>) at /usr/local/src/conda/python-3.10.14/Modules/_json.c:1072
--Type <RET> for more, q to quit, c to continue without paging--
#58 0x00002aab129d583f in _parse_object_unicode (next_idx_ptr=<optimized out>, idx=<optimized out>, pystr=0x2ab740437170, s=0x2aab5a4088e0) at /usr/local/src/conda/python-3.10.14/Modules/_json.c:743
#59 scan_once_unicode (s=0x2aab5a4088e0, pystr=0x2ab740437170, idx=<optimized out>, next_idx_ptr=<optimized out>) at /usr/local/src/conda/python-3.10.14/Modules/_json.c:1064
#60 0x00002aab129d55b9 in _parse_array_unicode (next_idx_ptr=<optimized out>, idx=<optimized out>, pystr=0x2ab740437170, s=0x2aab5a4088e0) at /usr/local/src/conda/python-3.10.14/Modules/_json.c:841
#61 scan_once_unicode (s=0x2aab5a4088e0, pystr=0x2ab740437170, idx=<optimized out>, next_idx_ptr=<optimized out>) at /usr/local/src/conda/python-3.10.14/Modules/_json.c:1072
#62 0x00002aab129d583f in _parse_object_unicode (next_idx_ptr=<optimized out>, idx=<optimized out>, pystr=0x2ab740437170, s=0x2aab5a4088e0) at /usr/local/src/conda/python-3.10.14/Modules/_json.c:743
#63 scan_once_unicode (s=0x2aab5a4088e0, pystr=0x2ab740437170, idx=<optimized out>, next_idx_ptr=<optimized out>) at /usr/local/src/conda/python-3.10.14/Modules/_json.c:1064
#64 0x00002aab129d55b9 in _parse_array_unicode (next_idx_ptr=<optimized out>, idx=<optimized out>, pystr=0x2ab740437170, s=0x2aab5a4088e0) at /usr/local/src/conda/python-3.10.14/Modules/_json.c:841
#65 scan_once_unicode (s=0x2aab5a4088e0, pystr=0x2ab740437170, idx=<optimized out>, next_idx_ptr=<optimized out>) at /usr/local/src/conda/python-3.10.14/Modules/_json.c:1072
#66 0x00002aab129d583f in _parse_object_unicode (next_idx_ptr=<optimized out>, idx=<optimized out>, pystr=0x2ab740437170, s=0x2aab5a4088e0) at /usr/local/src/conda/python-3.10.14/Modules/_json.c:743
#67 scan_once_unicode (s=0x2aab5a4088e0, pystr=0x2ab740437170, idx=<optimized out>, next_idx_ptr=<optimized out>) at /usr/local/src/conda/python-3.10.14/Modules/_json.c:1064
#68 0x00002aab129d55b9 in _parse_array_unicode (next_idx_ptr=<optimized out>, idx=<optimized out>, pystr=0x2ab740437170, s=0x2aab5a4088e0) at /usr/local/src/conda/python-3.10.14/Modules/_json.c:841
#69 scan_once_unicode (s=0x2aab5a4088e0, pystr=0x2ab740437170, idx=<optimized out>, next_idx_ptr=<optimized out>) at /usr/local/src/conda/python-3.10.14/Modules/_json.c:1072
#70 0x00002aab129d583f in _parse_object_unicode (next_idx_ptr=<optimized out>, idx=<optimized out>, pystr=0x2ab740437170, s=0x2aab5a4088e0) at /usr/local/src/conda/python-3.10.14/Modules/_json.c:743
#71 scan_once_unicode (s=0x2aab5a4088e0, pystr=0x2ab740437170, idx=<optimized out>, next_idx_ptr=<optimized out>) at /usr/local/src/conda/python-3.10.14/Modules/_json.c:1064
#72 0x00002aab129d4c48 in scanner_call (self=0x2aab5a4088e0, args=<optimized out>, kwds=<optimized out>) at /usr/local/src/conda/python-3.10.14/Modules/_json.c:1149
#73 0x00000000004f705b in _PyObject_MakeTpCall (tstate=0x2ab73c0608c0, callable=0x2aab5a4088e0, args=<optimized out>, nargs=<optimized out>, keywords=0x0) at /usr/local/src/conda/python-3.10.14/Objects/call.c:215
#74 0x00000000004f3106 in _PyObject_VectorcallTstate (kwnames=0x0, nargsf=<optimized out>, args=0x2ab6e5f706d0, callable=0x2aab5a4088e0, tstate=<optimized out>) at /usr/local/src/conda/python-3.10.14/Include/cpython/abstract.h:112
#75 _PyObject_VectorcallTstate (kwnames=0x0, nargsf=<optimized out>, args=0x2ab6e5f706d0, callable=0x2aab5a4088e0, tstate=<optimized out>) at /usr/local/src/conda/python-3.10.14/Include/cpython/abstract.h:99
#76 PyObject_Vectorcall (kwnames=0x0, nargsf=<optimized out>, args=0x2ab6e5f706d0, callable=0x2aab5a4088e0) at /usr/local/src/conda/python-3.10.14/Include/cpython/abstract.h:123
#77 call_function (kwnames=0x0, oparg=<optimized out>, pp_stack=<synthetic pointer>, trace_info=0x2ab24a4ffc60, tstate=<optimized out>) at /usr/local/src/conda/python-3.10.14/Python/ceval.c:5893
#78 _PyEval_EvalFrameDefault (tstate=<optimized out>, f=0x2ab6e5f70530, throwflag=<optimized out>) at /usr/local/src/conda/python-3.10.14/Python/ceval.c:4181
#79 0x00000000005095ce in _PyEval_EvalFrame (throwflag=0, f=0x2ab6e5f70530, tstate=0x2ab73c0608c0) at /usr/local/src/conda/python-3.10.14/Include/internal/pycore_ceval.h:46
#80 _PyEval_Vector (kwnames=<optimized out>, argcount=<optimized out>, args=0x2ab7507cd8b8, locals=0x0, con=0x2aab5a3c35c0, tstate=0x2ab73c0608c0) at /usr/local/src/conda/python-3.10.14/Python/ceval.c:5067
#81 _PyFunction_Vectorcall (kwnames=<optimized out>, nargsf=<optimized out>, stack=0x2ab7507cd8b8, func=0x2aab5a3c35b0) at /usr/local/src/conda/python-3.10.14/Objects/call.c:342
#82 _PyObject_VectorcallTstate (kwnames=<optimized out>, nargsf=<optimized out>, args=0x2ab7507cd8b8, callable=0x2aab5a3c35b0, tstate=0x2ab73c0608c0) at /usr/local/src/conda/python-3.10.14/Include/cpython/abstract.h:114
#83 method_vectorcall (method=<optimized out>, args=0x2ab7507cd8c0, nargsf=<optimized out>, kwnames=<optimized out>) at /usr/local/src/conda/python-3.10.14/Objects/classobject.c:53
#84 0x00000000004ef0e3 in _PyObject_VectorcallTstate (kwnames=0x2aab5a3db430, nargsf=<optimized out>, args=<optimized out>, callable=0x2aafa1a2f640, tstate=0x2ab73c0608c0) at /usr/local/src/conda/python-3.10.14/Include/cpython/abstract.h:114
#85 PyObject_Vectorcall (kwnames=0x2aab5a3db430, nargsf=<optimized out>, args=<optimized out>, callable=0x2aafa1a2f640) at /usr/local/src/conda/python-3.10.14/Include/cpython/abstract.h:123
#86 call_function (kwnames=0x2aab5a3db430, oparg=<optimized out>, pp_stack=<synthetic pointer>, trace_info=0x2ab24a4ffe70, tstate=<optimized out>) at /usr/local/src/conda/python-3.10.14/Python/ceval.c:5893
#87 _PyEval_EvalFrameDefault (tstate=<optimized out>, f=0x2ab7507cd730, throwflag=<optimized out>) at /usr/local/src/conda/python-3.10.14/Python/ceval.c:4231
#88 0x00000000004fdd4f in _PyEval_EvalFrame (throwflag=0, f=0x2ab7507cd730, tstate=0x2ab73c0608c0) at /usr/local/src/conda/python-3.10.14/Include/internal/pycore_ceval.h:46
#89 _PyEval_Vector (kwnames=<optimized out>, argcount=<optimized out>, args=<optimized out>, locals=0x0, con=0x2aab5a3c3530, tstate=0x2ab73c0608c0) at /usr/local/src/conda/python-3.10.14/Python/ceval.c:5067
#90 _PyFunction_Vectorcall (func=0x2aab5a3c3520, stack=<optimized out>, nargsf=<optimized out>, kwnames=<optimized out>) at /usr/local/src/conda/python-3.10.14/Objects/call.c:342
#91 0x00000000004ee461 in _PyObject_VectorcallTstate (kwnames=0x0, nargsf=<optimized out>, args=0x2ab760c7b2e8, callable=0x2aab5a3c3520, tstate=0x2ab73c0608c0) at /usr/local/src/conda/python-3.10.14/Include/cpython/abstract.h:114
#92 PyObject_Vectorcall (kwnames=0x0, nargsf=<optimized out>, args=0x2ab760c7b2e8, callable=0x2aab5a3c3520) at /usr/local/src/conda/python-3.10.14/Include/cpython/abstract.h:123
#93 call_function (kwnames=0x0, oparg=<optimized out>, pp_stack=<synthetic pointer>, trace_info=0x2ab24a500030, tstate=<optimized out>) at /usr/local/src/conda/python-3.10.14/Python/ceval.c:5893
#94 _PyEval_EvalFrameDefault (tstate=<optimized out>, f=0x2ab760c7b140, throwflag=<optimized out>) at /usr/local/src/conda/python-3.10.14/Python/ceval.c:4198
#95 0x00000000004fdd4f in _PyEval_EvalFrame (throwflag=0, f=0x2ab760c7b140, tstate=0x2ab73c0608c0) at /usr/local/src/conda/python-3.10.14/Include/internal/pycore_ceval.h:46
#96 _PyEval_Vector (kwnames=<optimized out>, argcount=<optimized out>, args=<optimized out>, locals=0x0, con=0x2aab5a3c3c80, tstate=0x2ab73c0608c0) at /usr/local/src/conda/python-3.10.14/Python/ceval.c:5067
#97 _PyFunction_Vectorcall (func=0x2aab5a3c3c70, stack=<optimized out>, nargsf=<optimized out>, kwnames=<optimized out>) at /usr/local/src/conda/python-3.10.14/Objects/call.c:342
#98 0x00000000004f08a9 in do_call_core (kwdict=0x2aafa1a23f00, callargs=0x2ab24b44c400, func=0x2aab5a3c3c70, trace_info=0x2ab24a5001f0, tstate=<optimized out>) at /usr/local/src/conda/python-3.10.14/Python/ceval.c:5945
#99 _PyEval_EvalFrameDefault (tstate=<optimized out>, f=0x2aabc3a76510, throwflag=<optimized out>) at /usr/local/src/conda/python-3.10.14/Python/ceval.c:4277
#100 0x00000000004fdd4f in _PyEval_EvalFrame (throwflag=0, f=0x2aabc3a76510, tstate=0x2ab73c0608c0) at /usr/local/src/conda/python-3.10.14/Include/internal/pycore_ceval.h:46
#101 _PyEval_Vector (kwnames=<optimized out>, argcount=<optimized out>, args=<optimized out>, locals=0x0, con=0x2aab5cd65490, tstate=0x2ab73c0608c0) at /usr/local/src/conda/python-3.10.14/Python/ceval.c:5067
#102 _PyFunction_Vectorcall (func=0x2aab5cd65480, stack=<optimized out>, nargsf=<optimized out>, kwnames=<optimized out>) at /usr/local/src/conda/python-3.10.14/Objects/call.c:342
#103 0x00000000004ee461 in _PyObject_VectorcallTstate (kwnames=0x0, nargsf=<optimized out>, args=0x2ab73c4f0310, callable=0x2aab5cd65480, tstate=0x2ab73c0608c0) at /usr/local/src/conda/python-3.10.14/Include/cpython/abstract.h:114
#104 PyObject_Vectorcall (kwnames=0x0, nargsf=<optimized out>, args=0x2ab73c4f0310, callable=0x2aab5cd65480) at /usr/local/src/conda/python-3.10.14/Include/cpython/abstract.h:123
#105 call_function (kwnames=0x0, oparg=<optimized out>, pp_stack=<synthetic pointer>, trace_info=0x2ab24a5003b0, tstate=<optimized out>) at /usr/local/src/conda/python-3.10.14/Python/ceval.c:5893
#106 _PyEval_EvalFrameDefault (tstate=<optimized out>, f=0x2ab73c4f0180, throwflag=<optimized out>) at /usr/local/src/conda/python-3.10.14/Python/ceval.c:4198
#107 0x00000000004fdd4f in _PyEval_EvalFrame (throwflag=0, f=0x2ab73c4f0180, tstate=0x2ab73c0608c0) at /usr/local/src/conda/python-3.10.14/Include/internal/pycore_ceval.h:46
#108 _PyEval_Vector (kwnames=<optimized out>, argcount=<optimized out>, args=<optimized out>, locals=0x0, con=0x2aab9595fad0, tstate=0x2ab73c0608c0) at /usr/local/src/conda/python-3.10.14/Python/ceval.c:5067
#109 _PyFunction_Vectorcall (func=0x2aab9595fac0, stack=<optimized out>, nargsf=<optimized out>, kwnames=<optimized out>) at /usr/local/src/conda/python-3.10.14/Objects/call.c:342
#110 0x00000000004ee461 in _PyObject_VectorcallTstate (kwnames=0x0, nargsf=<optimized out>, args=0x2ab73c008968, callable=0x2aab9595fac0, tstate=0x2ab73c0608c0) at /usr/local/src/conda/python-3.10.14/Include/cpython/abstract.h:114
#111 PyObject_Vectorcall (kwnames=0x0, nargsf=<optimized out>, args=0x2ab73c008968, callable=0x2aab9595fac0) at /usr/local/src/conda/python-3.10.14/Include/cpython/abstract.h:123
#112 call_function (kwnames=0x0, oparg=<optimized out>, pp_stack=<synthetic pointer>, trace_info=0x2ab24a500570, tstate=<optimized out>) at /usr/local/src/conda/python-3.10.14/Python/ceval.c:5893
#113 _PyEval_EvalFrameDefault (tstate=<optimized out>, f=0x2ab73c0087c0, throwflag=<optimized out>) at /usr/local/src/conda/python-3.10.14/Python/ceval.c:4198
#114 0x00000000004fdd4f in _PyEval_EvalFrame (throwflag=0, f=0x2ab73c0087c0, tstate=0x2ab73c0608c0) at /usr/local/src/conda/python-3.10.14/Include/internal/pycore_ceval.h:46
#115 _PyEval_Vector (kwnames=<optimized out>, argcount=<optimized out>, args=<optimized out>, locals=0x0, con=0x2aabc3a49760, tstate=0x2ab73c0608c0) at /usr/local/src/conda/python-3.10.14/Python/ceval.c:5067
#116 _PyFunction_Vectorcall (func=0x2aabc3a49750, stack=<optimized out>, nargsf=<optimized out>, kwnames=<optimized out>) at /usr/local/src/conda/python-3.10.14/Objects/call.c:342
#117 0x00000000004ee461 in _PyObject_VectorcallTstate (kwnames=0x0, nargsf=<optimized out>, args=0x2aabc3ab6550, callable=0x2aabc3a49750, tstate=0x2ab73c0608c0) at /usr/local/src/conda/python-3.10.14/Include/cpython/abstract.h:114
--Type <RET> for more, q to quit, c to continue without paging--
#118 PyObject_Vectorcall (kwnames=0x0, nargsf=<optimized out>, args=0x2aabc3ab6550, callable=0x2aabc3a49750) at /usr/local/src/conda/python-3.10.14/Include/cpython/abstract.h:123
#119 call_function (kwnames=0x0, oparg=<optimized out>, pp_stack=<synthetic pointer>, trace_info=0x2ab24a500730, tstate=<optimized out>) at /usr/local/src/conda/python-3.10.14/Python/ceval.c:5893
#120 _PyEval_EvalFrameDefault (tstate=<optimized out>, f=0x2aabc3ab63e0, throwflag=<optimized out>) at /usr/local/src/conda/python-3.10.14/Python/ceval.c:4198
#121 0x0000000000509857 in _PyEval_EvalFrame (throwflag=0, f=0x2aabc3ab63e0, tstate=0x2ab73c0608c0) at /usr/local/src/conda/python-3.10.14/Include/internal/pycore_ceval.h:46
#122 _PyEval_Vector (kwnames=0x0, argcount=1, args=0x2ab24a500818, locals=0x0, con=0x2aabc3a495b0, tstate=0x2ab73c0608c0) at /usr/local/src/conda/python-3.10.14/Python/ceval.c:5067
#123 _PyFunction_Vectorcall (kwnames=0x0, nargsf=1, stack=0x2ab24a500818, func=0x2aabc3a495a0) at /usr/local/src/conda/python-3.10.14/Objects/call.c:342
#124 _PyObject_VectorcallTstate (kwnames=0x0, nargsf=1, args=0x2ab24a500818, callable=0x2aabc3a495a0, tstate=0x2ab73c0608c0) at /usr/local/src/conda/python-3.10.14/Include/cpython/abstract.h:114
#125 method_vectorcall (method=<optimized out>, args=<optimized out>, nargsf=<optimized out>, kwnames=<optimized out>) at /usr/local/src/conda/python-3.10.14/Objects/classobject.c:61
#126 0x00000000004f08a9 in do_call_core (kwdict=0x2ab24a2a7040, callargs=0x2aaaaae78070, func=0x2aaf53f6b080, trace_info=0x2ab24a500940, tstate=<optimized out>) at /usr/local/src/conda/python-3.10.14/Python/ceval.c:5945
#127 _PyEval_EvalFrameDefault (tstate=<optimized out>, f=0x2aabc3ab67a0, throwflag=<optimized out>) at /usr/local/src/conda/python-3.10.14/Python/ceval.c:4277
#128 0x00000000004fdd4f in _PyEval_EvalFrame (throwflag=0, f=0x2aabc3ab67a0, tstate=0x2ab73c0608c0) at /usr/local/src/conda/python-3.10.14/Include/internal/pycore_ceval.h:46
#129 _PyEval_Vector (kwnames=<optimized out>, argcount=<optimized out>, args=<optimized out>, locals=0x0, con=0x2aaaab161370, tstate=0x2ab73c0608c0) at /usr/local/src/conda/python-3.10.14/Python/ceval.c:5067
#130 _PyFunction_Vectorcall (func=0x2aaaab161360, stack=<optimized out>, nargsf=<optimized out>, kwnames=<optimized out>) at /usr/local/src/conda/python-3.10.14/Objects/call.c:342
#131 0x00000000004ee461 in _PyObject_VectorcallTstate (kwnames=0x0, nargsf=<optimized out>, args=0x2aaf52c618f0, callable=0x2aaaab161360, tstate=0x2ab73c0608c0) at /usr/local/src/conda/python-3.10.14/Include/cpython/abstract.h:114
#132 PyObject_Vectorcall (kwnames=0x0, nargsf=<optimized out>, args=0x2aaf52c618f0, callable=0x2aaaab161360) at /usr/local/src/conda/python-3.10.14/Include/cpython/abstract.h:123
#133 call_function (kwnames=0x0, oparg=<optimized out>, pp_stack=<synthetic pointer>, trace_info=0x2ab24a500b00, tstate=<optimized out>) at /usr/local/src/conda/python-3.10.14/Python/ceval.c:5893
#134 _PyEval_EvalFrameDefault (tstate=<optimized out>, f=0x2aaf52c61780, throwflag=<optimized out>) at /usr/local/src/conda/python-3.10.14/Python/ceval.c:4198
#135 0x00000000004fdd4f in _PyEval_EvalFrame (throwflag=0, f=0x2aaf52c61780, tstate=0x2ab73c0608c0) at /usr/local/src/conda/python-3.10.14/Include/internal/pycore_ceval.h:46
#136 _PyEval_Vector (kwnames=<optimized out>, argcount=<optimized out>, args=<optimized out>, locals=0x0, con=0x2aaaab161640, tstate=0x2ab73c0608c0) at /usr/local/src/conda/python-3.10.14/Python/ceval.c:5067
#137 _PyFunction_Vectorcall (func=0x2aaaab161630, stack=<optimized out>, nargsf=<optimized out>, kwnames=<optimized out>) at /usr/local/src/conda/python-3.10.14/Objects/call.c:342
#138 0x00000000004ee461 in _PyObject_VectorcallTstate (kwnames=0x0, nargsf=<optimized out>, args=0x2aaf5284d470, callable=0x2aaaab161630, tstate=0x2ab73c0608c0) at /usr/local/src/conda/python-3.10.14/Include/cpython/abstract.h:114
#139 PyObject_Vectorcall (kwnames=0x0, nargsf=<optimized out>, args=0x2aaf5284d470, callable=0x2aaaab161630) at /usr/local/src/conda/python-3.10.14/Include/cpython/abstract.h:123
#140 call_function (kwnames=0x0, oparg=<optimized out>, pp_stack=<synthetic pointer>, trace_info=0x2ab24a500cc0, tstate=<optimized out>) at /usr/local/src/conda/python-3.10.14/Python/ceval.c:5893
#141 _PyEval_EvalFrameDefault (tstate=<optimized out>, f=0x2aaf5284d300, throwflag=<optimized out>) at /usr/local/src/conda/python-3.10.14/Python/ceval.c:4198
#142 0x0000000000509857 in _PyEval_EvalFrame (throwflag=0, f=0x2aaf5284d300, tstate=0x2ab73c0608c0) at /usr/local/src/conda/python-3.10.14/Include/internal/pycore_ceval.h:46
#143 _PyEval_Vector (kwnames=0x0, argcount=1, args=0x2ab24a500da8, locals=0x0, con=0x2aaaab161400, tstate=0x2ab73c0608c0) at /usr/local/src/conda/python-3.10.14/Python/ceval.c:5067
#144 _PyFunction_Vectorcall (kwnames=0x0, nargsf=1, stack=0x2ab24a500da8, func=0x2aaaab1613f0) at /usr/local/src/conda/python-3.10.14/Objects/call.c:342
#145 _PyObject_VectorcallTstate (kwnames=0x0, nargsf=1, args=0x2ab24a500da8, callable=0x2aaaab1613f0, tstate=0x2ab73c0608c0) at /usr/local/src/conda/python-3.10.14/Include/cpython/abstract.h:114
#146 method_vectorcall (method=<optimized out>, args=<optimized out>, nargsf=<optimized out>, kwnames=<optimized out>) at /usr/local/src/conda/python-3.10.14/Objects/classobject.c:61
#147 0x00000000005e5dd5 in thread_run (boot_raw=0x2aabc3aba940) at /usr/local/src/conda/python-3.10.14/Modules/_threadmodule.c:1100
#148 0x00000000005e5d34 in pythread_wrapper (arg=<optimized out>) at /usr/local/src/conda/python-3.10.14/Python/thread_pthread.h:248
#149 0x00002aaaaac83ac3 in start_thread (arg=<optimized out>) at ./nptl/pthread_create.c:442
#150 0x00002aaaaad14a04 in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:100

[rmast]

Those hyperlinks cause all kinds of mentioning-pollution. Can you put the log in between a 'code' block?

[HaoRenkk123]

I got the same question：
my server code:

error:

[mara004]

That's expected, as one of marker's dependencies (pypdfium2/pdfium) is not thread-compatible:
It is the caller's responsibility to use locks or similar to prevent threaded access of pdfium's APIs.
https://pypdfium2.readthedocs.io/en/stable/python_api.html#thread-incompatibility

[wciq1208]

That's expected, as one of marker's dependencies (pypdfium2/pdfium) is not thread-compatible: It is the caller's responsibility to use locks or similar to prevent threaded access of pdfium's APIs. https://pypdfium2.readthedocs.io/en/stable/python_api.html#thread-incompatibility

I have already used a thread lock during parsing, but the core dump occurred during GC

[mara004]

I have already used a thread lock during parsing, but the core dump occurred during GC

Hmm, pypdfium2 auto-closes pdfium objects on garbage collection using weakref.finalize(), if the caller did not close explicitly. In the above backtrace, there is an FPDF_ClosePage() call on #17.
The question is, who causes simultaneous pdfium calls during the GC phase, and how can we prevent that? Or is there some prior corruption that causes the close call to fail?

A possible workaround/test might be to add explicit close calls to all pdfium root objects throughout the dependencies and see if that fixes the issue.

Unfortunately, threading/GC-related issues are hard to debug.

[aj8907]

I had a similar error with the corrupted double-linked list (Colab A100). Running this code fixed the problem for me:

from threading import RLock
from contextlib import contextmanager
import logging
import time
import traceback

class SafeLock:
    def __init__(self, name="SafeLock", timeout=60, max_retries=3, retry_delay=1):
        self._lock = RLock()  # Reentrant lock is safer than regular Lock
        self.name = name
        self.timeout = timeout
        self.max_retries = max_retries
        self.retry_delay = retry_delay
        self.logger = logging.getLogger(__name__)

    @contextmanager
    def acquire_safely(self):
        attempt = 0
        while attempt < self.max_retries:
            try:
                acquired = self._lock.acquire(timeout=self.timeout)
                if acquired:
                    try:
                        yield
                    except Exception as e:
                        self.logger.error(f"Error while holding lock: {str(e)}\n{traceback.format_exc()}")
                        raise
                    finally:
                        try:
                            self._lock.release()
                        except Exception as e:
                            self.logger.error(f"Error releasing lock: {str(e)}")
                    return
                else:
                    attempt += 1
                    self.logger.warning(
                        f"Failed to acquire lock {self.name} (attempt {attempt}/{self.max_retries})"
                    )
                    time.sleep(self.retry_delay)
            except Exception as e:
                attempt += 1
                self.logger.error(f"Lock acquisition error: {str(e)}")
                time.sleep(self.retry_delay)
                
        raise TimeoutError(f"Failed to acquire {self.name} after {self.max_retries} attempts")

safe_lock = SafeLock(name="PDFConverter", timeout=120, max_retries=5, retry_delay=2)
fpath = "/path/to/pdf/file.pdf"
with safe_lock.acquire_safely():
    try:
        full_text, images, out_meta = convert_single_pdf(fpath, model_lst)
    except Exception as e:
        logging.error(f"PDF conversion error: {str(e)}\n{traceback.format_exc()}")
        raise

[mara004]

@aj8907 Sorry, I'm not much into threading, but I don't logically see how this is supposed to fix the above issue? And why is a bare RLock not sufficient?

The question is, who causes simultaneous pdfium calls during the GC phase, and how can we prevent that? Or is there some prior corruption that causes the close call to fail?

If the cause is indeed simultaneous calls due to GC, and not other caller-caused corruption, I figured we may be able to add an API to plug in a caller-provided lock into our auto-close machinery.
@wciq1208, or anyone else affected: The pre-requisite for me to work on this would be a minimal reproducible example (the snippet in the initial post is incomplete).

[wciq1208]

@aj8907 Sorry, I'm not much into threading, but I don't logically see how this is supposed to fix the above issue? And why is a bare RLock not sufficient?

The question is, who causes simultaneous pdfium calls during the GC phase, and how can we prevent that? Or is there some prior corruption that causes the close call to fail?

If the cause is indeed simultaneous calls due to GC, and not other caller-caused corruption, I figured we may be able to add an API to plug in a caller-provided lock into our auto-close machinery. @wciq1208, or anyone else affected: The pre-requisite for me to work on this would be a minimal reproducible example (the snippet in the initial post is incomplete).

I have switched from multithreading to multiprocessing for inference, and my project version is currently at 0.2.17. Since the project seems to be undergoing a restructuring for version 2, I don't need a solution to this issue for now.