_kendall_p_exact error when really large datasets are run

[rmflight]

_kendall_p_exact is no longer at mstats_basic._kendall_p_exact, at least in scipy v 1.13.0

It seems to now be at mstats._mstats_basic._kendall_p_exact.

We probably need an import check whether the function can be reached, and maybe we should just make our own copy as part of the icikt codebase.

[rmflight]

OK, I'm wrong. I'm not sure what is causing this, except that I get an error about not finding the function when I'm running the really large examples on KyRIC. Running smaller examples has no issues, and the error doesn't come up.

With the 45K case, I get this error:

AttributeError: `scipy.stats.mstats_basic` has no attribute `_kendall_p_exact`; furthermore, `
scipy.stats.mstats_basic` is deprecated and will be removed in SciPy 2.0.0.

However, running the exact same version of icikt on a small version of the data on KyRIC has no errors. I'm kind of flummoxed, honestly, as to why it fails with a really large dataset, but not with a much smaller one.

[hunter-moseley]

How much memory is still free in the 45K case? Maybe an import is failing due to lack of memory, but I would expect this to throw an error.

…

On Tue, Apr 30, 2024 at 3:47 PM Robert M Flight ***@***.***> wrote: OK, I'm wrong. I'm not sure what is causing this, except that I get an error about not finding the function when I'm running the *really* large examples on KyRIC. Running smaller examples has no issues, and the error doesn't come up. With the 45K case, I get this error: AttributeError: `scipy.stats.mstats_basic` has no attribute `_kendall_p_exact`; furthermore, ` scipy.stats.mstats_basic` is deprecated and will be removed in SciPy 2.0.0. However, running the exact same version of icikt on a small version of the data on KyRIC has no errors. I'm kind of flummoxed, honestly, as to why it fails with a really large dataset, but not with a much smaller one. — Reply to this email directly, view it on GitHub <#13 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ADEP7B7ER5ISSTY3HGJR4NDY77YOFAVCNFSM6AAAAABHAZGBMCVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDAOBWHEYTCNRSGE> . You are receiving this because you are subscribed to this thread.Message ID: ***@***.***>

-- Hunter Moseley, Ph.D. -- Univ. of Kentucky Professor, Dept. of Molec. & Cell. Biochemistry / Markey Cancer Center / Institute for Biomedical Informatics / UK Superfund Research Center Not just a scientist, but a fencer as well. My foil is sharp, but my mind sharper still. --------------------------------------------------------------- Email: ***@***.*** (work) ***@***.*** (personal) Phone: 859-218-2964 (office) 859-218-2965 (lab) 859-257-7715 (fax) Web: http://bioinformatics.cesb.uky.edu/ Address: CC434 Roach Building, 800 Rose Street, Lexington, KY 40536-0093

[rmflight]

I'm really not sure. It takes forever to run, so if there is a memory issue, it may be happening late in the game, or with later entries.

I can try tracking memory with gpu_tracker, or I'll write a little memory tracker script in R to log the memory usage every minute or so over the course of the calculation.