Hitting a stack overflow during marking

[helixbass]

Hi, I got the project where I've been swapping in this library (vs Rc/RefCell's) compiling and when then trying to run it I'm encountering a stack overflow during the mark phase of garbage collection, here's the stack trace

From a little perusal of the stack trace it doesn't look like it's getting stuck in an infinite loop or anything, I know little about garbage collection but from a little Googling it sounds like this may be a known problem if you use the call stack "recursively" for traversing nested/linked data structures during marking?

[Manishearth]

Hmm, we're not recursively chasing Gc types, we only recursively call .trace() on pointers not managed by the Gc.

It's possible the Rc stuff is causing this if you still have some around, the development version of this crate no longer includes traceable Rcs by default, opting to let the user figure that out instead.

[andersk]

Hmm, we're not recursively chasing Gc types, we only recursively call .trace() on pointers not managed by the Gc.

This must not be the case, since I can reproduce a stack overflow with

use gc::{Finalize, Gc, Trace};

#[derive(Finalize, Trace)]
struct Node(Option<Gc<Node>>);

#[test]
fn test_recursion() {
    let mut node = Node(None);
    for _ in 0..100000 {
        node = Node(Some(Gc::new(node)));
    }
}

[Manishearth]

... huh

[helixbass]

Ya looking at the codebase it looks like Gc's Trace implementation calls into GcBox::trace_inner() which calls self.data.trace() (so calls into the Trace implementation of the Gc-contained data)

I tried taking an ultra naive stab at converting that to pushing onto a queue instead but so far am failing lol

Also as I figured might be the case (because "stuff gets inlined at non-default compiler optimization levels so the stack doesn't get as big"?) it's not stack overflowing for that test case when I compile in release mode fwiw

[helixbass]

Ok I seem to have a basic "push onto a queue when we encounter Gc's during tracing instead of synchronously tracing their nested contents" working (not stack overflowing + seems to collect the same amount of garbage as the release-built previous version for my test case, tests are passing)

I opened a PR against my own fork (helixbass#1) as a way to leave a couple comments there if you want to actually look at what I did but I'm guessing that/would not be offended if you're better off starting from scratch if this is indeed roughly the idea of how to adjust existing behavior