Add support for Hyperlight KVM guest debugging using gdb #111

dblnz · 2024-12-13T17:54:02Z

The current implementation supports:

add/remove HW breakpoints (max 4)
add/remove SW breakpoints
read/write registers
read/write addresses
step/continue
get code offset from target

The overall architecture is designed to work like a Request - Response protocol from the gdb thread to the hypervisor handler over a DebugCommChannel.

All the functionality is implemented on the hypervisor side so it has access to the shared memory and VcpuFd
The gdb thread uses gdbstub to handle the communication with the gdb client
when the gdb client requests the supported features mentioned above, a request is sent forward to the hypervisor handler for the running sandbox to run

devigned

Super excited to see this landing. Nice work! My feedback mostly consists of nits and questions.

docs/how-to-debug-a-hyperlight-guest.md

src/hyperlight_host/src/hypervisor/gdb/mod.rs

src/hyperlight_host/src/hypervisor/gdb/target.rs

src/hyperlight_host/src/hypervisor/hypervisor_handler.rs

src/hyperlight_host/src/hypervisor/kvm.rs

src/hyperlight_host/src/hypervisor/hypervisor_handler.rs

ludfjig

This is awesome!! Looked briefly though everything and I will give a more in-depth review tomorrow

src/hyperlight_host/Cargo.toml

src/hyperlight_host/src/hypervisor/gdb/mod.rs

src/hyperlight_host/src/sandbox/config.rs

src/hyperlight_host/src/hypervisor/gdb/x86_64_target.rs

ludfjig

This is awesome. I tried it out and it works great. Left some more comments, feel free to dismiss if you disagree with any of them. I realize some of them might be hard to implement as well. GREAT WORK!!

src/hyperlight_host/src/sandbox/config.rs

docs/how-to-debug-a-hyperlight-guest.md

src/hyperlight_host/Cargo.toml

src/hyperlight_host/src/hypervisor/kvm.rs

src/hyperlight_host/src/hypervisor/gdb/event_loop.rs

src/hyperlight_host/src/hypervisor/gdb/mod.rs

docs/how-to-debug-a-hyperlight-guest.md

simongdavies

a few minor issues, but overall this is great. I don't know why the spelling issues are not getting picked up by the spell check in GH actions

docs/how-to-debug-a-hyperlight-guest.md

src/hyperlight_host/build.rs

src/hyperlight_host/src/hypervisor/gdb/mod.rs

src/hyperlight_host/src/hypervisor/kvm.rs

src/hyperlight_host/src/sandbox/mem_access.rs

syntactically

I really like how this looks overall! I've left a few small things in the code comments. I also have a couple of nits about the commit organization:

Commit message for 579b324: s/tread/thread/

Commits b175f41, af0a52a, 6cc6184, and 98ac60e: please convert
each one to a series of fixup! commits to the original commits
introducing code, so that we can autosquash before merging, or just
rebase the changes into the original commits now if nobody else
objects to that.

src/hyperlight_host/src/hypervisor/hypervisor_handler.rs

src/hyperlight_host/src/sandbox/config.rs

src/hyperlight_host/src/sandbox/uninitialized.rs

src/hyperlight_host/src/hypervisor/gdb/event_loop.rs

src/hyperlight_host/src/hypervisor/gdb/x86_64_target.rs

src/hyperlight_host/src/sandbox/mem_access.rs

src/hyperlight_host/src/hypervisor/gdb/mod.rs

src/hyperlight_host/src/hypervisor/gdb/x86_64_target.rs

src/hyperlight_host/src/hypervisor/kvm.rs

Signed-off-by: Doru Blânzeanu <[email protected]>

…en gdb feature is on Signed-off-by: Doru Blânzeanu <[email protected]>

Signed-off-by: Doru Blânzeanu <[email protected]>

- this type will be used by the gdb and the hypervisor handler to send requests and receive responses Signed-off-by: Doru Blânzeanu <[email protected]>

- the target implements the traits to provide callbacks for the gdb commands Signed-off-by: Doru Blânzeanu <[email protected]>

- adds a specific handler for the vcpu exit debug that waits for debug messages and processes them Signed-off-by: Doru Blânzeanu <[email protected]>

Signed-off-by: Doru Blânzeanu <[email protected]>

- also adds handling of gdb client disconnect by resuming execution Signed-off-by: Doru Blânzeanu <[email protected]>

Signed-off-by: Doru Blânzeanu <[email protected]>

- sometimes gdb tries to read from an address where the vcpu translate method fails, so we need to handle that in such a way that the hypervisor handler doesn't crash Signed-off-by: Doru Blânzeanu <[email protected]>

Signed-off-by: Doru Blânzeanu <[email protected]>

dblnz added the kind/enhancement New feature or request label Dec 13, 2024

dblnz self-assigned this Dec 13, 2024

dblnz marked this pull request as draft December 13, 2024 17:54

dblnz force-pushed the gdb-support-latest branch 2 times, most recently from be21b85 to ca61def Compare December 19, 2024 11:41

devigned requested changes Jan 22, 2025

View reviewed changes

dblnz force-pushed the gdb-support-latest branch from 4a44b2e to 16492de Compare January 22, 2025 15:40

dblnz marked this pull request as ready for review January 22, 2025 16:07

dblnz changed the title ~~Initial support for gdb debugging in Hyperlight KVM guest~~ Add support for Hyperlight KVM guest debugging using gdb Jan 22, 2025

danbugs reviewed Jan 28, 2025

View reviewed changes

src/hyperlight_host/src/hypervisor/hypervisor_handler.rs Outdated Show resolved Hide resolved

src/hyperlight_host/src/hypervisor/hypervisor_handler.rs Outdated Show resolved Hide resolved

dblnz force-pushed the gdb-support-latest branch from 2a36f06 to d0db26e Compare February 4, 2025 21:07

ludfjig previously approved these changes Feb 5, 2025

View reviewed changes

dblnz dismissed ludfjig’s stale review via 6cc6184 February 5, 2025 12:57

dblnz force-pushed the gdb-support-latest branch from 17a5d14 to 98ac60e Compare February 5, 2025 16:06

ludfjig previously approved these changes Feb 7, 2025

View reviewed changes

simongdavies requested changes Feb 11, 2025

View reviewed changes

syntactically reviewed Feb 11, 2025

View reviewed changes

dblnz added 13 commits February 13, 2025 19:07

add gdb cargo feature

e2832da

Signed-off-by: Doru Blânzeanu <[email protected]>

disable timeouts for messaging between host and hypervisor handler wh…

77972ad

…en gdb feature is on Signed-off-by: Doru Blânzeanu <[email protected]>

add guest debug port to sandbox configuration

5b7de0d

Signed-off-by: Doru Blânzeanu <[email protected]>

add gdb debug tread creation method

03c68a1

Signed-off-by: Doru Blânzeanu <[email protected]>

add debug communication channel type

431a99b

- this type will be used by the gdb and the hypervisor handler to send requests and receive responses Signed-off-by: Doru Blânzeanu <[email protected]>

add hyperlight sandbox target to handle gdb commands support

3cf00f7

- the target implements the traits to provide callbacks for the gdb commands Signed-off-by: Doru Blânzeanu <[email protected]>

handle debug vcpu exit on kvm side

308825a

- adds a specific handler for the vcpu exit debug that waits for debug messages and processes them Signed-off-by: Doru Blânzeanu <[email protected]>

add kvm debug configuration that handles vCPU interraction

b4cb43d

Signed-off-by: Doru Blânzeanu <[email protected]>

add read/write registers support

3752649

Signed-off-by: Doru Blânzeanu <[email protected]>

hw breakpoint and resume support added

a6cce6b

Signed-off-by: Doru Blânzeanu <[email protected]>

add stepping support

4860fb6

- also adds handling of gdb client disconnect by resuming execution Signed-off-by: Doru Blânzeanu <[email protected]>

add a way to read/write host shared mem when debugging

f2e13d7

Signed-off-by: Doru Blânzeanu <[email protected]>

add read/write address and text section offset support

d3da3b1

Signed-off-by: Doru Blânzeanu <[email protected]>

dblnz added 4 commits February 13, 2025 19:26

add sw breakpoint support

04bd6b3

Signed-off-by: Doru Blânzeanu <[email protected]>

add ci checks for the gdb feature

07da801

Signed-off-by: Doru Blânzeanu <[email protected]>

add documentation about the gdb debugging feature

ea250ac

Signed-off-by: Doru Blânzeanu <[email protected]>

add a way to report a non-fatal error to gdb

459b1c9

- sometimes gdb tries to read from an address where the vcpu translate method fails, so we need to handle that in such a way that the hypervisor handler doesn't crash Signed-off-by: Doru Blânzeanu <[email protected]>

dblnz dismissed ludfjig’s stale review via 459b1c9 February 13, 2025 17:29

dblnz force-pushed the gdb-support-latest branch from 98ac60e to 459b1c9 Compare February 13, 2025 17:29

fix typo from untouched file

baa46d8

Signed-off-by: Doru Blânzeanu <[email protected]>

dblnz requested a review from devigned February 13, 2025 17:59

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add support for Hyperlight KVM guest debugging using gdb #111

Add support for Hyperlight KVM guest debugging using gdb #111

dblnz commented Dec 13, 2024 •

edited

Loading

devigned left a comment

ludfjig left a comment

ludfjig left a comment

simongdavies left a comment

syntactically left a comment

Add support for Hyperlight KVM guest debugging using gdb #111

Are you sure you want to change the base?

Add support for Hyperlight KVM guest debugging using gdb #111

Conversation

dblnz commented Dec 13, 2024 • edited Loading

devigned left a comment

Choose a reason for hiding this comment

ludfjig left a comment

Choose a reason for hiding this comment

ludfjig left a comment

Choose a reason for hiding this comment

simongdavies left a comment

Choose a reason for hiding this comment

syntactically left a comment

Choose a reason for hiding this comment

dblnz commented Dec 13, 2024 •

edited

Loading