Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Capture Create fails to execute IP Tables commands #895

Open
kamilprz opened this issue Oct 25, 2024 · 0 comments · May be fixed by #903
Open

Capture Create fails to execute IP Tables commands #895

kamilprz opened this issue Oct 25, 2024 · 0 comments · May be fixed by #903
Assignees
@kamilprz @ibezrukavyi
Labels
area/captures priority/1 P1 type/bug Something isn't working type/fix Fixes something

Comments

@kamilprz
Copy link
Contributor

Describe the bug
In the current release 0.0.16, the capture create fails to run IP Tables commands failing with an error such as:
msg="Failed to execute command" command=iptables-nft-save error="exec: \"iptables-nft-save\": executable file not found in $PATH".

To Reproduce
Steps to reproduce the behavior:

  1. Have Retina CLI installed -
  2. Run capture command kubectl retina capture create --name <name> --host-path /mnt/retina/captures --namespace capture --node-names "<node-name>"
  3. The failure can be seen in the logs of the pod. Run kubectl get pods.
  4. Run kubectl logs <pod> where matches the job name output by the CLI. e.g. job - testing-5k5xp, pod - testing-5k5xp-t8h9v

Expected behavior
A clear and concise description of what you expected to happen.

Logs

ts=2024-10-21T23:16:42.815Z level=info caller=captureworkload/main.go:27 msg="Start to capture network traffic"
ts=2024-10-21T23:16:42.815Z level=info caller=captureworkload/main.go:28 msg="Version: " version=v0.0.16
ts=2024-10-21T23:16:42.817Z level=info caller=provider/network_capture_unix.go:46 msg="Created temporary folder for network capture" capture temporary folder=/tmp/kapinger-capture-aks-nodepool1-29743410-vmss000001-20241021111642UTC
ts=2024-10-21T23:16:42.818Z level=info caller=provider/network_capture_unix.go:95 msg="Running tcpdump with args" tcpdump command="/usr/bin/tcpdump -w /tmp/kapinger-capture-aks-nodepool1-29743410-vmss000001-20241021111642UTC/kapinger-capture-aks-nodepool1-29743410-vmss000001-20241021111642UTC.pcap --relinquish-privileges=root" tcpdump args="tcpdump,-w,/tmp/kapinger-capture-aks-nodepool1-29743410-vmss000001-20241021111642UTC/kapinger-capture-aks-nodepool1-29743410-vmss000001-20241021111642UTC.pcap,--relinquish-privileges=root"
ts=2024-10-21T23:16:42.820Z level=info caller=provider/network_capture_unix.go:124 msg="Tcpdump will stop after 90 seconds"
ts=2024-10-21T23:16:42.821Z level=info caller=provider/network_capture_unix.go:136 msg="Tcpdump will stop when the capture file size reaches 100MB."
ts=2024-10-21T23:18:12.827Z level=info caller=provider/network_capture_unix.go:172 msg="Stop tcpdump"
ts=2024-10-21T23:18:12.864Z level=info caller=provider/network_capture_unix.go:107 msg="Tcpdump command output: /usr/bin/tcpdump -w /tmp/kapinger-capture-aks-nodepool1-29743410-vmss000001-20241021111642UTC/kapinger-capture-aks-nodepool1-29743410-vmss000001-20241021111642UTC.pcap --relinquish-privileges=root\n\ndropped privs to root\ntcpdump: listening on eth0, link-type EN10MB (Ethernet), snapshot length 262144 bytes\n15009 packets captured\n15244 packets received by filter\n0 packets dropped by kernel\n"
ts=2024-10-21T23:18:12.864Z level=info caller=provider/network_capture_unix.go:197 msg="Start to collect network metadata"
ts=2024-10-21T23:18:12.875Z level=info caller=provider/network_capture_unix.go:200 msg="Iptables mode nft is used"
ts=2024-10-21T23:18:12.902Z level=error caller=provider/network_capture_unix.go:332 msg="Failed to execute command" command=iptables-nft-save error="exec: \"iptables-nft-save\": executable file not found in $PATH"
ts=2024-10-21T23:18:12.902Z level=error caller=provider/network_capture_unix.go:332 msg="Failed to execute command" command="iptables-nft -vnx -L" error="exec: \"iptables-nft\": executable file not found in $PATH"
ts=2024-10-21T23:18:12.902Z level=error caller=provider/network_capture_unix.go:332 msg="Failed to execute command" command="iptables-nft -vnx -L -t nat" error="exec: \"iptables-nft\": executable file not found in $PATH"
ts=2024-10-21T23:18:12.902Z level=error caller=provider/network_capture_unix.go:332 msg="Failed to execute command" command="iptables-nft -vnx -L -t mangle" error="exec: \"iptables-nft\": executable file not found in $PATH"
ts=2024-10-21T23:18:14.910Z level=error caller=provider/network_capture_unix.go:347 msg="Failed to execute command" command="/usr/bin/cp -r /proc/sys/net /tmp/kapinger-capture-aks-nodepool1-29743410-vmss000001-20241021111642UTC/proc-sys-net" output="cp: cannot open '/proc/sys/net/ipv4/route/flush' for reading: Permission denied\ncp: error reading '/proc/sys/net/ipv6/conf/all/stable_secret': Input/output error\ncp: error reading '/proc/sys/net/ipv6/conf/azv17791f70475/stable_secret': Input/output error\ncp: error reading '/proc/sys/net/ipv6/conf/azv18fde3e97f5/stable_secret': Input/output error\ncp: error reading '/proc/sys/net/ipv6/conf/azv4604a4ca891/stable_secret': Input/output error\ncp: error reading '/proc/sys/net/ipv6/conf/azv51ad0378aba/stable_secret': Input/output error\ncp: error reading '/proc/sys/net/ipv6/conf/azv5dcd2ef4dda/stable_secret': Input/output error\ncp: error reading '/proc/sys/net/ipv6/conf/azvcdaea4a954a/stable_secret': Input/output error\ncp: error reading '/proc/sys/net/ipv6/conf/azvebfafc62717/stable_secret': Input/output error\ncp: error reading '/proc/sys/net/ipv6/conf/default/stable_secret': Input/output error\ncp: error reading '/proc/sys/net/ipv6/conf/enP3685s1/stable_secret': Input/output error\ncp: error reading '/proc/sys/net/ipv6/conf/eth0/stable_secret': Input/output error\ncp: error reading '/proc/sys/net/ipv6/conf/lo/stable_secret': Input/output error\ncp: cannot open '/proc/sys/net/ipv6/route/flush' for reading: Permission denied\n" error="exit status 1"
ts=2024-10-21T23:18:14.910Z level=info caller=provider/network_capture_unix.go:353 msg="Done for collecting network metadata"
ts=2024-10-21T23:18:21.304Z level=info caller=outputlocation/hostpath.go:42 msg="Copy file" location=HostPath source file path=/tmp/kapinger-capture-aks-nodepool1-29743410-vmss000001-20241021111642UTC.tar.gz destination file path=/tmp
ts=2024-10-21T23:18:21.309Z level=info caller=captureworkload/main.go:67 msg="Done for capturing network traffic"
ts=2024-10-21T23:18:21.309Z level=info caller=provider/network_capture_unix.go:359 msg="Cleanup network capture" capture name=kapinger-capture temporary dir=/tmp/kapinger-capture-aks-nodepool1-29743410-vmss000001-20241021111642UTC
Stream closed EOF for default/kapinger-capture-smzvd-gvjr2 (capture)

Platform (please complete the following information):

  • OS: WSL
  • Kubernetes Version: 1.31.0
  • Host: AKS
  • Retina Version: 0.0.16
@kamilprz kamilprz added area/captures priority/1 P1 type/bug Something isn't working type/fix Fixes something labels Oct 25, 2024
@kamilprz kamilprz self-assigned this Oct 25, 2024
@ibezrukavyi ibezrukavyi self-assigned this Oct 25, 2024
@kamilprz kamilprz linked a pull request Nov 4, 2024 that will close this issue
fix(capture): ignore known copy failure and fix iptables issue #903
Open
7 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@kamilprz kamilprz

@ibezrukavyi ibezrukavyi

Labels
area/captures priority/1 P1 type/bug Something isn't working type/fix Fixes something
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

fix(capture): ignore known copy failure and fix iptables issue mainred/retina
2 participants
@kamilprz @ibezrukavyi