docker容器中的blktrace输出错误
我的docker容器的基本形象是Ubuntu的,我运行它的完整权限选项,这意味着在运行命令我使用这些开关:
--cap-add=SYS_ADMIN --security-opt apparmor:unconfined
我想用下面的命令使用blktrce:
sudo blktrace -d / -a issue -o - | blkparse -f "%p %T.%9t %D %S ^C %d\n" -i - >stream.out
但是,第一次使用这个命令时,我得到这个错误:
Debugfs is not mounted at /sys/kernel/debug
我search,发现这个解决scheme ,导致使用这个命令:
mount -t debugfs none /sys/kernel/debug
之后,当我再次使用blktrace
命令,我得到这个错误:
BLKTRACESETUP(2) / failed: 25/Inappropriate ioctl for device Thread 1 failed open /sys/kernel/debug/block/(null)/trace1: 2/No such file or directory Thread 0 failed open /sys/kernel/debug/block/(null)/trace0: 2/No such file or directory Thread 2 failed open /sys/kernel/debug/block/(null)/trace2: 2/No such file or directory Thread 4 failed open /sys/kernel/debug/block/(null)/trace4: 2/No such file or directory Thread 3 failed open /sys/kernel/debug/block/(null)/trace3: 2/No such file or directory Thread 5 failed open /sys/kernel/debug/block/(null)/trace5: 2/No such file or directory Thread 7 failed open /sys/kernel/debug/block/(null)/trace7: 2/No such file or directory Thread 8 failed open /sys/kernel/debug/block/(null)/trace8: 2/No such file or directory Thread 6 failed open /sys/kernel/debug/block/(null)/trace6: 2/No such file or directory Thread 12 failed open /sys/kernel/debug/block/(null)/trace12: 2/No such file or directory Thread 10 failed open /sys/kernel/debug/block/(null)/trace10: 2/No such file or directory Thread 13 failed open /sys/kernel/debug/block/(null)/trace13: 2/No such file or directory Thread 15 failed open /sys/kernel/debug/block/(null)/trace15: 2/No such file or directory Thread 14 failed open /sys/kernel/debug/block/(null)/trace14: 2/No such file or directory Thread 17 failed open /sys/kernel/debug/block/(null)/trace17: 2/No such file or directory Thread 16 failed open /sys/kernel/debug/block/(null)/trace16: 2/No such file or directory Thread 18 failed open /sys/kernel/debug/block/(null)/trace18: 2/No such file or directory Thread 11 failed open /sys/kernel/debug/block/(null)/trace11: 2/No such file or directory Thread 19 failed open /sys/kernel/debug/block/(null)/trace19: 2/No such file or directory Thread 20 failed open /sys/kernel/debug/block/(null)/trace20: 2/No such file or directory Thread 9 failed open /sys/kernel/debug/block/(null)/trace9: 2/No such file or directory Thread 21 failed open /sys/kernel/debug/block/(null)/trace21: 2/No such file or directory Thread 22 failed open /sys/kernel/debug/block/(null)/trace22: 2/No such file or directory Thread 23 failed open /sys/kernel/debug/block/(null)/trace23: 2/No such file or directory FAILED to start thread on CPU 0: 1/Operation not permitted FAILED to start thread on CPU 1: 1/Operation not permitted FAILED to start thread on CPU 2: 1/Operation not permitted FAILED to start thread on CPU 3: 1/Operation not permitted FAILED to start thread on CPU 4: 1/Operation not permitted FAILED to start thread on CPU 5: 1/Operation not permitted FAILED to start thread on CPU 6: 1/Operation not permitted FAILED to start thread on CPU 7: 1/Operation not permitted FAILED to start thread on CPU 8: 1/Operation not permitted FAILED to start thread on CPU 9: 1/Operation not permitted FAILED to start thread on CPU 10: 1/Operation not permitted FAILED to start thread on CPU 11: 1/Operation not permitted FAILED to start thread on CPU 12: 1/Operation not permitted FAILED to start thread on CPU 13: 1/Operation not permitted FAILED to start thread on CPU 14: 1/Operation not permitted FAILED to start thread on CPU 15: 1/Operation not permitted FAILED to start thread on CPU 16: 1/Operation not permitted FAILED to start thread on CPU 17: 1/Operation not permitted FAILED to start thread on CPU 18: 1/Operation not permitted FAILED to start thread on CPU 19: 1/Operation not permitted FAILED to start thread on CPU 20: 1/Operation not permitted FAILED to start thread on CPU 21: 1/Operation not permitted FAILED to start thread on CPU 22: 1/Operation not permitted FAILED to start thread on CPU 23: 1/Operation not permitted
我怎么解决这个问题?
UPDATE1:
在/sys/kernel/debug/block/
有sda
文件夹,并在这个文件夹中有这些文件:
trace0 trace1 trace2 etc.
UPDATE2:
@abligh谢谢你的回答,但没有帮助。 关于ioctl
的strace
输出是:
ioctl(3, BLKTRACESETUP, {act_mask=64, buf_size=524288, buf_nr=4, start_lba=0, end_lba=0, pid=0}, 0x7ffe8a4ceac0) = -1 ENOTTY (Inappropriate ioctl for device) write(2, "BLKTRACESETUP(2) / failed: 25/Inappropriate ioctl for device\n", 61) = 61 ioctl(3, BLKTRACESTOP, 0x7f6fd19789d0) = -1 ENOTTY (Inappropriate ioctl for device) ioctl(3, BLKTRACESTOP, 0x7ffe8a4ce540) = -1 ENOTTY (Inappropriate ioctl for device) ioctl(3, BLKTRACESTOP, 0x7f6fd19789e0) = -1 ENOTTY (Inappropriate ioctl for device)` ioctl(3, BLKTRACETEARDOWN, 0x7f6fd19789e0) = -1 ENOTTY (Inappropriate ioctl for device)
回答你为什么在容器中运行blktrace
问题,我会说我使用容器作为集群,所以我需要每个节点的跟踪。
我没有足够的信息来debugging,但是你要求发布一个答案。
问题的根源似乎在这里:
BLKTRACESETUP(2) / failed: 25/Inappropriate ioctl for device
这表示ioctl
调用设置块跟踪失败。 它在容器内部是失败的,但根据容器外部的评论作品。 这将表明问题出在容器设置上,或者是内核中的一个限制,使得ioctl
根本无法在容器中使用。
inappropriate ioctl for device
错误是errno 25,即ENOTTY
。 我不能立即看到什么将返回,除非找不到设备节点(因为你已经certificate了它的工作原理,它在块容器跟踪代码编译的容器之外)。 我不记得这是否在一个模块,但它是值得首先尝试在容器外部的块跟踪(然后检查它在容器内),只是为了检查这不是一个模块加载问题。
debugging这个的第一步就是使用strace
工具(如上面所build议的),这样你就可以确切地知道哪个系统调用正在使用什么参数。 EG运行:
sudo strace -f -s2048 -o/tmp/trace blktrace -d / -a issue -o - | blkparse -f "%p %T.%9t %D %S ^C %d\n" -i - >stream.out
然后看看/tmp/trace
。 strace
将列出所有系统调用。 看看你能否确定哪个ioctl
失败。
其次,我会确保你试图跟踪的块设备实际上出现在/proc/
和/sys/
的正确位置。 这里发生了一些错误:
Thread 0 failed open /sys/kernel/debug/block/(null)/trace0: 2/No such file or directory
注意debugging行中的(null)
,显然不应该存在 – 应该是块设备的名称。 这可能是失败的ioctl
的后果,或者可能表示/sys/
层次结构中的问题。
BLKTRACESETUP
在这里的内核中处理。 这最终在这里调用doblktracesetup
。 我不能立即看到任何理由,为什么这不会从一个适当的权限的容器,这使我怀疑你的/sys
层次结构可能没有设置正确的工作。 但strace
的输出将会有所帮助。
另外,不可避免的评论:你为什么在一个容器中做这个? 为什么不在容器外运行?
编辑 :看起来这可能是一个内核的东西。 见https://github.com/scaleway/kernel-tools/issues/107 – 这表明你(a)需要首先修改特定的模块,(b)可能需要一个特定的内核。