Linux 调试

发表于 | 分类于 | 阅读数

介绍Linux 调试内核，调试应用程序

基本命令

  • man gdb 查看命令帮助信息，q退出
  • r 运行调试
  • n 下一步
  • c 继续运行
  • ctrl+c 中断后续运行
  • s 进入一个函数
  • finish 退出函数
  • l 列出代码行 比如 l main l xxx.go:1
  • b 断点
  • info b 显示断点列表
  • delete number 删除断点
  • clear 清除断点
  • until 2 跳转至行号
  • p 打印 p/x 按十六进制显示变量
  • bt/backtrace <-n> 显示所有栈帧，栈 -n 表一个负整数，表示只打印栈底下n层的栈信息
  • bt full 显示栈帧以及局部变量
  • info frame <栈帧编号> 查看帧详细信息
  • f/frame <number> 进入指定堆栈层
  • thread apply all bt 显示线程所以堆栈
  • attach <-p pid> 绑定进程调试
  • detach 取消绑定调试
  • disassemble <func> 查看二进制数据
  • x 查看内存
  • focus 显示源码界面
  • display *entry 显示变量
  • display <num> 取消显示
  • info registers 查看寄存器
  • gdb -p xxx 调试根据进程号
  • ctrl + c中断程序
  • ptype xxx 查看对象类型
  • disable <断点编号> 当前断点设置为无效
  • enable <断点编号> 当前断点设置为有效
  • x $pc 显示程序指针指向位置的内容
  • x/i $pc 显示程序当前位置的汇编指令
  • x/10i $pc 显示程序当前位置开始往后的10条汇编指令
  • disassem $pc 反汇编当前函数。简写为：disas $pc
  • gcore 生产dump文件

Linux内核调试

虚拟机内核编译调试

  1. 修改内核源码增加系统调用
  2. 编译，修改启动项目进入新内核
  3. 内核运行在qemu，gdb调试，断点可以打在系统调用入口处
    写个小程序编译的时候加-g 表示加入调试信息。

Redis调试

安装：

1
2
3
4
wget http://download.redis.io/releases/redis-6.2.3.tar.gz # 下载
tar xzf redis-6.2.3.tar.gz # 解压
cd redis-6.2.3
vi src/Makefile

修改Makefile内容：

1
2
3
4
5
# 关闭编译优化
# OPTIMIZATION?=-O2
OPTIMIZATION?=-O0
# REDIS_LD=$(QUIET_LINK)$(CC) $(FINAL_LDFLAGS)
REDIS_LD=$(QUIET_LINK)$(CC) $(FINAL_LDFLAGS) $(OPTIMIZATION)

编译

1
make

调试过程

1
2
3
4
5
gdb --args ./src/redis-server redis.conf # 启动调试
r # 运行
ctrl+c # 中断程序
b dict.c：dictAdd  # 添加断点
c # 继续运行
1
redis-cli set abc abc # 开启一个新的Shell终端进行操作，启动client
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
[root@test redis-6.2.3]# gdb --args ./src/redis-server redis.conf
GNU gdb (GDB) Red Hat Enterprise Linux (7.2-60.el6_4.1)
Copyright (C) 2010 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-redhat-linux-gnu".
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>...
Reading symbols from /root/redis/redis-6.2.3/src/redis-server...done.
(gdb) b dict.c:dictAdd
Breakpoint 1 at 0x4352ed: file dict.c, line 290.
(gdb) r
Starting program: /root/redis/redis-6.2.3/src/redis-server redis.conf
[Thread debugging using libthread_db enabled]

Breakpoint 1, dictAdd (d=0x7ffff7a0b060, key=0x7ffff7a09011, val=0x7e0920) at dict.c:290
290	    dictEntry *entry = dictAddRaw(d,key,NULL);
Missing separate debuginfos, use: debuginfo-install glibc-2.12-1.132.el6.x86_64
(gdb) p *entry
Cannot access memory at address 0x6
(gdb) f 0  # 可以f 1 f 2 等进入不同栈帧
#0  dictAdd (d=0x7ffff7a0b060, key=0x7ffff7a09011, val=0x7e0920) at dict.c:290
290	    dictEntry *entry = dictAddRaw(d,key,NULL);
(gdb) n # 下一步
292	    if (!entry) return DICT_ERR;
(gdb) p *entry
$1 = {key = 0x7ffff7a09011, v = {val = 0x0, u64 = 0, s64 = 0, d = 0}, next = 0x0}
(gdb) ptype *entry
type = struct dictEntry	{
    void *key;
    union {
       	void *val;
       	uint64_t u64;
       	int64_t s64;
       	double d;
    } v;
    struct dictEntry *next;
}
(gdb) bt
#0  dictAdd (d=0x7ffff7a0b060, key=0x7ffff7a09011, val=0x7e0920) at dict.c:292
#1  0x000000000043c5bc in populateCommandTable () at server.c:3446
#2  0x000000000043a90e in initServerConfig () at server.c:2735
#3  0x00000000004432c6 in main (argc=2,	argv=0x7fffffffe2d8) at server.c:6216
(gdb) info registers
rax            0x7ffff7a12030	140737347919920
rbx            0x5c2373821092b	1620917304232235
rcx            0xf0	240
rdx            0x7ffff7a09011	140737347883025
rsi            0x2	2
rdi            0x7fffc0000000	140736414613504
rbp            0x7fffffffe120	0x7fffffffe120
rsp            0x7fffffffe0f0	0x7fffffffe0f0
r8             0x20	32
r9             0x0	0
r10            0x0	0
r11            0x3588683d9e	229921799582
r12            0x42e680	4384384
r13            0x7fffffffe2d0	140737488347856
r14            0x0	0
r15            0x0	0
rip            0x435309	0x435309 <dictAdd+48>
eflags         0x206	[ PF IF ]
cs             0x33	51
ss             0x2b	43
ds             0x0	0
es             0x0	0
fs             0x0	0
gs             0x0	0
(gdb) x/20x $rsp
0x7fffffffe0f0:	0xffffe120	0x00007fff	0x007e0920	0x00000000
0x7fffffffe100:	0xf7a09011	0x00007fff	0xf7a0b060	0x00007fff
0x7fffffffe110:	0x3821092b	0x0005c237	0xf7a12030	0x00007fff
0x7fffffffe120:	0xffffe150	0x00007fff	0x0043c5bc	0x00000000
0x7fffffffe130:	0x00000000	0x00000000	0x00000000	0x000000e0
1
focus # 进入源码调试窗口
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
dict.cqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqk
   x280      * This function is called by common lookup or update operations in the                  x
   x281      * dictionary so that the hash table automatically migrates from H1 to H2                x
   x282      * while it is actively used. */                                                         x
   x283     static void _dictRehashStep(dict *d) {                                                   x
   x284         if (d->pauserehash == 0) dictRehash(d,1);                                            x
   x285     }                                                                                        x
   x286                                                                                              x
   x287     /* Add an element to the target hash table */                                            x
   x288     int dictAdd(dict *d, void *key, void *val)                                               x
   x289     {                                                                                        x
B+ x290         dictEntry *entry = dictAddRaw(d,key,NULL);                                           x
   x291                                                                                              x
  >x292         if (!entry) return DICT_ERR;                                                         x
   x293         dictSetVal(d, entry, val);                                                           x
   x294         return DICT_OK;                                                                      x
   x295     }                                                                                        x
   x296                                                                                              x
   x297     /* Low level add or find:                                                                x
   x298      * This function adds the entry but instead of setting a value returns the               x
   x299      * dictEntry structure to the user, that will make sure to fill the value                x
   x300      * field as they wish.                                                                   x
   x301      *

后面不是同一次操作，内存地址可能对应不上。
layout regs 打开寄存器窗口

1
2
3
4
5
6
7
8
9
10
11
lqqRegister group: generalqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqk
xrax            0x7ffff7a0b420   140737347892256   rbx            0x7ffff7a8a000   140737348411392   x
xrcx            0x7ffff7a099e1   140737347885537   rdx            0x7ffff7a4ce48   140737348161096   x
xrsi            0x7ffff7a099e1   140737347885537   rdi            0x7ffff7a0b420   140737347892256   x
xrbp            0x7fffffffdd10   0x7fffffffdd10    rsp            0x7fffffffdce0   0x7fffffffdce0    x
xr8             0x0      0                         r9             0x0      0                         x
xr10            0x358872c130     229922488624      r11            0x246    582                       x
xr12            0x0      0                         r13            0x7fffffffe2d0   140737488347856   x
xr14            0x0      0                         r15            0x0      0                         x
xrip            0x4352ed 0x4352ed <dictAdd+20>     eflags         0x202    [ IF ]                    x
xcs             0x33     51                        ss             0x2b     43

layout asm 打开汇编窗口

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
B+>x0x4352ed <dictAdd+20>   mov    -0x20(%rbp),%rcx                                                  x
   x0x4352f1 <dictAdd+24>   mov    -0x18(%rbp),%rax                                                  x
   x0x4352f5 <dictAdd+28>   mov    $0x0,%edx                                                         x
   x0x4352fa <dictAdd+33>   mov    %rcx,%rsi                                                         x
   x0x4352fd <dictAdd+36>   mov    %rax,%rdi                                                         x
   x0x435300 <dictAdd+39>   callq  0x435363 <dictAddRaw>                                             x
   x0x435305 <dictAdd+44>   mov    %rax,-0x8(%rbp)                                                   x
   x0x435309 <dictAdd+48>   cmpq   $0x0,-0x8(%rbp)                                                   x
   x0x43530e <dictAdd+53>   jne    0x435317 <dictAdd+62>                                             x
   x0x435310 <dictAdd+55>   mov    $0x1,%eax                                                         x
   x0x435315 <dictAdd+60>   jmp    0x435361 <dictAdd+136>                                            x
   x0x435317 <dictAdd+62>   mov    -0x18(%rbp),%rax                                                  x
   x0x43531b <dictAdd+66>   mov    (%rax),%rax                                                       x
   x0x43531e <dictAdd+69>   mov    0x10(%rax),%rax                                                   x
   x0x435322 <dictAdd+73>   test   %rax,%rax                                                         x
   x0x435325 <dictAdd+76>   je     0x435350 <dictAdd+119>                                            x
   x0x435327 <dictAdd+78>   mov    -0x18(%rbp),%rax                                                  x
   x0x43532b <dictAdd+82>   mov    (%rax),%rax                                                       x
   x0x43532e <dictAdd+85>   mov    0x10(%rax),%rcx                                                   x
   x0x435332 <dictAdd+89>   mov    -0x18(%rbp),%rax                                                  x
   x0x435336 <dictAdd+93>   mov    0x8(%rax),%rax                                                    x
   x0x43533a <dictAdd+97>   mov    -0x28(%rbp),%rdx

layout src 源码窗口

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
   lqqdict.cqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqk
B+ x290         dictEntry *entry = dictAddRaw(d,key,NULL);                                           x
   x291                                                                                              x
   x292         if (!entry) return DICT_ERR;                                                         x
   x293         dictSetVal(d, entry, val);                                                           x
   x294         return DICT_OK;                                                                      x
   x295     }                                                                                        x
   x296                                                                                              x
   x297     /* Low level add or find:                                                                x
   x298      * This function adds the entry but instead of setting a value returns the               x
   x299      * dictEntry structure to the user, that will make sure to fill the value                x
   x300      * field as they wish.                                                                   x
   x301      *                                                                                       x
   x302      * This function is also directly exposed to the user API to be called                   x
   x303      * mainly in order to store non-pointers inside the hash value, example:                 x
   x304      *                                                                                       x
   x305      * entry = dictAddRaw(dict,mykey,NULL);                                                  x
   x306      * if (entry != NULL) dictSetSignedIntegerVal(entry,1000);                               x
   x307      *                                                                                       x
   x308      * Return values:                                                                        x
   x309      *                                                                                       x
   x310      * If key already exists NULL is returned, and "*existing" is populated                  x
   x311      * with the existing entry if existing is not NULL.

ctrl+x+a 关闭窗口

java调试

  • 普通javadebug配置java -agentlib:jdwp=transport=dt_socket,server=y,suspend=n,address=5555 -jar ~/test.jar
  • tomcat开启debug模式./catalina.sh jpda start;默认端口8000
  • java使用JDB命令。
  • 普通java程序jdb -connect com.sun.jdi.SocketAttach:hostname=127.0.0.1,port=5555
  • tomcat部署方式下jdb连接debug端口jdb -attach localhost:8000
  • 基本命令stop at com.xx.xxx.abc:10 clear print x next