Tombstone日志的生成

Android默认是不会抓取coredump文件的，AOSP在进程发生内存访问异常的时候一般会在 data/tombstones/ 下面生成“tombstone_0x”形式命名的文件，这个文件是debuggerd进程来抓取的，在可执行文件被加载的时候会执行到bionic/linker/debugger.cpp这个文件的debuggerd_init函数,在这个函数里面注册了信号处理函数为debuggerd_signal_handler，所以当这个进程接收到SIGABRT、SIGBUS、SIGSEGV等signal的时候kernel都会回调debuggerd_signal_handler这个函数来处理.

__LIBC_HIDDEN__ void debuggerd_init() {
  struct sigaction action;
  memset(&action, 0, sizeof(action));
  sigemptyset(&action.sa_mask);
  action.sa_sigaction = debuggerd_signal_handler;
  action.sa_flags = SA_RESTART | SA_SIGINFO;

  // Use the alternate signal stack if available so we can catch stack overflows.
  action.sa_flags |= SA_ONSTACK;

  sigaction(SIGABRT, &action, nullptr);
  sigaction(SIGBUS, &action, nullptr);
  sigaction(SIGFPE, &action, nullptr);
  sigaction(SIGILL, &action, nullptr);
  sigaction(SIGSEGV, &action, nullptr);
#if defined(SIGSTKFLT)
  sigaction(SIGSTKFLT, &action, nullptr);
#endif
  sigaction(SIGTRAP, &action, nullptr);
}

debuggerd_signal_handler的实现如下，它的作用就是给守护进程发送一个socket请求，由后者通过ptrace调用来获取出错进程的出错地址，出错信号，线程堆栈、/proc/*/maps等信息，也就是上面所讲述的tombstone文件，zygote进程的app_process可执行文件在第一次被加载的时候也会执行上面的debuggerd_init逻辑，得益于Linux的fock调用，所有的zygote的子进程都有这个信号处理过程，不需要再另外添加代码.

static void debuggerd_signal_handler(int signal_number, siginfo_t* info, void*) {
  // It's possible somebody cleared the SA_SIGINFO flag, which would mean
  // our "info" arg holds an undefined value.
  if (!have_siginfo(signal_number)) {
    info = nullptr;
  }

  log_signal_summary(signal_number, info);
  {
  send_debuggerd_packet(info);

  // We need to return from the signal handler so that debuggerd can dump the
  // thread that crashed, but returning here does not guarantee that the signal
  // will be thrown again, even for SIGSEGV and friends, since the signal could
  // have been sent manually. Resend the signal with rt_tgsigqueueinfo(2) to
  // preserve the SA_SIGINFO contents.
  signal(signal_number, SIG_DFL);

  struct siginfo si;
  if (!info) {
    memset(&si, 0, sizeof(si));
    si.si_code = SI_USER;
    si.si_pid = getpid();
    si.si_uid = getuid();
    info = &si;
  } else if (info->si_code >= 0 || info->si_code == SI_TKILL) {
    // rt_tgsigqueueinfo(2)'s documentation appears to be incorrect on kernels
    // that contain commit 66dd34a (3.9+). The manpage claims to only allow
    // negative si_code values that are not SI_TKILL, but 66dd34a changed the
    // check to allow all si_code values in calls coming from inside the house.
  }

  int rc = syscall(SYS_rt_tgsigqueueinfo, getpid(), gettid(), signal_number, info);
  if (rc != 0) {
    __libc_format_log(ANDROID_LOG_FATAL, "libc", "failed to resend signal during crash: %s",
                      strerror(errno));
    _exit(0);
  }
}

Tombstone日志

pid: 27730, tid: 27736, name: thermalloadalgo  >>> /vendor/bin/thermalloadalgod <<<
signal 6 (SIGABRT), code --------, fault addr --------
      r0 00000000  r1 00006c52  r2 00000006  r3 00000008
      r4 f0ce558c  r5 00000006  r6 f0ce5534  r7 0000010c
      r8 05f481e2  r9 f0bfca90  sl 00000000  fp ffea81cc
      ip 00000002  sp ffea7f70  lr f0896507  pc f0898d64  cpsr 20070010

backtrace:
    #00 pc 0004ad64  /system/lib/libc.so (tgkill+12)
    #01 pc 00048503  /system/lib/libc.so (pthread_kill+34)
    #02 pc 0001d4a5  /system/lib/libc.so (raise+10)
    #03 pc 00018fe1  /system/lib/libc.so (__libc_android_abort+34)
    #04 pc 00017044  /system/lib/libc.so (abort+4)
    #05 pc 0001b49f  /system/lib/libc.so (__libc_fatal+22)
    #06 pc 0001b47f  /system/lib/libc.so (__fortify_chk_fail+26)
    #07 pc 00050cbb  /system/lib/libc.so (__sprintf_chk+66)
    #08 pc 000039fb  /system/vendor/lib/libthermalalgo.so (_Z12get_uid_tputi+54)
    #09 pc 00003b3b  /system/vendor/lib/libthermalalgo.so (_Z18update_avg_fg_tputv+14)
    #10 pc 00002df1  /system/vendor/lib/libthermalalgo.so (_Z3spav+324)
    #11 pc 000020ed  /system/vendor/lib/libthermalalgo.so (libthermal_algo_setup+244)
    #12 pc 000008df  /system/vendor/bin/thermalloadalgod
    #13 pc 00016c5d  /system/lib/libc.so (__libc_init+48)
    #14 pc 00000770  /system/vendor/bin/thermalloadalgod

stack:
         ffea7f30  f0bfd114  /system/vendor/lib/libthermalalgo.so
         ffea7f34  00000000
         ffea7f38  ffea81cc  [stack]
         ffea7f3c  f0893dd3  /system/lib/libc.so (vsnprintf+138)
         ffea7f40  ffffffff
         ffea7f44  898045b6
         ffea7f48  00000000
         ffea7f4c  00000000
         ffea7f50  00000000
         ffea7f54  00000000
         ffea7f58  00000000
         ffea7f5c  00000000
         ffea7f60  ffffffff
         ffea7f64  00004000
         ffea7f68  f0ce558c
         ffea7f6c  f08964f7  /system/lib/libc.so (pthread_kill+22)
    #00  ffea7f70  00000000
         ........  ........
    #01  ffea7f70  00000000
         ffea7f74  00000006
         ffea7f78  00000000
         ffea7f7c  f0bff168  /system/vendor/lib/libthermalalgo.so
         ffea7f80  f0bff160  /system/vendor/lib/libthermalalgo.so
         ffea7f84  f086b4a9  /system/lib/libc.so (raise+14)
......

debuggerd守护进程抓取的tombstone文件内容大概如上所示，这个Log具体说明了如下信息：

• 出错的进程名字为/vendor/bin/thermalloadalgod，它的pid是27730，而具体导致接收到这次信号的tid也为27736.
• 进程收到的是SIGABRT信号.
• 抓取这个日志的时候，每个寄存器的内容,分析问题的时候一般需要结合反汇编代码和这里的寄存器内容分析.
• 这个线程的函数栈帧信息，也就是backtrace，从中我们可以看到这个线程的函数调用流程,因为tombstone文件一般是在终端抓取的，由于终端的可执行文件和so文件并没有保存有可供调试的符号信息，所以只能看到是哪个函数，但是对应不到对应的代码行，如果有对应的符号信息，可以用addr2line等工具把这里的地址信息对应的代码行，建议用add2line的时候 加上 “-ie”参数，可以把inline函数也打印出来.
• 这个线程对应的函数栈的内容，函数栈用来保存局部变量、返回地址、SP寄存器等内容.

常见信号

• SIGABRT

在Android里面，这个信号一般是进程自己检测到不可恢复的错误之后，主动调用abort函数之后会接收到的一种信号，一般在log里面会打印出调用abort的原因，这种问题相对来说可能容易解决一些，只要根据错误提示信息，就可以去找相应的原因.

• SIGSEGV

在POSIX兼容的平台上，SIGSEGV是当一个进程执行了一个无效的内存引用，或发生段错误时发送给它的信号,例如访问没有实际物理内存对应的虚拟地址，或者访问的时候出现权限异常,比较常见的就是空指针,对于SIGSEGV这一类信号，在tombstone log里面会给出fault addr，根据汇编代码和寄存器内容，可以接着往下分析.

• SIGBUS

SIGBUS与SIGSEGV有点类似，也都是访问内存的时候出错，他们之间的区别可能在于如果那个地址是正常的（也就是说正常的虚拟地址，而且它也有对应的物理内存），如果访问出错，那么就会上报SIGBUS，否则就上报的是SIGSEGV，例如有些芯片总线访问需要地址是4字节对齐的，如果那个地址不是对齐的，那么就会上报SIGBUS.

这三种信号在Android稳定性问题分析当中是比较常见的，除了这三种信号之外，其实Linux有64种信号，一般来说前面的32种是标准信号，后面32种是新增加的实时信号，对于标准信号，一个进程里面同一时间只能存在一个，而实时信号却没有这个限制，可以存在多个，这里借用 Unix系统中常用的信号含义 这个文章的内容对标准信号做一些简单的介绍

objdump工具

tombstone文件抓取的内容比较少，没有coredump文件那么全，分析这种Log的时候，不可缺少的要用到objdump这类反编译工具，objdump能反编译ELF文件，将它的代码段反汇编出来，结合tombstone文件里面的fault addr地址信息、寄存器信息、函数栈帧信息等来分析出问题的原因.

Usage: aarch64-linux-android-objdump <option(s)> <file(s)>
 Display information from object <file(s)>.
 At least one of the following switches must be given:
  -a, --archive-headers    Display archive header information
  -f, --file-headers       Display the contents of the overall file header
  -p, --private-headers    Display object format specific file header contents
  -P, --private=OPT,OPT... Display object format specific contents
  -h, --[section-]headers  Display the contents of the section headers
  -x, --all-headers        Display the contents of all headers
  -d, --disassemble        Display assembler contents of executable sections
  -D, --disassemble-all    Display assembler contents of all sections
  -S, --source             Intermix source code with disassembly
  -s, --full-contents      Display the full contents of all sections requested
  -g, --debugging          Display debug information in object file
  -e, --debugging-tags     Display debug information using ctags style
  -G, --stabs              Display (in raw form) any STABS info in the file
  -W[lLiaprmfFsoRt] or
  --dwarf[=rawline,=decodedline,=info,=abbrev,=pubnames,=aranges,=macro,=frames,
          =frames-interp,=str,=loc,=Ranges,=pubtypes,
          =gdb_index,=trace_info,=trace_abbrev,=trace_aranges,
          =addr,=cu_index]
                           Display DWARF info in the file
  -t, --syms               Display the contents of the symbol table(s)
  -T, --dynamic-syms       Display the contents of the dynamic symbol table
  -r, --reloc              Display the relocation entries in the file
  -R, --dynamic-reloc      Display the dynamic relocation entries in the file
  @<file>                  Read options from <file>
  -v, --version            Display this program's version number
  -i, --info               List object formats and architectures supported
  -H, --help               Display this information

 The following switches are optional:
  -b, --target=BFDNAME           Specify the target object format as BFDNAME
  -m, --architecture=MACHINE     Specify the target architecture as MACHINE
  -j, --section=NAME             Only display information for section NAME
  -M, --disassembler-options=OPT Pass text OPT on to the disassembler
  -EB --endian=big               Assume big endian format when disassembling
  -EL --endian=little            Assume little endian format when disassembling
      --file-start-context       Include context from start of file (with -S)
  -I, --include=DIR              Add DIR to search list for source files
  -l, --line-numbers             Include line numbers and filenames in output
  -F, --file-offsets             Include file offsets when displaying information
  -C, --demangle[=STYLE]         Decode mangled/processed symbol names
                                  The STYLE, if specified, can be `auto', `gnu',
                                  `lucid', `arm', `hp', `edg', `gnu-v3', `java'
                                  or `gnat'
  -w, --wide                     Format output for more than 80 columns
  -z, --disassemble-zeroes       Do not skip blocks of zeroes when disassembling
      --start-address=ADDR       Only process data whose address is >= ADDR
      --stop-address=ADDR        Only process data whose address is <= ADDR
      --prefix-addresses         Print complete address alongside disassembly
      --[no-]show-raw-insn       Display hex alongside symbolic disassembly
      --insn-width=WIDTH         Display WIDTH bytes on a single line for -d
      --adjust-vma=OFFSET        Add OFFSET to all displayed section addresses
      --special-syms             Include special symbols in symbol dumps
      --prefix=PREFIX            Add PREFIX to absolute paths for -S
      --prefix-strip=LEVEL       Strip initial directory names for -S
      --dwarf-depth=N        Do not display DIEs at depth N or greater
      --dwarf-start=N        Display DIEs starting with N, at the same depth
                             or deeper
      --dwarf-check          Make additional dwarf internal consistency checks.      

aarch64-linux-android-objdump: supported targets: elf64-littleaarch64 elf64-bigaarch64 elf32-littleaarch64 elf32-bigaarch64 elf32-littlearm elf32-bigarm elf64-little elf64-big elf32-little elf32-big plugin srec symbolsrec verilog tekhex binary ihex
aarch64-linux-android-objdump: supported architectures: aarch64 aarch64:ilp32 arm armv2 armv2a armv3 armv3m armv4 armv4t armv5 armv5t armv5te xscale ep9312 iwmmxt iwmmxt2 plugin

The following AARCH64 specific disassembler options are supported for use
with the -M switch (multiple options should be separated by commas):

  no-aliases         Don't print instruction aliases.

  aliases            Do print instruction aliases.


The following ARM specific disassembler options are supported for use with
the -M switch:
  reg-names-special-atpcs  Select special register names used in the ATPCS
  reg-names-atpcs          Select register names used in the ATPCS
  reg-names-apcs           Select register names used in the APCS
  reg-names-std            Select register names used in ARM's ISA documentation
  reg-names-gcc            Select register names used by GCC
  reg-names-raw            Select raw register names
  force-thumb              Assume all insns are Thumb insns
  no-force-thumb           Examine preceding label to determine an insn's type