Posted on   |   In   |  

The 3rd apk is hung up 11s while playback.

The direct appearance is ANR: Input event dispatching timed out sending. This ANR is caused by mediaplayer process.

Firstly, we use top to watch CPU loading, but find that CPU loading is not high when playback.

Secondly, check /proc/interrupts, we cannot find any interrupt abnormal.

This is a customer on-site issue, we only digg the log. We add some debug logs, and find that sometimes the interval printing two logs is 11s,it’s weird.

We doubt that system scheduling maybe occur anomaly. So capture ftrace data.

  1. start ftrace

    1
    2
    3
    4
    5
    6
    7
    8
    mount -t debugfs nodev /sys/kernel/debug
    echo nop > /sys/kernel/debug/tracing/current_tracer
    echo 0 > /sys/kernel/debug/tracing/tracing_on
    echo > /sys/kernel/debug/tracing/trace
    echo "sched_switch sched_wakeup sched_wakeup_new sched_migrate_task irq timer" > /sys/kernel/debug/tracing/set_event
    echo "workqueue_execute_start workqueue_execute_end block_rq_issue block_rq_insert block_rq_complete" >> /sys/kernel/debug/tracing/set_event
    echo 20480 > /sys/kernel/debug/tracing/buffer_size_kb
    echo 1 > /sys/kernel/debug/tracing/tracing_on

  2. stop ftrace (stop ftrace immediately once be reproduced)

    1
    2
    3
    echo 0 > /sys/kernel/debug/tracing/tracing_on
    cat /sys/kernel/debug/tracing/trace > /data/ftrace.log
    echo 0 > /sys/kernel/debug/tracing/trace

Notes: How to make ANR time printed by logcat to match Unix Timestamp. There is a very good tool: http://rimzy.net/tools/php_timestamp_converter.php

Go through digging ftrace data, we found a doubtful point: process X hasn’t scheduled about 11s.

  • use renice and tasklet to improve process priority and bind process X to specific CPU, it’s no improvement.
  • We found that process X sched_wakeup from idle process, it means process X is waked up by interrupt. The latest interrupt is uart interrupt.
    So we disable uart log or change uart baud rate to 921600, the ANR disappeared.

At last, we review n_tty_write() function.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
2019 static ssize_t n_tty_write(struct tty_struct *tty, struct file *file,
2020                            const unsigned char *buf, size_t nr)
2021 {
2022         const unsigned char *b = buf;
2023         DECLARE_WAITQUEUE(wait, current);
2024         int c;
2025         ssize_t retval = 0;
2026
2027         /* Job control check -- must be done at start (POSIX.1 7.1.1.4). */
2028         if (L_TOSTOP(tty) && file->f_op->write != redirected_tty_write) {
2029                 retval = tty_check_change(tty);
2030                 if (retval)
2031                         return retval;
2032         }
2033
2034         /* Write out any echoed characters that are still pending */
2035         process_echoes(tty);
2036
2037         add_wait_queue(&tty->write_wait, &wait);
2038         while (1) {
2039                 set_current_state(TASK_INTERRUPTIBLE);
2040                 if (signal_pending(current)) {
2041                         retval = -ERESTARTSYS;
2042                         break;
2043                 }
2044                 if (tty_hung_up_p(file) || (tty->link && !tty->link->count)) {
2045                         retval = -EIO;
2046                         break;
2047                 }
2048                 if (O_OPOST(tty)) {
2049                         while (nr > 0) {
2050                                 ssize_t num = process_output_block(tty, b, nr);
2051                                 if (num < 0) {
2052                                         if (num == -EAGAIN)
2053                                                 break;
2054                                         retval = num;
2055                                         goto break_out;
2056                                 }
2057                                 b += num;
2058                                 nr -= num;
2059                                 if (nr == 0)
2060                                         break;
2061                                 c = *b;
2062                                 if (process_output(c, tty) < 0)
2063                                         break;
2064                                 b++; nr--;
2065                         }
2066                         if (tty->ops->flush_chars)
2067                                 tty->ops->flush_chars(tty);
2068                 } else {
2069                         while (nr > 0) {
2070                                 c = tty->ops->write(tty, b, nr);
2071                                 if (c < 0) {
2072                                         retval = c;
2073                                         goto break_out;
2074                                 }
2075                                 if (!c)
2076                                         break;
2077                                 b += c;
2078                                 nr -= c;
2079                         }
2080                 }
2081                 if (!nr)
2082                         break;
2083                 if (file->f_flags & O_NONBLOCK) {
2084                         retval = -EAGAIN;
2085                         break;
2086                 }
2087                 schedule();
2088         }
2089 break_out:
2090         __set_current_state(TASK_RUNNING);
2091         remove_wait_queue(&tty->write_wait, &wait);
2092         if (b - buf != nr && tty->fasync)
2093                 set_bit(TTY_DO_WRITE_WAKEUP, &tty->flags);
2094         return (b - buf) ? b - buf : retval;
2095 }

Write data in while loop. When user space process uses blocking write method, if write fail, it will yield cpu and wake up untill the condition is met. In this ANR case, the root cause is that user space process instant log data is huge and block in n_tty_write().

Posted on   |   In   |  

Background

Kernel: linux3.10

发生概率性死机：

mm/memory.c:399: bad pmd 15141312
Segmentation fault
BUG: Bad rss-counter state mm:ce964380 idx:0 val:5
BUG: Bad rss-counter state mm:ce964380 idx:1 val:1

Read more »

Posted on   |   In   |  

Background

CPU: ARMv7

开机到kernel某个固定阶段发生死机，死机信息都是imprecise external abort.

Unhandled fault: imprecise external abort (0x1c06) at 0x7cab1234

Read more »

Posted on   |   In   |  

直接使用vim-as-golang-ide来配置非常简单，具体操作如下。
http://farazdagi.com/blog/2015/vim-as-golang-ide/
对应Github地址：https://github.com/farazdagi/vim-go-ide
然后执行vim -u ~/.vimrc.go即可，如果嫌麻烦，可以设置alias.

1
alias vimgo='vim -u ~/.vimrc.go'

vim-as-golang-ide实际上用到的仍然是vim-go. vim-as-golang-ide的好处时不破坏系统vim的设置。
vim-go: https://github.com/fatih/vim-go

执行完vim -u ~/.vimrc.go出现如下错误。

1
2
3
CSApprox skipped; terminal only has 8 colors, not 88/256
Try checking :help csapprox-terminal for workarounds
请按 ENTER 或其它命令继续

可在~/.vimrc.go中进行如下设置。

1
set t_Co=256

在进行:GoInstallBinaries之前需要临时设置$GOBIN环境变量，以便vim-go需要的binary放在/usr/local/go/bin下。

1
export GOBIN=$GOROOT/bin

Please be sure all necessary binaries are installed (such as gocode, godef, goimports, etc.). You can easily install them with the included :GoInstallBinaries command. If invoked, all necessary binaries will be automatically downloaded and installed to your $GOBIN environment (if not set it will use $GOPATH/bin). Note that this command requires git for fetching the individual Go packages. Additionally, use :GoUpdateBinaries to update the installed binaries.
– https://github.com/fatih/vim-go

vim-go依赖于很多其他binary，需自备梯子。
国内有个Go Package Manager: https://gopm.io/ 可以下载到被墙的binaries.

Posted on   |   In   |  

Install Golang

Golang安装参考https://golang.org/doc/install

相关环境变量

环境变量设置，将如下设置写到~/.profile文件中：

1
2
3
4
export GOROOT=/usr/local/go
export GOBIN=$GOROOT/bin
export GOPATH=$HOME/go
export PATH=$PATH:$GOPATH:$GOBIN

GOROOT

GOROOT是go的安装路径
package runtime中：
/src/runtime/extern.go:213

1
2
3
4
5
6
7
8
9
10
210	// GOROOT returns the root of the Go tree.
211	// It uses the GOROOT environment variable, if set,
212	// or else the root used during the Go build.
213	func GOROOT() string {
214		s := gogetenv("GOROOT")
215		if s != "" {
216			return s
217		}
218		return sys.DefaultGoroot
219	}

Read more »

Posted on   |   In   |  

hexo虽然可以方便地部署github静态博客，但是仅仅是把最终生成的html保存在repository中，像原始的Markdown文件，hexo配置文件，主题配置文件，修改文件都仅仅是保存在本地。这样不利于保存，也无法查看每篇博客的修改历史。更重要的是无法做到跨平台，也不易于多人写作。

想法是每次写博客，只需要push md文件及博客所需的资源文件即可。Travis CI持续集成tool可以满足此需求。

Read more »

Posted on   |   In   |  

当Jekyll教程在收藏夹中沉睡很久之后，发现时兴已经是Hexo了。
于是折腾了一把Hexo+Github搭建博客，总算落实夙愿了。

Install Git & Github配置

Install git

1
apt-get install git

然后配置username和email

1
2
git config --global user.name "xxx"
git config --global user.email "xxx@xxx.com"

Generate ssh-key

1
ssh-keygen -t rsa -C "xxx@xxx.com"

如果没有Github账户的话，则注册一个，将.ssh/id_rsa.pub中的内容复制到Github的Settings-> SSH Keys-> New SSH Key

1
ssh -T git@github.com

会提示

The authenticity of host ‘github.com (207.97.227.239)’ can’t be established.RSA key fingerprint is 16:27:ac:a5:76:28:2d:36:63:1b:56:4d:eb:df:a6:48.Are you sure you want to continue connecting (yes/no)?

输入yes就好，然后会提示：

Hi xxx! You’ve successfully authenticated, but GitHub does not provide shell access.

Create Repository

名字必须是GithubId.github.io

Read more »