システムのボトルネック発見の為に、参考になるサイトを確認していたらGDB,AWK,SORTなどの
普段の運用で利用するコマンドを利用してボトルネックを確認するシェルを作成している人がいたので
参考にさせていただきました。Profilerとして利用するのに良さそうです。
GDB
GDB, the GNU Project debugger, allows you to see what is going on `inside’ another
program while it executes — or what another program was doing at the moment it crashed.
#!/bin/bash nsamples=1 sleeptime=0 pid=$(pidof mysqld) for x in $(seq 1 $nsamples) do gdb -ex "set pagination 0" -ex "thread apply all bt" -batch -p $pid sleep $sleeptime done | \ awk ' BEGIN { s = ""; } /Thread/ { print s; s = ""; } /^\#/ { if (s != "" ) { s = s "," $4} else { s = $4 } } END { print s }' | \ sort | uniq -c | sort -r -n -k 1,1
GDBがインストールされて無かったのでインストール
[root@CentOS64VM tools]# yum install gdb
Loaded plugins: fastestmirror, presto
Loading mirror speeds from cached hostfile
* base: ftp.nara.wide.ad.jp
* extras: ftp.nara.wide.ad.jp
* updates: ftp.nara.wide.ad.jp
Setting up Install Process
Resolving Dependencies
–> Running transaction check
—> Package gdb.x86_64 0:7.2-56.el6 will be installed
–> Finished Dependency ResolutionDependencies Resolved
=====================================================================================
Package Arch Version Repository Size
=====================================================================================
Installing:
gdb x86_64 7.2-56.el6 base 2.3 MTransaction Summary
=====================================================================================
Install 1 Package(s)Total download size: 2.3 M
Installed size: 5.2 M
Is this ok [y/N]: y
Downloading Packages:
Setting up and reading Presto delta metadata
Processing delta metadata
Package(s) data still to download: 2.3 M[省略]
Trying other mirror.
gdb-7.2-56.el6.x86_64.rpm | 2.3 MB 00:13
Running rpm_check_debug
Running Transaction Test
Transaction Test Succeeded
Running Transaction
Warning: RPMDB altered outside of yum.
Installing : gdb-7.2-56.el6.x86_64 1/1
Verifying : gdb-7.2-56.el6.x86_64 1/1Installed:
gdb.x86_64 0:7.2-56.el6Complete!
[root@CentOS64VM tools]#
TPCC-MYSQLを実行して負荷をかけた状態でシステム状態を確認してみる。
[root@CentOS64VM tools]# ./poor_mans_profiler.sh
20
2 pthread_cond_wait@@GLIBC_2.3.2,os_cond_wait,reset_sig_count=2912),log_write_up_to,trx_commit_complete_for_mysql,innobase_commit,ha_commit_one_phase,he_command,mysql_parse,dispatch_command,do_handle_one_connection,handle_one_connection,pfs_spawn_thread,start_thread,clone
1 sigwait,signal_hand,pfs_spawn_thread,start_thread,clone
1 select,os_thread_sleep,srv_master_thread,start_thread,clone
1 pthread_cond_timedwait@@GLIBC_2.3.2,os_cond_wait_timed,time_in_usec=
こちらも合わせて確認:gdb -p `pidof mysqld`
How to debug lock (hang)
参考サイト
poor man’s profiler
漢(オトコ)のコンピュータ道
poormans-profiler
GDB: The GNU Project Debugger
ファイヤープロジェクト