AIX 플렛폼에서 WAS hang 현상 발생시 대처요령

 

제목 : AIX 플렛폼에서 WAS hang 현상 발생시 대처요령 글쓴이: 이원영(javaservice) 2003/02/24 08:39:21 조회수:2791 줄수:77

If Application Server Hang on AIX platform. To troubleshoot the hang, please collect the following doc during the time that the hang is occurring. login as "root", 1. netstat -an > netstat.txt (if webserver is remote, gather netstat output on Appserver machine AND webserver machine) 2. ps -ef > ps.txt 3. df -k > df.txt 4. lsof -p [PID of hung appserver] > lsof.txt (You need to install "lsof" utility) 5. Can you serve a static document from the webserver? 6. tprof -k -s -e -x sleep 60 7. ./dbxtrace_aix.sh -a [PID of hung appserver] > dbxtrace.txt (wait until finished) 8. WAIT TWO MINUTES 9. kill -3 [PID of hung appserver] 10. WAIT TWO MINUTES 11. kill -3 [PID of hung appserver] 12. WAIT TWO MINUTES 13. kill -3 [PID of hung appserver] The kill -3's should create javacore.txt files in the working directory of WAS, normally Websphere\Appserver\bin directory. Please send in the following files: 1. Everything in <was root>\logs directory (from the WebSphere machine) 2. If the webserver is remote, send the trace.log.<webserver name>.<date> (3.5.X), native.log (4.0.X), or http_plugin.log (5.0.X) file from the webserver machine. 3. dbxtrace.txt 4. All javacore?????.txt files created 5. netstat.txt, ps.txt, df.txt 6. Send in all files generated by tprof (especially _prof.all). These files will be in the directory from which the tprof command was run and will begin with "_". 7. Additionally, for WebSphere, (1) uname -a > uname.txt (2) lslpp -i |grep AIX_ML | sort > aix_sw.txt (3) errpt -a > errpt.txt (4) httpd.conf of HTTP Server (5) WAS_HOME/bin/admin.config (6) WAS_HOME/bin# ./XMLConfig.sh -adminNodeName <server> -export xmlconfig.xml (send "xmlconfig.xml") (7) WAS_HOME/properties/com/ibm/websphere/product.xml 8. If you think the problem caused by JDBC Connection Pool, send me all Java source programs you have. Appendix (사전 install) 1. lsof - LiSt Open Files http://www.javaservice.net/~java/bbs/read.cgi?m=unix&b=unix&c=r_p&n=959319048 2. dbxtrace.sh a. "dbx" 라는 프로그램이 설치되어 있는지 확인해 보십시요. 만약 설치되어 있지 않다면, 설치를 하도록 하십시요. 대부분 이미 설치되어 있을 겁니다. b. 아래에 첨부한 dbxtrace.sh 라는 shell script 를 다운받아 적당한 디렉토리에 넣어두고, 실행권한을 부여하십시요. # cp /tmp/dbxtrace.sh /usr/local/bin/ # chmod 755 /usr/local/bin/dbxtrace.sh dbxtrace.sh script download http://w3.hursley.ibm.com/java/communication/aix/service/AIX/Tools.html http://w3.hursley.ibm.com/java/communication/aix/service/AIX/dbxtrace c. root 유저에서 다음의 명령을 통해 system fullcore 옵션을 true 로 변경하시요. 옵션확인방법 : lsattr -El sys0 | grep fullcore 옵션변경방법 : chdev -a fullcore=true -l sys0 d. /usr/WebSphere/AppServer/bin/ 디렉토리에 있는 javacore* 및 core 파일은 (현재 개발중이시니) 더이상 필요없으며, 가급적 삭제하시고, 각종 로그들도 삭제하여 /usr 디스크의 free space 를 가급적 500MB 이상을 확보하십시요. dbxtrace.sh (5456 Bytes) lsof_aix_binary.zip (232832 Bytes) 출처 : 자바서비스넷 이원영(javaservice@hanmail.net)