OSDev.org
https://forum.osdev.org/

Apache hung
https://forum.osdev.org/viewtopic.php?f=6&t=24806
Page 1 of 1

Author:  chase [ Tue Feb 07, 2012 10:18 am ]
Post subject:  Apache hung

Looks like apache hung yesterday. The max num of httpd processes were running and non of them were responding. Looking at a stack track it seems that all of them got stuck in the apc module (php cache). I've changed some of the settings for apc, hopefully it won't happen again.

I've also setup a pingdom.com account so I should be notified within a couple of minutes of wiki.osdev.org going down.

Code:
# pstack 16655
#0  0xf57fe416 in __kernel_vsyscall ()
#1  0xb741c0d9 in __lll_lock_wait () from /lib/i686/nosegneg/libpthread.so.0
#2  0xb7417450 in _L_lock_677 () from /lib/i686/nosegneg/libpthread.so.0
#3  0xb7417321 in pthread_mutex_lock () from /lib/i686/nosegneg/libpthread.so.0
#4  0xb736aeb6 in pthread_mutex_lock () from /lib/i686/nosegneg/libc.so.6
#5  0xb68fedae in apc_pthreadmutex_lock () from /usr/lib/php/modules/apc.so
#6  0xb68f87a1 in apc_cache_user_delete () from /usr/lib/php/modules/apc.so
#7  0xb68f4ac4 in zif_apc_delete () from /usr/lib/php/modules/apc.so
#8  0xb7082169 in ?? () from /etc/httpd/modules/libphp5.so
#9  0xb7058b9e in execute () from /etc/httpd/modules/libphp5.so
#10 0xb7031216 in zend_execute_scripts () from /etc/httpd/modules/libphp5.so
#11 0xb6fd7f96 in php_execute_script () from /etc/httpd/modules/libphp5.so
#12 0xb70c06d3 in ?? () from /etc/httpd/modules/libphp5.so
#13 0xb7712561 in ap_run_handler ()
#14 0xb77162e6 in ap_invoke_handler ()
#15 0xb7722eb4 in ap_internal_redirect ()
#16 0xb71b0319 in ?? () from /etc/httpd/modules/mod_rewrite.so
#17 0xb7712561 in ap_run_handler ()
#18 0xb77162e6 in ap_invoke_handler ()
#19 0xb7723068 in ap_process_request ()
#20 0xb771fb58 in ?? ()
#21 0xb771add1 in ap_run_process_connection ()
#22 0xb77289aa in ?? ()
#23 0xb7728d3e in ?? ()
#24 0xb7729d13 in ap_mpm_run ()
#25 0xb76fcad2 in main ()

Author:  Brynet-Inc [ Tue Feb 07, 2012 11:07 am ]
Post subject:  Re: Apache hung

Time to switch to nginx!

Author:  chase [ Tue Feb 07, 2012 12:01 pm ]
Post subject:  Re: Apache hung

I was looking at apache alternatives (mainly http://www.lighttpd.net/) but just replacing all the mod_rewrite rules looked like such a time consuming PITA that I just wanted to get the site running on the new server with a setup as similar to the old one as I could. I'll probably have to spend some more serious time looking into this in the future.

If baiduspider and bingbot would stop hammering the site it'd probably help too.

Author:  gravaera [ Tue Feb 07, 2012 12:10 pm ]
Post subject:  Re: Apache hung

Down with bing, don't let them index your site, stop them now...do it for software freedom huehuehue

Author:  Brynet-Inc [ Tue Feb 07, 2012 12:12 pm ]
Post subject:  Re: Apache hung

nginx's equivalent to mod_rewrite looks slightly more comprehensible, some relevant info:

http://wiki.nginx.org/NginxHttpRewriteModule
http://www.nginx.org/en/docs/http/conve ... rules.html

Also handy, http://wiki.nginx.org/ModuleComparisonMatrix.

Author:  bubach [ Tue Feb 07, 2012 12:34 pm ]
Post subject:  Re: Apache hung

Hmm, I could serve like 2 million hits per day on one of my sites without any problems on Apache, guess my custom PHP was a bit more tweaked towards performance than phpBB and mediawiki but still... Should never have to worry about any issues on a site this size, standard server settings suck.

Author:  Jezze [ Wed Feb 08, 2012 6:44 am ]
Post subject:  Re: Apache hung

If you have performance problems you might look at adding a varnish layer.

https://www.varnish-cache.org/

Haven't tried it myself but it is suppose to be very good as a temporary fix

Author:  brain [ Wed Feb 08, 2012 7:49 am ]
Post subject:  Re: Apache hung

Jezze wrote:
If you have performance problems you might look at adding a varnish layer.

https://www.varnish-cache.org/

Haven't tried it myself but it is suppose to be very good as a temporary fix



I run a reasonably large and busy wiki too and ive found that using a reverse proxy does work wonders. We had all kinds of issues until we:

1) Installed reverse squid proxy
2) Moved seperated wiki and forums to two different virtul servers on the same box
3) Installed APC cache and memcached
4) Banned a lot of misbehaving spiders at the firewall and via useragent in apache, such as biadu and various spam harvesters

Page 1 of 1 All times are UTC - 6 hours
Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group
http://www.phpbb.com/