nginx freebsd 官方网站优化

OS & Hardware
FreeBSD 7.1
AMD64, dual-core CPU
4GB RAM
Purpose
Web server & reverse proxy
Load description
45K inactive keep-alive connections
HTTP request is about 5,000 req/s, mostly small static files, all are cached by VM
System config
/boot/loader.conf:

vm.kmem_size=1844M
kern.maxbcache=64M
kern.ipc.maxpipekva=4M

/etc/sysctl.conf:

kern.ipc.nmbjumbop=192000
kern.ipc.nmbclusters=229376
kern.ipc.maxsockets=204800
net.inet.tcp.maxtcptw=163840
kern.maxfiles=204800

kern.ipc.somaxconn=4096

Retrieved from “http://wiki.nginx.org/FreeBSDOptimizations”

freebsd下 nginx 日志每天分割

#!/bin/sh
date
today=`date +"%Y_%m_%d"`
logdir=/pub/log
daydir=/pub/log/$today
expdays=7
if [ ! -d $daydir ] ; then
 mkdir -p $daydir
fi

cp /pub/log/wwwlogs.log $daydir/wwwlogs.log

#清空当前日志
/usr/bin/true > /pub/log/wwwlogs.log
#删除7天前的过期文件
#find $daydir -type d -mtime +$expdays -maxdepth 1 | xargs rm -rf

#每天12点运行
#0      0       *       *       *       /cut_nginx_log.sh

nginx echsop rewrite规则

从网上搜索到的规则,但不能用,修改了一下,

rewrite ^/index.html$ /index.php;
		rewrite ^/category$ /index.php;
		rewrite ^/feed-c([0-9]+).xml$ /feed.php?cat=$1;
		rewrite ^/feed-b([0-9]+).xml$ /feed.php?brand=$1;
		rewrite ^/feed.xml$ /feed.php;
		rewrite ^/category-([0-9]+)-b([0-9]+)-min([0-9]+)-max([0-9]+)-attr([^-]*)-([0-9]+)-(.+)-([a-zA-Z]+)(.*).html$  /category.php?id=$1&brand=$2&price_min=$3&price_max=$4&filter_attr=$5&page=$6&sort=$7&order=$8;
		rewrite ^/category-([0-9]+)-b([0-9]+)-min([0-9]+)-max([0-9]+)-attr([^-]*)(.*).html$ /category.php?id=$1&brand=$2&price_min=$3&price_max=$4&filter_attr=$5;
		rewrite ^/category-([0-9]+)-b([0-9]+)-([0-9]+)-(.+)-([a-zA-Z]+)(.*).html$ /category.php?id=$1&brand=$2&page=$3&sort=$4&order=$5;
		rewrite ^/category-([0-9]+)-b([0-9]+)-([0-9]+)(.*).html$ /category.php?id=$1&brand=$2&page=$3;
		rewrite ^/category-([0-9]+)-b([0-9]+)(.*).html$  /category.php?id=$1&brand=$2;
		rewrite ^/category-([0-9]+)(.*).html$ /category.php?id=$1;
		rewrite ^/goods-([0-9]+)(.*).html$  /goods.php?id=$1;
		rewrite ^/article_cat-([0-9]+)-([0-9]+)-(.+)-([a-zA-Z]+)(.*).html$  /article_cat.php?id=$1&page=$2&sort=$3&order=$4;
		rewrite ^/article_cat-([0-9]+)-([0-9]+)(.*).html$ /article_cat.php?id=$1&page=$2;
		rewrite ^/article_cat-([0-9]+)(.*).html$ /article_cat.php?id=$1;
		rewrite ^/article-([0-9]+)(.*).html$ /article.php?id=$1;
		rewrite ^/brand-([0-9]+)-c([0-9]+)-([0-9]+)-(.+)-([a-zA-Z]+).html /brand.php?id=$1&cat=$2&page=$3&sort=$4&order=$5;
		rewrite ^/brand-([0-9]+)-c([0-9]+)-([0-9]+)(.*).html /brand.php?id=$1&cat=$2&page=$3;
		rewrite ^/brand-([0-9]+)-c([0-9]+)(.*).html /brand.php?id=$1&cat=$2;
		rewrite ^/brand-([0-9]+)(.*).html /brand.php?id=$1;
		rewrite ^/tag-(.*).html /search.php?keywords=$1;
		rewrite ^/snatch-([0-9]+).html$ /snatch.php?id=$1;
		rewrite ^/group_buy-([0-9]+).html$ /group_buy.php?act=view&id=$1;
		rewrite ^/auction-([0-9]+).html$ /auction.php?act=view&id=$1;

nginx 0.7x
ecshop v2.5.0 RELEASE 20071128

Nginx + PHP mysql_pconnect = Database errors (Too many connections)

If you’re using NGinx spawn-cgi or FPM with PHP and calling mysql_pconnect, you are likely going to experience frequent database crashes and “Too many connections” errors.

This took a while to trace, but once you understand the issue, it all makes sense.

mysql_pconnect opens a “persistent” connection to the database. From the documentation: “the connection to the SQL server will not be closed when the execution of the script ends. Instead, the link will remain open for future use (mysql_close() will not close links established by mysql_pconnect()).”

The issue is that FPM keeps a number of php-cgi processes running in the background to process php scripts. These php-cgi processes never die and so MySQL connections keep open forever…

Sooner or later, you are going to run out of MySQL connections (or worse yet – run out of file descriptors) and that’s when all hell breaks loose.

And if that’s not enough, after doing some digging into mysql_pconnect I found a few additional reasons NOT to use mysql_pconnect:

1. If you use mysql_pconnect on a machine that has a local database and you are connecting to a remote database, PHP will try to use the same mysql connection for both databases.

2. Temporary tables don’t work with persistent connections (they are only visible to the connection that was used to open the table)

3. Setting charset variables on a persistent connection, is going to impact all future queries on that connection as well

4. Calling mysql_pconnect twice (in the same script) with different parameters doesn’t work as expected

5. PHP 4.1 on Apache running with MySQL persistent connections, is known to memory leak (not flushing properly).

Bottom line, never ever use mysql_pconnect.

Replace all occurrences of mysql_pconnect with mysql_connect in your code and in your php.ini file, prevent persistent connections:

[MySQL] 
; Allow or prevent persistent links. 
mysql.allow_persistent = Off 

原文 http://www.softwareprojects.com/resources/programming/t-nginx-+-php-mysql_pconnect–database-errors-too-m-1751.html

关于Nginx+PHP fastcgi 胜过Apache十倍 自己验证

记得自己每一次知道nginx 还是在 群里讨论张老师的文章

他的最新文章
Nginx 0.7.x + PHP 5.2.8(FastCGI)搭建胜过Apache十倍的Web服务器(第4版)[原创]
http://blog.s135.com/post/366/

我自己是在 freebsd 7.0服务器上
硬件配置

Intel(R) Xeon(R) CPU E5410 @ 2.33GHz *2
内存 4G
sas 146 * 2 raid1
基本和张老师的配置差不多,

nginx 配置

user  www www;
worker_processes  8;

#error_log  logs/error.log;
#error_log  logs/error.log  notice;
#error_log  logs/error.log  info;

#pid        logs/nginx.pid;


worker_rlimit_nofile 200000;
events {
    worker_connections  200000;
    #use kqueue;
}

php-cgi 开启 128 进程

实际结果, nginx 也许处理静态页面可以达到3W并发,但是如是果php 得估计不行,我自己用 webbench 开起1W连接,本机差点没死,但此时 php页面打开已经相当的慢了,但status 速度仍然很快,说明nginx 的承载能力确实很强,但php处理这块依然是最大的瓶颈,此文只是本人自己测试,也许是本本配置系统有问题,导致达不到3W并发!

NginxSimplePythonFCGI

Edit section: Dispatching TurboGears Python via FCGI Dispatching TurboGears Python via FCGI

This confirmed to run on Mac OS X 10.4.7 under Turbogears 0.9.9 and 1.1a (so, no reason not to run under the 1.0b release).

Information was drawn from the Turbogears trac wiki (http://trac.turbogears.org/turbogears/wiki/NginxIntegration) which show how to use Nginx to proxy to TG, and the nearby [NginxFcgiExample] page, the latter detailing the PHP/FCGI process.

This is for Nginx/FCGI/Turbogears

Edit section: Conventions Conventions

Substitute thoughout with the values relevant to your own set-up:

${HOST} = localhost – (or whatever you choose)

${PORT} = 8080 – (or whatever you choose)

${NGINX} = /usr/local/nginx – location of nginx installation

${PROJECTBASE} /opt/projects/wiki20 – location of Turbogears project

${PROJECTNAME} wiki20

Edit section: Getting the required files Getting the required files

Two files are required to be created: ${NGINX}/scripts/fcgi.py and ${NGINX}/scripts/${PROJECTNAME}.fcgi.

To create ${NGINX}/scripts/fcgi.py

mkdir ${NGINX}/scripts
curl -o ${NGINX}/scripts/fcgi.py http://www.saddi.com/software/py-lib/py-lib/fcgi.py

To create ${NGINX}/scripts/${PROJECTNAME}.fcgi

Copy and paste the following to ${NGINX}/scripts/${PROJECTNAME}.fcgi. Edit the file, navigate to the “USER EDIT SECTION” and replace each instance of ${PROJECTBASE} and ${PROJECTNAME} with the corresponding values for your project.

#!/usr/bin/python
#
# File name: project.fcgi
#
# This module provides the glue for running TurboGears applications behind
# FastCGI-enabled web servers. The code in this module depends on the fastcgi
# module downloadable from here:
#
# http://www.saddi.com/software/py-lib/py-lib/fcgi.py
#
# NOTE: The fcgi.py file needs to be placed in a location that is on the
# system path, such as the same the directory as the tg_fastcgi.py file
# or in the base directory of the TG app code.
#
# To configure this module, please edit the three variables in the "USER EDIT
# SECTION" before starting the TG application. Also remember to edit the
# top of this file with the correct Python installation information.
 
import cherrypy
import sys
import os
from os.path import *
import pkg_resources
import turbogears
 
pkg_resources.require("TurboGears")
 
# -- START USER EDIT SECTION
# -- Users must edit this section --
code_dir = '${PROJECTBASE}' # (Required) The base directory of the TG app code.
root_class_name = '${PROJECTNAME}.controllers.Root' # (Required) The fully qualified Root class name.
project_module_name = '${PROJECTNAME}.config' # (Required) The config module name.
log_dir = '' # (Optional) The log directory. Default = code_dir.
# -- END USER EDIT SECTION
 
class VirtualPathFilter(object):
    def on_start_resource(self):
        if not cherrypy.config.get('virtual_path_filter.on', False):
     return
 prefix = cherrypy.config.get('virtual_path_filter.prefix', '')
 if not prefix:
     return
 
 path = cherrypy.request.object_path
 if path == prefix:
     path = '/'
 elif path.startswith(prefix):
     path = path[len(prefix):]
 else:
     raise cherrypy.NotFound(path)
 cherrypy.request.object_path = path
 
 
def tg_init():
    """ Checks for the required data and initializes the application. """
 
    global code_dir
    global root_class_name
    global log_dir
    global project_module_name
    last_mark = 0
 
    # Input checks
    if not code_dir or not isdir(code_dir):
        raise ValueError("""The code directory setting is missing.
                            The fastcgi code will be unable to find
                            the TG code without this setting.""")
 
    if not root_class_name:
        raise ValueError("""The fully qualified root class name must
                            be provided.""")
 
    last_mark = root_class_name.rfind('.')
 
    if (last_mark < 1) or (last_mark + 1) == len(root_class_name):
        raise ValueError("""The user-defined class name is invalid.
                            Please make sure to include a fully
                            qualified class name for the root_class
                            value (e.g. wiki20.controllers.Root).""")
 
    sys.path.append(code_dir)
 
    # Change the directory so the TG log file will not be written to the
    # web app root.
    if log_dir and isdir(log_dir):
        os.chdir(log_dir)
    else:
        os.chdir(code_dir)
        log_dir = code_dir
 
    sys.stdout = open(join(log_dir, 'stdout.log'),'a')
    sys.stderr = open(join(log_dir, 'stderr.log'),'a')
 
    if exists(join(code_dir, "setup.py")):
        turbogears.update_config(configfile=join(code_dir, "dev.cfg"),modulename=project_module_name)
    else:
        turbogears.update_config(configfile=join(code_dir, "prod.cfg"),modulename=project_module_name)
 
    # Set environment to production to disable auto-reload and
    # add virutal path information.
    cherrypy.config.update({
        'global': {'server.environment': 'production'},
     '/' : { 'virtual_path_filter.on' : True,
     'virtual_path_filter.prefix' : '/bel.fcgi' }
  })
 
    # Parse out the root class information for Cherrypy Root class.
    package_name = root_class_name[:last_mark]
    class_name = root_class_name[last_mark+1:]
    exec('from %s import %s as Root' % (package_name, class_name))
    Root._cp_filters = [VirtualPathFilter()]
    cherrypy.root = Root()
 
# Main section -
# Initialize the application, then start the server.
tg_init()
 
from fcgi import WSGIServer
cherrypy.server.start(initOnly=True, serverClass=None)
 
from cherrypy._cpwsgi import wsgiApp
WSGIServer(application=wsgiApp).run()

Edit section: Adjust the TurboGears configuration Adjust the TurboGears configuration

Edit the ${PROJECTBASE}/dev.cfg or ${PROJECTBASE}/prod.cfg file (whichever you are using), uncomment the server.socket_port assignment and change ${PORT} to a value of your choice (make sure nothing else is running on that port, Tomcat defaults to 8080, as does Jetty. Save yourself some time and check first with a telnet localhost 8080, you should see Connection refused).

The relevant lines in prod/dev.cfg are:

server.socket_port=${PORT}

Edit section: Spawning a FastCGI TurboGears process Spawning a FastCGI TurboGears process

The lighttpd “spawn-fcgi” script is useful: download, compile and install lighttpd. Then (replacing ${HOST} and ${PORT} values appropriately), execute the following:

/usr/local/bin/spawn-fcgi -a ${HOST} -p ${PORT} -u nobody -f ${NGINX}/scripts/${PROJECTNAME}.fcgi

Edit section: Nginx configuration Nginx configuration

Save the following into ${NGINX}/conf/fastcgi_params

 

#fastcgi.conf
fastcgi_param GATEWAY_INTERFACE CGI/1.1;
fastcgi_param SERVER_SOFTWARE nginx;
 
fastcgi_param QUERY_STRING $query_string;
fastcgi_param REQUEST_METHOD $request_method;
fastcgi_param CONTENT_TYPE $content_type;
fastcgi_param CONTENT_LENGTH $content_length;
 
fastcgi_param SCRIPT_NAME $fastcgi_script_name;
fastcgi_param REQUEST_URI $request_uri;
fastcgi_param DOCUMENT_URI $document_uri;
fastcgi_param DOCUMENT_ROOT $document_root;
fastcgi_param SERVER_PROTOCOL $server_protocol;
 
fastcgi_param REMOTE_ADDR $remote_addr;
fastcgi_param REMOTE_PORT $remote_port;
fastcgi_param SERVER_ADDR $server_addr;
fastcgi_param SERVER_PORT $server_port;
fastcgi_param SERVER_NAME $server_name;

Add the following to the server section of the ${NGINX}/conf/nginx.conf configuration file, changing ${HOST} and ${PORT} as appropriate:

 

  # static files
  location ~ ^/(images|javascript|js|css|flash|media|static)/ {
    root ${PROJECTBASE}/${PROJECTNAME}/static;
  }
 
  location = /favicon.ico {
    root ${PROJECTBASE}/${PROJECTNAME}/static/images;
  }
 
  # pass all requests to FastCGI TG server listening on ${HOST}:${PORT}
  #
  location / {
    fastcgi_pass ${HOST}:${PORT};
    fastcgi_index index;
    fastcgi_param SCRIPT_FILENAME /scripts$fastcgi_script_name;
    include conf/fastcgi_params;
 }

Edit section: Starting nginx Starting nginx

Start nginx with ${NGINX}/sbin/nginx. Point your browser to http://${HOST}:${PORT}/, your Turboears project should be serving via FastCGI. If so … congratulations.

Edit section: Performance test software Performance test software

Basic but usefully free http://www.hpl.hp.com/research/linux/httperf/

Edit section: Note Note

[I left the IP address as 0.0.0.0 because it worked for me, whereas 127.0.0.1 did not. If you’re experiencing difficulties connecting to 0.0.0.0:8080, these are both alternative options; localhost:8080, 127.0.0.1:8080.]

Good luck.