FastDFS + Nginx 反向代理缓存多节点安装与配置（转载）_上

FastDFS + Nginx 反向代理缓存 安装与配置

  • 作者：斯巴达克斯
  • 时间：March 26, 2015

操作系统 CentOS release 6.5 (Final) 64

nginx相关软件

nginx-1.4.7 下载地址: http://nginx.org/en/download.html#
nginx清除缓存模块 ngx_cache_purge-2.1 http://labs.frickle.com/nginx_ngx_cache_purge/
pcre-8.36 ftp://ftp.csx.cam.ac.uk/pub/software/programming/pcre/pcre-8.36.tar.gz
zlib库 http://zlib.net/zlib-1.2.8.tar.gz

FastDFS相关软件

FastDFS_v5.04
http://sourceforge.net/projects/fastdfs/files

FastDFS Nginx Module 1.16
http://sourceforge.net/projects/fastdfs/files/FastDFS%20Nginx%20Module%20Source%20Code/

libfastcommon
https://github.com/happyfish100/libfastcommon.git

拓扑图

2dd42041-2aba-4a82-8a6b-b48dac7c0e86.jpg

二、tracker的安装与配置

1.安装

以下操作是在 tracker 10.10.10.124上操作

通过rz工具，把软件上传到/usr/local/src目录下，如果没有rz工具，可以使用 yum -y install lrzsz 安装

[root@tracker src]# ls
FastDFS_v5.04.tar.gz
[root@tracker src]# tar xf FastDFS_v5.04.tar.gz 
[root@tracker src]# cd FastDFS
[root@tracker FastDFS]# ll
总用量 132
drwxr-xr-x. 3 8980 users  4096 9月  21 22:14 client
drwxr-xr-x. 2 8980 users  4096 9月  21 22:14 common
drwxr-xr-x. 2 8980 users  4096 9月  21 22:14 conf
-rw-r--r--. 1 8980 users 35067 9月  21 22:14 COPYING-3_0.txt
-rw-r--r--. 1 8980 users  2802 9月  21 22:14 fastdfs.spec
-rw-r--r--. 1 8980 users 31224 9月  21 22:14 HISTORY
drwxr-xr-x. 2 8980 users  4096 9月  21 22:14 init.d
-rw-r--r--. 1 8980 users  7755 9月  21 22:14 INSTALL
-rwxr-xr-x. 1 8980 users  5735 9月  21 22:14 make.sh
drwxr-xr-x. 2 8980 users  4096 9月  21 22:14 php_client
-rw-r--r--. 1 8980 users  2380 9月  21 22:14 README.md
-rwxr-xr-x. 1 8980 users  1768 9月  21 22:14 restart.sh
-rwxr-xr-x. 1 8980 users  1680 9月  21 22:14 stop.sh
drwxr-xr-x. 4 8980 users  4096 9月  21 22:14 storage
drwxr-xr-x. 2 8980 users  4096 9月  21 22:14 test
drwxr-xr-x. 2 8980 users  4096 9月  21 22:14 tracker
[root@tracker FastDFS]# ./make.sh

执行make.sh 遇到的错误

[root@tracker FastDFS]# ./make.sh 
cc -Wall -D_FILE_OFFSET_BITS=64 -D_GNU_SOURCE -g -O -DDEBUG_FLAG -DOS_LINUX -DIOEVENT_USE_EPOLL -c -o ../common/fdfs_global.o ../common/fdfs_global.c  -I../common -I/usr/include/fastcommon
../common/fdfs_global.c:20:20: 错误：logger.h：没有那个文件或目录
在包含自 ../common/fdfs_global.c：21 的文件中:
../common/fdfs_global.h:14:27: 错误：common_define.h：没有那个文件或目录
../common/fdfs_global.h:16:29: 错误：connection_pool.h：没有那个文件或目录
In file included from ../common/fdfs_global.c:21:
../common/fdfs_global.h:26: 错误：‘MAX_PATH_SIZE’未声明(不在函数内)
../common/fdfs_global.h:27: 错误：expected ‘=’, ‘,’, ‘;’, ‘asm’ or ‘__attribute__’ before ‘g_fdfs_version’
../common/fdfs_global.h:28: 错误：expected ‘=’, ‘,’, ‘;’, ‘asm’ or ‘__attribute__’ before ‘g_use_connection_pool’
../common/fdfs_global.h:29: 错误：expected ‘=’, ‘,’, ‘;’, ‘asm’ or ‘__attribute__’ before ‘g_connection_pool’
../common/fdfs_global.c:23: 错误：‘DEFAULT_CONNECT_TIMEOUT’未声明(不在函数内)
../common/fdfs_global.c:24: 错误：‘DEFAULT_NETWORK_TIMEOUT’未声明(不在函数内)
../common/fdfs_global.c:26: 错误：expected ‘=’, ‘,’, ‘;’, ‘asm’ or ‘__attribute__’ before ‘g_fdfs_version’
../common/fdfs_global.c:27: 错误：expected ‘=’, ‘,’, ‘;’, ‘asm’ or ‘__attribute__’ before ‘g_use_connection_pool’
../common/fdfs_global.c:28: 错误：expected ‘=’, ‘,’, ‘;’, ‘asm’ or ‘__attribute__’ before ‘g_connection_pool’
../common/fdfs_global.c: 在函数‘fdfs_check_data_filename’中:
../common/fdfs_global.c:39: 警告：隐式声明函数‘logError’
../common/fdfs_global.c:45: 警告：隐式声明函数‘IS_UPPER_HEX’
make: *** [../common/fdfs_global.o] 错误 1
cc -Wall -D_FILE_OFFSET_BITS=64 -D_GNU_SOURCE -g -O -DDEBUG_FLAG -DOS_LINUX -DIOEVENT_USE_EPOLL -c -o ../common/fdfs_global.o ../common/fdfs_global.c  -I. -Itrunk_mgr -I../common -I../tracker -I../client -Ifdht_client -I/usr/include/fastcommon
../common/fdfs_global.c:20:20: 错误：logger.h：没有那个文件或目录
在包含自 ../common/fdfs_global.c：21 的文件中:
../common/fdfs_global.h:14:27: 错误：common_define.h：没有那个文件或目录
../common/fdfs_global.h:16:29: 错误：connection_pool.h：没有那个文件或目录
In file included from ../common/fdfs_global.c:21:
../common/fdfs_global.h:26: 错误：‘MAX_PATH_SIZE’未声明(不在函数内)
../common/fdfs_global.h:27: 错误：expected ‘=’, ‘,’, ‘;’, ‘asm’ or ‘__attribute__’ before ‘g_fdfs_version’
../common/fdfs_global.h:28: 错误：expected ‘=’, ‘,’, ‘;’, ‘asm’ or ‘__attribute__’ before ‘g_use_connection_pool’
../common/fdfs_global.h:29: 错误：expected ‘=’, ‘,’, ‘;’, ‘asm’ or ‘__attribute__’ before ‘g_connection_pool’
../common/fdfs_global.c:23: 错误：‘DEFAULT_CONNECT_TIMEOUT’未声明(不在函数内)
../common/fdfs_global.c:24: 错误：‘DEFAULT_NETWORK_TIMEOUT’未声明(不在函数内)
../common/fdfs_global.c:26: 错误：expected ‘=’, ‘,’, ‘;’, ‘asm’ or ‘__attribute__’ before ‘g_fdfs_version’
../common/fdfs_global.c:27: 错误：expected ‘=’, ‘,’, ‘;’, ‘asm’ or ‘__attribute__’ before ‘g_use_connection_pool’
../common/fdfs_global.c:28: 错误：expected ‘=’, ‘,’, ‘;’, ‘asm’ or ‘__attribute__’ before ‘g_connection_pool’
../common/fdfs_global.c: 在函数‘fdfs_check_data_filename’中:
../common/fdfs_global.c:39: 警告：隐式声明函数‘logError’
../common/fdfs_global.c:45: 警告：隐式声明函数‘IS_UPPER_HEX’
make: *** [../common/fdfs_global.o] 错误 1
cc -Wall -D_FILE_OFFSET_BITS=64 -D_GNU_SOURCE -g -O -DDEBUG_FLAG -DOS_LINUX -DIOEVENT_USE_EPOLL -c -o ../common/fdfs_global.o ../common/fdfs_global.c  -I../common -I../tracker -I/usr/include/fastcommon
../common/fdfs_global.c:20:20: 错误：logger.h：没有那个文件或目录
在包含自 ../common/fdfs_global.c：21 的文件中:
../common/fdfs_global.h:14:27: 错误：common_define.h：没有那个文件或目录
../common/fdfs_global.h:16:29: 错误：connection_pool.h：没有那个文件或目录
In file included from ../common/fdfs_global.c:21:
../common/fdfs_global.h:26: 错误：‘MAX_PATH_SIZE’未声明(不在函数内)
../common/fdfs_global.h:27: 错误：expected ‘=’, ‘,’, ‘;’, ‘asm’ or ‘__attribute__’ before ‘g_fdfs_version’
../common/fdfs_global.h:28: 错误：expected ‘=’, ‘,’, ‘;’, ‘asm’ or ‘__attribute__’ before ‘g_use_connection_pool’
../common/fdfs_global.h:29: 错误：expected ‘=’, ‘,’, ‘;’, ‘asm’ or ‘__attribute__’ before ‘g_connection_pool’
../common/fdfs_global.c:23: 错误：‘DEFAULT_CONNECT_TIMEOUT’未声明(不在函数内)
../common/fdfs_global.c:24: 错误：‘DEFAULT_NETWORK_TIMEOUT’未声明(不在函数内)
../common/fdfs_global.c:26: 错误：expected ‘=’, ‘,’, ‘;’, ‘asm’ or ‘__attribute__’ before ‘g_fdfs_version’
../common/fdfs_global.c:27: 错误：expected ‘=’, ‘,’, ‘;’, ‘asm’ or ‘__attribute__’ before ‘g_use_connection_pool’
../common/fdfs_global.c:28: 错误：expected ‘=’, ‘,’, ‘;’, ‘asm’ or ‘__attribute__’ before ‘g_connection_pool’
../common/fdfs_global.c: 在函数‘fdfs_check_data_filename’中:
../common/fdfs_global.c:39: 警告：隐式声明函数‘logError’
../common/fdfs_global.c:45: 警告：隐式声明函数‘IS_UPPER_HEX’
make: *** [../common/fdfs_global.o] 错误 1

看 源码目录下的INSTALL文件发现，需要安装 libfastcommon

#step 1. download libfastcommon source package from github and install it,
   the github address:
   https://github.com/happyfish100/libfastcommon.git

到如上的网址下载 libfastcommon进行安装，上传至服务器的src目录下

[root@tracker src]# unzip libfastcommon-master.zip 
[root@tracker src]# cd libfastcommon-master
[root@tracker libfastcommon-master]# ls
HISTORY  INSTALL  libfastcommon.spec  make.sh  README  src
[root@tracker libfastcommon-master]# ./make.sh 
[root@tracker libfastcommon-master]# ./make.sh install
mkdir -p /usr/lib64
install -m 755 libfastcommon.so /usr/lib64
mkdir -p /usr/include/fastcommon
install -m 644 common_define.h hash.h chain.h logger.h base64.h shared_func.h pthread_func.h ini_file_reader.h _os_bits.h sockopt.h sched_thread.h http_func.h md5.h local_ip_func.h avl_tree.h ioevent.h ioevent_loop.h fast_task_queue.h fast_timer.h process_ctrl.h fast_mblock.h connection_pool.h /usr/include/fastcommon

至此libfastcommon安装完成。

再回到 FastDFS目录下来安装FastDFS

[root@tracker libfastcommon-master]# cd ../FastDFS
[root@tracker FastDFS]# ./make.sh 
cc -Wall -D_FILE_OFFSET_BITS=64 -D_GNU_SOURCE -g -O -DDEBUG_FLAG -DOS_LINUX -DIOEVENT_USE_EPOLL -c -o ../common/fdfs_global.o ../common/fdfs_global.c  -I../common -I/usr/include/fastcommon
cc -Wall -D_FILE_OFFSET_BITS=64 -D_GNU_SOURCE -g -O -DDEBUG_FLAG -DOS_LINUX -DIOEVENT_USE_EPOLL -c -o tracker_proto.o tracker_proto.c  -I../common -I/usr/include/fastcommon
cc -Wall -D_FILE_OFFSET_BITS=64 -D_GNU_SOURCE -g -O -DDEBUG_FLAG -DOS_LINUX -DIOEVENT_USE_EPOLL -c -o tracker_mem.o tracker_mem.c  -I../common -I/usr/include/fastcommon
cc -Wall -D_FILE_OFFSET_BITS=64 -D_GNU_SOURCE -g -O -DDEBUG_FLAG -DOS_LINUX -DIOEVENT_USE_EPOLL -c -o tracker_service.o tracker_service.c  -I../common -I/usr/include/fastcommon
cc -Wall -D_FILE_OFFSET_BITS=64 -D_GNU_SOURCE -g -O -DDEBUG_FLAG -DOS_LINUX -DIOEVENT_USE_EPOLL -c -o tracker_status.o tracker_status.c  -I../common -I/usr/include/fastcommon
cc -Wall -D_FILE_OFFSET_BITS=64 -D_GNU_SOURCE -g -O -DDEBUG_FLAG -DOS_LINUX -DIOEVENT_USE_EPOLL -c -o tracker_global.o tracker_global.c  -I../common -I/usr/include/fastcommon
cc -Wall -D_FILE_OFFSET_BITS=64 -D_GNU_SOURCE -g -O -DDEBUG_FLAG -DOS_LINUX -DIOEVENT_USE_EPOLL -c -o tracker_func.o tracker_func.c  -I../common -I/usr/include/fastcommon
cc -Wall -D_FILE_OFFSET_BITS=64 -D_GNU_SOURCE -g -O -DDEBUG_FLAG -DOS_LINUX -DIOEVENT_USE_EPOLL -c -o fdfs_shared_func.o fdfs_shared_func.c  -I../common -I/usr/include/fastcommon
cc -Wall -D_FILE_OFFSET_BITS=64 -D_GNU_SOURCE -g -O -DDEBUG_FLAG -DOS_LINUX -DIOEVENT_USE_EPOLL -c -o tracker_nio.o tracker_nio.c  -I../common -I/usr/include/fastcommon
cc -Wall -D_FILE_OFFSET_BITS=64 -D_GNU_SOURCE -g -O -DDEBUG_FLAG -DOS_LINUX -DIOEVENT_USE_EPOLL -c -o tracker_relationship.o tracker_relationship.c  -I../common -I/usr/include/fastcommon
cc -Wall -D_FILE_OFFSET_BITS=64 -D_GNU_SOURCE -g -O -DDEBUG_FLAG -DOS_LINUX -DIOEVENT_USE_EPOLL -c -o tracker_dump.o tracker_dump.c  -I../common -I/usr/include/fastcommon
cc -Wall -D_FILE_OFFSET_BITS=64 -D_GNU_SOURCE -g -O -DDEBUG_FLAG -DOS_LINUX -DIOEVENT_USE_EPOLL -c -o ../common/linux_stack_trace.o ../common/linux_stack_trace.c  -I../common -I/usr/include/fastcommon
cc -Wall -D_FILE_OFFSET_BITS=64 -D_GNU_SOURCE -g -O -DDEBUG_FLAG -DOS_LINUX -DIOEVENT_USE_EPOLL -o fdfs_trackerd fdfs_trackerd.c  ../common/fdfs_global.o tracker_proto.o tracker_mem.o tracker_service.o tracker_status.o tracker_global.o tracker_func.o fdfs_shared_func.o tracker_nio.o tracker_relationship.o tracker_dump.o ../common/linux_stack_trace.o -L/usr/local/lib  -lpthread -ldl -rdynamic -lfastcommon -I../common -I/usr/include/fastcommon
cc -Wall -D_FILE_OFFSET_BITS=64 -D_GNU_SOURCE -g -O -DDEBUG_FLAG -DOS_LINUX -DIOEVENT_USE_EPOLL -c -o tracker_client_thread.o tracker_client_thread.c  -I. -Itrunk_mgr -I../common -I../tracker -I../client -Ifdht_client -I/usr/include/fastcommon
cc -Wall -D_FILE_OFFSET_BITS=64 -D_GNU_SOURCE -g -O -DDEBUG_FLAG -DOS_LINUX -DIOEVENT_USE_EPOLL -c -o storage_global.o storage_global.c  -I. -Itrunk_mgr -I../common -I../tracker -I../client -Ifdht_client -I/usr/include/fastcommon
cc -Wall -D_FILE_OFFSET_BITS=64 -D_GNU_SOURCE -g -O -DDEBUG_FLAG -DOS_LINUX -DIOEVENT_USE_EPOLL -c -o storage_func.o storage_func.c  -I. -Itrunk_mgr -I../common -I../tracker -I../client -Ifdht_client -I/usr/include/fastcommon
cc -Wall -D_FILE_OFFSET_BITS=64 -D_GNU_SOURCE -g -O -DDEBUG_FLAG -DOS_LINUX -DIOEVENT_USE_EPOLL -c -o storage_service.o storage_service.c  -I. -Itrunk_mgr -I../common -I../tracker -I../client -Ifdht_client -I/usr/include/fastcommon
cc -Wall -D_FILE_OFFSET_BITS=64 -D_GNU_SOURCE -g -O -DDEBUG_FLAG -DOS_LINUX -DIOEVENT_USE_EPOLL -c -o storage_sync.o storage_sync.c  -I. -Itrunk_mgr -I../common -I../tracker -I../client -Ifdht_client -I/usr/include/fastcommon
...
[root@tracker FastDFS]# ./make.sh install
mkdir -p /usr/bin
mkdir -p /etc/fdfs
cp -f fdfs_trackerd /usr/bin
if [ ! -f /etc/fdfs/tracker.conf.sample ]; then cp -f ../conf/tracker.conf /etc/fdfs/tracker.conf.sample; fi
mkdir -p /usr/bin
mkdir -p /etc/fdfs
cp -f fdfs_storaged  /usr/bin
if [ ! -f /etc/fdfs/storage.conf.sample ]; then cp -f ../conf/storage.conf /etc/fdfs/storage.conf.sample; fi
mkdir -p /usr/bin
mkdir -p /etc/fdfs
mkdir -p /usr/lib64
cp -f fdfs_monitor fdfs_test fdfs_test1 fdfs_crc32 fdfs_upload_file fdfs_download_file fdfs_delete_file fdfs_file_info fdfs_appender_test fdfs_appender_test1 fdfs_append_file fdfs_upload_appender /usr/bin
if [ 0 -eq 1 ]; then cp -f libfdfsclient.a /usr/lib64; fi
if [ 1 -eq 1 ]; then cp -f libfdfsclient.so /usr/lib64; fi
mkdir -p /usr/include/fastdfs
cp -f ../common/fdfs_define.h ../common/fdfs_global.h ../common/mime_file_parser.h ../common/fdfs_http_shared.h ../tracker/tracker_types.h ../tracker/tracker_proto.h ../tracker/fdfs_shared_func.h ../storage/trunk_mgr/trunk_shared.h tracker_client.h storage_client.h storage_client1.h client_func.h client_global.h fdfs_client.h /usr/include/fastdfs
if [ ! -f /etc/fdfs/client.conf.sample ]; then cp -f ../conf/client.conf /etc/fdfs/client.conf.sample; fi

至此FastDFS在 tracker上安装完成，所有的可执行文件位于 /usr/bin 目录下，以fdfs开头的文件

[root@tracker FastDFS]# ll /usr/bin/fd*
-rwxr-xr-x. 1 root root 257631 10月 28 11:29 /usr/bin/fdfs_appender_test
-rwxr-xr-x. 1 root root 257424 10月 28 11:29 /usr/bin/fdfs_appender_test1
-rwxr-xr-x. 1 root root 247664 10月 28 11:29 /usr/bin/fdfs_append_file
-rwxr-xr-x. 1 root root 247340 10月 28 11:29 /usr/bin/fdfs_crc32
-rwxr-xr-x. 1 root root 247723 10月 28 11:29 /usr/bin/fdfs_delete_file
-rwxr-xr-x. 1 root root 248850 10月 28 11:29 /usr/bin/fdfs_download_file
-rwxr-xr-x. 1 root root 248584 10月 28 11:29 /usr/bin/fdfs_file_info
-rwxr-xr-x. 1 root root 260848 10月 28 11:29 /usr/bin/fdfs_monitor
-rwxr-xr-x. 1 root root 886610 10月 28 11:29 /usr/bin/fdfs_storaged
-rwxr-xr-x. 1 root root 264031 10月 28 11:29 /usr/bin/fdfs_test
-rwxr-xr-x. 1 root root 263208 10月 28 11:29 /usr/bin/fdfs_test1
-rwxr-xr-x. 1 root root 374872 10月 28 11:29 /usr/bin/fdfs_trackerd
-rwxr-xr-x. 1 root root 248730 10月 28 11:29 /usr/bin/fdfs_upload_appender
-rwxr-xr-x. 1 root root 249676 10月 28 11:29 /usr/bin/fdfs_upload_file
[root@tracker FastDFS]#

所有的配置文件在 /etc/fdfs/ 目录下

[root@tracker FastDFS]# ll /etc/fdfs/
总用量 20
-rw-r--r--. 1 root root 1461 10月 28 11:29 client.conf.sample
-rw-r--r--. 1 root root 7647 10月 28 11:29 storage.conf.sample
-rw-r--r--. 1 root root 7102 10月 28 11:29 tracker.conf.sample
[root@tracker FastDFS]#

至此 FastDFS在tracker上安装完成，以下进行配置

2. 配置

进入到 /etc/fdfs 目录下，复制模版文件到 tracker.conf

[root@tracker fdfs]# cp tracker.conf.sample tracker.conf

创建数据文件和日志文件目录

[root@tracker fdfs]# mkdir -pv /data/fastdfs/tracker
mkdir: 已创建目录 "/data"
mkdir: 已创建目录 "/data/fastdfs"
mkdir: 已创建目录 "/data/fastdfs/tracker"

编辑 tracker.conf 文件，测试的时候只需要修改以下参数即可

disabled=false                        #启用配置文件
port=22122                            #设置 tracker 的端口号
base_path=/data/fastdfs/tracker       #设置 tracker 的数据文件和日志目录（需预先创建）
http.server_port=8080                 #设置 http 端口号

这个http.server_port=8080 指的是在tracker服务器上启动http服务进程，如:apache或者nginx 启动时所监听的端口，这个似乎是可以不用管的，因为tracker本身就没有安装http服务

3.运行

直接使用 fdfs_trackerd 来启动tracker进程，然后使用netstat 查看端口是否起来。

[root@tracker fdfs]# fdfs_trackerd /etc/fdfs/tracker.conf restart
[root@tracker fdfs]# netstat -antp | grep trackerd
tcp        0      0 0.0.0.0:22122               0.0.0.0:*                   LISTEN      14520/fdfs_trackerd 
[root@tracker fdfs]#

也可以查看日志看服务是否正常运行

[root@tracker tracker]# cat /data/fastdfs/tracker/logs/trackerd.log 
[2014-10-28 13:53:43] INFO - FastDFS v5.04, base_path=/data/fastdfs/tracker, run_by_group=, run_by_user=, connect_timeout=30s, network_timeout=60s, port=22122, bind_addr=, max_connections=256, accept_threads=1, work_threads=4, store_lookup=2, store_group=, store_server=0, store_path=0, reserved_storage_space=10.00%, download_server=0, allow_ip_count=-1, sync_log_buff_interval=10s, check_active_interval=120s, thread_stack_size=64 KB, storage_ip_changed_auto_adjust=1, storage_sync_file_max_delay=86400s, storage_sync_file_max_time=300s, use_trunk_file=0, slot_min_size=256, slot_max_size=16 MB, trunk_file_size=64 MB, trunk_create_file_advance=0, trunk_create_file_time_base=02:00, trunk_create_file_interval=86400, trunk_create_file_space_threshold=20 GB, trunk_init_check_occupying=0, trunk_init_reload_from_binlog=0, trunk_compress_binlog_min_interval=0, use_storage_id=0, id_type_in_filename=ip, storage_id_count=0, rotate_error_log=0, error_log_rotate_time=00:00, rotate_error_log_size=0, log_file_keep_days=0, store_slave_file_use_link=0, use_connection_pool=0, g_connection_pool_max_idle_time=3600s
[2014-10-28 13:53:43] INFO - local_host_ip_count: 2,  127.0.0.1  10.10.10.124

4.设置开机自动启动

[root@tracker tracker]# echo "/usr/bin/fdfs_trackerd /etc/fdfs/tracker.conf restart" >> /etc/rc.local
[root@tracker tracker]# cat /etc/rc.local
#!/bin/sh
#
# This script will be executed *after* all the other init scripts.
# You can put your own initialization stuff in here if you don't
# want to do the full Sys V style init stuff.
touch /var/lock/subsys/local
/usr/bin/fdfs_trackerd /etc/fdfs/tracker.conf restart
[root@tracker tracker]#

至此tracker上的安装与配置结束。

三、storage的安装与配置

1.安装

以下操作是在 10.10.10.120 storage1上操作

利用上传 FastDFS_v5.04.tar.gz 和 libfastcommon-master.zip 至 storage服务器的/usr/local/src 目录下

[root@server1 src]# rz
rz waiting to receive.
Starting zmodem transfer.  Press Ctrl+C to cancel.
Transferring FastDFS_v5.04.tar.gz...
  100%     335 KB     335 KB/sec    00:00:01       0 Errors  
[root@server1 src]# ls
FastDFS_v5.04.tar.gz
[root@server1 src]# rz
rz waiting to receive.
Starting zmodem transfer.  Press Ctrl+C to cancel.
Transferring libfastcommon-master.zip...
  100%      95 KB      95 KB/sec    00:00:01       0 Errors  
[root@server1 src]# ll
总用量 432
-rw-r--r-- 1 root root 343477 10月 27 09:26 FastDFS_v5.04.tar.gz
-rw-r--r-- 1 root root  97548 10月 28 11:14 libfastcommon-master.zip
[root@server1 src]#

解压安装 libfastcommon

[root@server1 src]# unzip libfastcommon-master.zip 
[root@server1 libfastcommon-master]# ll
总用量 24
-rw-r--r-- 1 root root 2397 10月 26 11:53 HISTORY
-rw-r--r-- 1 root root  582 10月 26 11:53 INSTALL
-rw-r--r-- 1 root root 1341 10月 26 11:53 libfastcommon.spec
-rwxr-xr-x 1 root root 2151 10月 26 11:53 make.sh
-rw-r--r-- 1 root root  617 10月 26 11:53 README
drwxr-xr-x 2 root root 4096 10月 26 11:53 src
[root@server1 libfastcommon-master]# ./make.sh 
cc -Wall -D_FILE_OFFSET_BITS=64 -g -DDEBUG_FLAG -DOS_LINUX -DIOEVENT_USE_EPOLL -c -fPIC -o hash.lo hash.c  
cc -Wall -D_FILE_OFFSET_BITS=64 -g -DDEBUG_FLAG -DOS_LINUX -DIOEVENT_USE_EPOLL -c -fPIC -o chain.lo chain.c  
cc -Wall -D_FILE_OFFSET_BITS=64 -g -DDEBUG_FLAG -DOS_LINUX -DIOEVENT_USE_EPOLL -c -fPIC -o shared_func.lo shared_func.c  
cc -Wall -D_FILE_OFFSET_BITS=64 -g -DDEBUG_FLAG -DOS_LINUX -DIOEVENT_USE_EPOLL -c -fPIC -o ini_file_reader.lo ini_file_reader.c  
cc -Wall -D_FILE_OFFSET_BITS=64 -g -DDEBUG_FLAG -DOS_LINUX -DIOEVENT_USE_EPOLL -c -fPIC -o logger.lo logger.c  
cc -Wall -D_FILE_OFFSET_BITS=64 -g -DDEBUG_FLAG -DOS_LINUX -DIOEVENT_USE_EPOLL -c -fPIC -o sockopt.lo sockopt.c  
cc -Wall -D_FILE_OFFSET_BITS=64 -g -DDEBUG_FLAG -DOS_LINUX -DIOEVENT_USE_EPOLL -c -fPIC -o base64.lo base64.c  
...
[root@server1 libfastcommon-master]# ./make.sh install
mkdir -p /usr/lib64
install -m 755 libfastcommon.so /usr/lib64
mkdir -p /usr/include/fastcommon
install -m 644 common_define.h hash.h chain.h logger.h base64.h shared_func.h pthread_func.h ini_file_reader.h _os_bits.h sockopt.h sched_thread.h http_func.h md5.h local_ip_func.h avl_tree.h ioevent.h ioevent_loop.h fast_task_queue.h fast_timer.h process_ctrl.h fast_mblock.h connection_pool.h /usr/include/fastcommon

解压安装 FastDFS_v5.04.tar.gz

[root@server1 src]# tar xf FastDFS_v5.04.tar.gz 
[root@server1 src]# cd FastDFS
[root@server1 FastDFS]# ll
总用量 132
drwxr-xr-x 3 8980 users  4096 9月  21 22:14 client
drwxr-xr-x 2 8980 users  4096 9月  21 22:14 common
drwxr-xr-x 2 8980 users  4096 9月  21 22:14 conf
-rw-r--r-- 1 8980 users 35067 9月  21 22:14 COPYING-3_0.txt
-rw-r--r-- 1 8980 users  2802 9月  21 22:14 fastdfs.spec
-rw-r--r-- 1 8980 users 31224 9月  21 22:14 HISTORY
drwxr-xr-x 2 8980 users  4096 9月  21 22:14 init.d
-rw-r--r-- 1 8980 users  7755 9月  21 22:14 INSTALL
-rwxr-xr-x 1 8980 users  5735 9月  21 22:14 make.sh
drwxr-xr-x 2 8980 users  4096 9月  21 22:14 php_client
-rw-r--r-- 1 8980 users  2380 9月  21 22:14 README.md
-rwxr-xr-x 1 8980 users  1768 9月  21 22:14 restart.sh
-rwxr-xr-x 1 8980 users  1680 9月  21 22:14 stop.sh
drwxr-xr-x 4 8980 users  4096 9月  21 22:14 storage
drwxr-xr-x 2 8980 users  4096 9月  21 22:14 test
drwxr-xr-x 2 8980 users  4096 9月  21 22:14 tracker
[root@server1 FastDFS]# 
[root@server1 FastDFS]# ./make.sh 
cc -Wall -D_FILE_OFFSET_BITS=64 -D_GNU_SOURCE -g -O -DDEBUG_FLAG -DOS_LINUX -DIOEVENT_USE_EPOLL -c -o ../common/fdfs_global.o ../common/fdfs_global.c  -I../common -I/usr/include/fastcommon
cc -Wall -D_FILE_OFFSET_BITS=64 -D_GNU_SOURCE -g -O -DDEBUG_FLAG -DOS_LINUX -DIOEVENT_USE_EPOLL -c -o tracker_proto.o tracker_proto.c  -I../common -I/usr/include/fastcommon
cc -Wall -D_FILE_OFFSET_BITS=64 -D_GNU_SOURCE -g -O -DDEBUG_FLAG -DOS_LINUX -DIOEVENT_USE_EPOLL -c -o tracker_mem.o tracker_mem.c  -I../common -I/usr/include/fastcommon
cc -Wall -D_FILE_OFFSET_BITS=64 -D_GNU_SOURCE -g -O -DDEBUG_FLAG -DOS_LINUX -DIOEVENT_USE_EPOLL -c -o tracker_service.o tracker_service.c  -I../common -I/usr/include/fastcommon
cc -Wall -D_FILE_OFFSET_BITS=64 -D_GNU_SOURCE -g -O -DDEBUG_FLAG -DOS_LINUX -DIOEVENT_USE_EPOLL -c -o tracker_status.o tracker_status.c  -I../common -I/usr/include/fastcommon
...
[root@server1 FastDFS]# ./make.sh install
mkdir -p /usr/bin
mkdir -p /etc/fdfs
cp -f fdfs_trackerd /usr/bin
if [ ! -f /etc/fdfs/tracker.conf.sample ]; then cp -f ../conf/tracker.conf /etc/fdfs/tracker.conf.sample; fi
mkdir -p /usr/bin
mkdir -p /etc/fdfs
cp -f fdfs_storaged  /usr/bin
if [ ! -f /etc/fdfs/storage.conf.sample ]; then cp -f ../conf/storage.conf /etc/fdfs/storage.conf.sample; fi
mkdir -p /usr/bin
mkdir -p /etc/fdfs
mkdir -p /usr/lib64
cp -f fdfs_monitor fdfs_test fdfs_test1 fdfs_crc32 fdfs_upload_file fdfs_download_file fdfs_delete_file fdfs_file_info fdfs_appender_test fdfs_appender_test1 fdfs_append_file fdfs_upload_appender /usr/bin
if [ 0 -eq 1 ]; then cp -f libfdfsclient.a /usr/lib64; fi
if [ 1 -eq 1 ]; then cp -f libfdfsclient.so /usr/lib64; fi
mkdir -p /usr/include/fastdfs
cp -f ../common/fdfs_define.h ../common/fdfs_global.h ../common/mime_file_parser.h ../common/fdfs_http_shared.h ../tracker/tracker_types.h ../tracker/tracker_proto.h ../tracker/fdfs_shared_func.h ../storage/trunk_mgr/trunk_shared.h tracker_client.h storage_client.h storage_client1.h client_func.h client_global.h fdfs_client.h /usr/include/fastdfs
if [ ! -f /etc/fdfs/client.conf.sample ]; then cp -f ../conf/client.conf /etc/fdfs/client.conf.sample; fi

安装完成后，可执行文件在/usr/bin/下以 fdfs 开头

[root@server1 FastDFS]# ll /usr/bin/fdfs_*
-rwxr-xr-x 1 root root 257631 10月 28 14:18 /usr/bin/fdfs_appender_test
-rwxr-xr-x 1 root root 257424 10月 28 14:18 /usr/bin/fdfs_appender_test1
-rwxr-xr-x 1 root root 247664 10月 28 14:18 /usr/bin/fdfs_append_file
-rwxr-xr-x 1 root root 247340 10月 28 14:18 /usr/bin/fdfs_crc32
-rwxr-xr-x 1 root root 247723 10月 28 14:18 /usr/bin/fdfs_delete_file
-rwxr-xr-x 1 root root 248850 10月 28 14:18 /usr/bin/fdfs_download_file
-rwxr-xr-x 1 root root 248584 10月 28 14:18 /usr/bin/fdfs_file_info
-rwxr-xr-x 1 root root 260848 10月 28 14:18 /usr/bin/fdfs_monitor
-rwxr-xr-x 1 root root 886610 10月 28 14:18 /usr/bin/fdfs_storaged
-rwxr-xr-x 1 root root 264031 10月 28 14:18 /usr/bin/fdfs_test
-rwxr-xr-x 1 root root 263208 10月 28 14:18 /usr/bin/fdfs_test1
-rwxr-xr-x 1 root root 374872 10月 28 14:18 /usr/bin/fdfs_trackerd
-rwxr-xr-x 1 root root 248730 10月 28 14:18 /usr/bin/fdfs_upload_appender
-rwxr-xr-x 1 root root 249676 10月 28 14:18 /usr/bin/fdfs_upload_file
[root@server1 FastDFS]#

配置文件在 /etc/fdfs 目录下

[root@server1 FastDFS]# ll /etc/fdfs/
总用量 20
-rw-r--r-- 1 root root 1461 10月 28 14:18 client.conf.sample
-rw-r--r-- 1 root root 7647 10月 28 14:18 storage.conf.sample
-rw-r--r-- 1 root root 7102 10月 28 14:18 tracker.conf.sample
[root@server1 FastDFS]#

2.配置

进入到 /etc/fdfs 目录下，复制/usr/local/src/FastDFS/conf/* 下面的所有文件到当前目录下(/etc/fdfs)

[root@server1 FastDFS]# cd /etc/fdfs/
[root@server1 fdfs]# cp /usr/local/src/FastDFS/conf/* .
[root@server1 fdfs]# ll
总用量 64
-rw-r--r-- 1 root root  1461 10月 29 14:33 client.conf
-rw-r--r-- 1 root root   858 10月 29 14:33 http.conf
-rw-r--r-- 1 root root 31172 10月 29 14:33 mime.types
-rw-r--r-- 1 root root  3697 10月 29 14:34 mod_fastdfs.conf
-rw-r--r-- 1 root root  7646 10月 28 14:34 storage.conf
-rw-r--r-- 1 root root   105 10月 29 14:33 storage_ids.conf
-rw-r--r-- 1 root root  7102 10月 29 14:33 tracker.conf
[root@server1 fdfs]#

编辑配置文件 storage.conf 测试的时候，只需修改以下内容即可

[root@server1 fdfs]# vim storage.conf
disabled=false                        #启用配置文件
group_name=group1                     #组名，根据实际情况修改
port=23000                            #设置 storage 的端口号
base_path=/data/fastdfs/storage       #设置 storage 的日志目录（需预先创建）
store_path_count=1                    #存储路径个数，需要和 store_path 个数匹配
store_path0=/data/fastdfs/storage     #存储路径
tracker_server=10.10.10.124:22122     #tracker 服务器的 IP 地址和端口号
http.server_port=8080                 #设置storage上启动的http服务的端口号，如安装的nginx的端口号

3.运行

[root@server1 fdfs]# fdfs_storaged /etc/fdfs/storage.conf restart

查看端口是否起来

[root@server1 fdfs]# netstat -antp | grep storage
tcp        0      0 0.0.0.0:23000               0.0.0.0:*                   LISTEN      10316/fdfs_storaged 
[root@server1 fdfs]#

观察日志 /data/fastdfs/storage/logs/storaged.log 看运行情况

mkdir data path: FB ...
mkdir data path: FC ...
mkdir data path: FD ...
mkdir data path: FE ...
mkdir data path: FF ...
data path: /data/fastdfs/storage/data, mkdir sub dir done.
[2014-10-28 14:42:55] INFO - file: storage_param_getter.c, line: 191, use_storage_id=0, id_type_in_filename=ip, storage_ip_changed_auto_adjust=1, store_path=0, reserved_storage_space=10.00%, use_trunk_file=0, slot_min_size=256, slot_max_size=16 MB, trunk_file_size=64 MB, trunk_create_file_advance=0, trunk_create_file_time_base=02:00, trunk_create_file_interval=86400, trunk_create_file_space_threshold=20 GB, trunk_init_check_occupying=0, trunk_init_reload_from_binlog=0, trunk_compress_binlog_min_interval=0, store_slave_file_use_link=0
[2014-10-28 14:42:55] INFO - file: storage_func.c, line: 175, tracker_client_ip: 10.10.10.120, my_server_id_str: 10.10.10.120, g_server_id_in_filename: 2013923850
[2014-10-28 14:42:55] INFO - local_host_ip_count: 2,  127.0.0.1  10.10.10.120
[2014-10-28 14:42:55] INFO - file: tracker_client_thread.c, line: 310, successfully connect to tracker server 10.10.10.124:22122, as a tracker client, my ip is 10.10.10.120
[2014-10-28 14:43:25] INFO - file: tracker_client_thread.c, line: 1235, tracker server 10.10.10.124:22122, set tracker leader: 10.10.10.124:22122

看到先创建二级目录，然后已经成功连接到tracker

可以使用 fdfs_monitor 来查看一下storage的状态，看是否已经成功注册到了tracker

[root@server2 logs]# fdfs_monitor /etc/fdfs/storage.conf

也可以以下命令来监控服务器的状态

[root@tracker ~]# fdfs_monitor /etc/fdfs/client.conf

结果如下:

[2014-10-28 15:57:54] DEBUG - base_path=/data/fastdfs/storage, connect_timeout=30, network_timeout=60, tracker_server_count=1, anti_steal_token=0, anti_steal_secret_key length=0, use_connection_pool=0, g_connection_pool_max_idle_time=3600s, use_storage_id=0, storage server id count: 0
server_count=1, server_index=0
tracker server is 10.10.10.124:22122
group count: 1
Group 1:
group name = group1
disk total space = 14020 MB
disk free space = 10267 MB
trunk free space = 0 MB
storage server count = 2
active server count = 2
storage server port = 23000
storage HTTP port = 8080
store path count = 1
subdir count per path = 256
current write server index = 0
current trunk file id = 0
        Storage 1:
                id = 10.10.10.120
                ip_addr = 10.10.10.120  ACTIVE
                http domain = 
                version = 5.04
                join time = 2014-10-28 14:42:43
                up time = 2014-10-28 14:42:43
                total storage = 14020 MB
                free storage = 10385 MB
                upload priority = 10
                store_path_count = 1
                subdir_count_per_path = 256
                storage_port = 23000
                storage_http_port = 8080
                current_write_path = 0
                source storage id = 
                if_trunk_server = 0
                connection.alloc_count = 256
                connection.current_count = 1
                connection.max_count = 1
                total_upload_count = 0
                success_upload_count = 0
                total_append_count = 0
                success_append_count = 0
                total_modify_count = 0
                success_modify_count = 0
                total_truncate_count = 0
                success_truncate_count = 0
                total_set_meta_count = 0
                success_set_meta_count = 0
                total_delete_count = 0
                success_delete_count = 0
                total_download_count = 0
                success_download_count = 0
                total_get_meta_count = 0
                success_get_meta_count = 0
                total_create_link_count = 0
                success_create_link_count = 0
                total_delete_link_count = 0
                success_delete_link_count = 0
                total_upload_bytes = 0
                success_upload_bytes = 0
                total_append_bytes = 0
                success_append_bytes = 0
                total_modify_bytes = 0
                success_modify_bytes = 0
                stotal_download_bytes = 0
                success_download_bytes = 0
                total_sync_in_bytes = 0
                success_sync_in_bytes = 0
                total_sync_out_bytes = 0
                success_sync_out_bytes = 0
                total_file_open_count = 0
                success_file_open_count = 0
                total_file_read_count = 0
                success_file_read_count = 0
                total_file_write_count = 0
                success_file_write_count = 0
                last_heart_beat_time = 2014-10-28 15:56:42
                last_source_update = 1970-01-01 08:00:00
                last_sync_update = 1970-01-01 08:00:00
                last_synced_timestamp = 1970-01-01 08:00:00 
        Storage 2:
                id = 10.10.10.122
                ip_addr = 10.10.10.122 (shizhouliang.mobimtech.local)  ACTIVE
                http domain = 
                version = 5.04
                join time = 2014-10-28 15:53:51
                up time = 2014-10-28 15:53:51
                total storage = 14020 MB
                free storage = 10267 MB
                upload priority = 10
                store_path_count = 1
                subdir_count_per_path = 256
                storage_port = 23000
                storage_http_port = 8080
                current_write_path = 0
                source storage id = 10.10.10.120
                if_trunk_server = 0
                connection.alloc_count = 256
                connection.current_count = 1
                connection.max_count = 1
                total_upload_count = 0
                success_upload_count = 0
                total_append_count = 0
                success_append_count = 0
                total_modify_count = 0
                success_modify_count = 0
                total_truncate_count = 0
                success_truncate_count = 0
                total_set_meta_count = 0
                success_set_meta_count = 0
                total_delete_count = 0
                success_delete_count = 0
                total_download_count = 0
                success_download_count = 0
                total_get_meta_count = 0
                success_get_meta_count = 0
                total_create_link_count = 0
                success_create_link_count = 0
                total_delete_link_count = 0
                success_delete_link_count = 0
                total_upload_bytes = 0
                success_upload_bytes = 0
                total_append_bytes = 0
                success_append_bytes = 0
                total_modify_bytes = 0
                success_modify_bytes = 0
                stotal_download_bytes = 0
                success_download_bytes = 0
                total_sync_in_bytes = 0
                success_sync_in_bytes = 0
                total_sync_out_bytes = 0
                success_sync_out_bytes = 0
                total_file_open_count = 0
                success_file_open_count = 0
                total_file_read_count = 0
                success_file_read_count = 0
                total_file_write_count = 0
                success_file_write_count = 0
                last_heart_beat_time = 2014-10-28 15:56:53
                last_source_update = 1970-01-01 08:00:00
                last_sync_update = 1970-01-01 08:00:00
                last_synced_timestamp = 1970-01-01 08:00:00

看到ACTIVE,就说明已经成功注册到了tracker。

4.设置开机启动

[root@server1 fdfs]# echo "/usr/bin/fdfs_storaged /etc/fdfs/storage.conf restart" >> /etc/rc.local
[root@server1 fdfs]# cat /etc/rc.local
#!/bin/sh
#
# This script will be executed *after* all the other init scripts.
# You can put your own initialization stuff in here if you don't
# want to do the full Sys V style init stuff.
touch /var/lock/subsys/local
/usr/bin/fdfs_storaged /etc/fdfs/storage.conf restart

至此 storage 已经成功安装，再把另外一台服务器也同样的步骤进行安装。

等第二台storage2(10.10.10.122)安装完成并成功启动后，看日志会发现

[2014-10-28 15:54:05] INFO - file: storage_sync.c, line: 2698, successfully connect to storage server 10.10.10.120:23000

说明已经成功连接到另外一台storage了。

也可以使用

[root@server2 logs]# fdfs_monitor /etc/fdfs/storage.conf

来查看一下storage2的状态

FastDFS + Nginx 反向代理缓存 安装与配置（转载）_下

最后编辑于 ：
©著作权归作者所有,转载或内容合作请联系作者
  • 人面猴
    序言：七十年代末，一起剥皮案震惊了整个滨河市，随后出现的几起案子，更是在滨河造成了极大的恐慌，老刑警刘岩，带你破解...
    沈念sama阅读 203,362评论 5赞 477
  • 死咒
    序言：滨河连续发生了三起死亡事件，死亡现场离奇诡异，居然都是意外死亡，警方通过查阅死者的电脑和手机，发现死者居然都...
    沈念sama阅读 85,330评论 2赞 381
  • 救了他两次的神仙让他今天三更去死
    文/潘晓璐 我一进店门，熙熙楼的掌柜王于贵愁眉苦脸地迎上来，“玉大人，你说我怎么就摊上这事。” “怎么了？”我有些...
    开封第一讲书人阅读 150,247评论 0赞 337
  • 道士缉凶录：失踪的卖姜人
    文/不坏的土叔 我叫张陵，是天一观的道长。 经常有香客问我，道长，这世上最难降的妖魔是什么？ 我笑而不...
    开封第一讲书人阅读 54,560评论 1赞 273
  • 港岛之恋（遗憾婚礼）
    正文 为了忘掉前任，我火速办了婚礼，结果婚礼上，老公的妹妹穿的比我还像新娘。我一直安慰自己，他们只是感情好，可当我...
    茶点故事阅读 63,580评论 5赞 365
  • 恶毒庶女顶嫁案：这布局不是一般人想出来的
    文/花漫 我一把揭开白布。 她就那样静静地躺着，像睡着了一般。 火红的嫁衣衬着肌肤如雪。 梳的纹丝不乱的头发上，一...
    开封第一讲书人阅读 48,569评论 1赞 281
  • 城市分裂传说
    那天，我揣着相机与录音，去河边找鬼。 笑死，一个胖子当着我的面吹牛，可吹牛的内容都是我干的。 我是一名探鬼主播，决...
    沈念sama阅读 37,929评论 3赞 395
  • 双鸳鸯连环套：你想象不到人心有多黑
    文/苍兰香墨 我猛地睁开眼，长吁一口气：“原来是场噩梦啊……” “哼！你这毒妇竟也来了？” 一声冷哼从身侧响起，我...
    开封第一讲书人阅读 36,587评论 0赞 258
  • 万荣杀人案实录
    序言：老挝万荣一对情侣失踪，失踪者是张志新（化名）和其女友刘颖，没想到半个月后，有当地人在树林里发现了一具尸体，经...
    沈念sama阅读 40,840评论 1赞 297
  • 护林员之死
    正文 独居荒郊野岭守林人离奇死亡，尸身上长有42处带血的脓包…… 初始之章·张勋 以下内容为张勋视角 年9月15日...
    茶点故事阅读 35,596评论 2赞 321
  • 白月光启示录
    正文 我和宋清朗相恋三年，在试婚纱的时候发现自己被绿了。 大学时的朋友给我发了我未婚夫和他白月光在一起吃饭的照片。...
    茶点故事阅读 37,678评论 1赞 329
  • 活死人
    序言：一个原本活蹦乱跳的男人离奇死亡，死状恐怖，灵堂内的尸体忽然破棺而出，到底是诈尸还是另有隐情，我是刑警宁泽，带...
    沈念sama阅读 33,366评论 4赞 318
  • 日本核电站爆炸内幕
    正文 年R本政府宣布，位于F岛的核电站，受9级特大地震影响，放射性物质发生泄漏。R本人自食恶果不足惜，却给世界环境...
    茶点故事阅读 38,945评论 3赞 307
  • 男人毒药：我在死后第九天来索命
    文/蒙蒙 一、第九天 我趴在偏房一处隐蔽的房顶上张望。 院中可真热闹，春花似锦、人声如沸。这庄子的主人今日做“春日...
    开封第一讲书人阅读 29,929评论 0赞 19
  • 一桩弑父案，背后竟有这般阴谋
    文/苍兰香墨 我抬头看了看天上的太阳。三九已至，却和暖如春，着一层夹袄步出监牢的瞬间，已是汗流浃背。 一阵脚步声响...
    开封第一讲书人阅读 31,165评论 1赞 259
  • 情欲美人皮
    我被黑心中介骗来泰国打工， 没想到刚下飞机就差点儿被人妖公主榨干…… 1. 我叫王不留，地道东北人。 一个月前我还...
    沈念sama阅读 43,271评论 2赞 349
  • 代替公主和亲
    正文 我出身青楼，却偏偏与公主长得像，于是被迫代替她去往敌国和亲。 传闻我的和亲对象是个残疾皇子，可洞房花烛夜当晚...
    茶点故事阅读 42,403评论 2赞 342

推荐阅读更多精彩内容