Note: cdn-manager is a scaffold created by quazero, the author of the cloud transcoding express-ffmpeg, based on @dadi/cdn. It seems to be an API service. After the CDN server you set up is accessed for the first time by your designated website, it will cache some resources from the original site on the server. It can not only cache jpg, css, js, etc., but also ts, m3u8, mp4. The default caching time is one hour, but it can be set to permanent caching. Moreover, when your concurrency is high and the disk IO cannot keep up, it converts the images into binary and stores them in memory. What's even more powerful is that you can add parameters to the images, such as height, width, format, filters, proportions, rotation, etc., and then fetch the images, process them as required, and return the images you need in real-time and cache them on the server. If you have the need, you can set up a server with low latency to reduce the pressure on your source site, especially for high-concurrency websites.
Installation#
cdn-manager: https://gitee.com/quazero/cdn-manager
dadi/cdn: https://github.com/dadi/cdn
This tutorial is suitable for CentOS and it is recommended to use the CentOS 7 system.
1. Install dependencies
# Update the system
yum update -y
# Install development tools
yum -y groupinstall "Development Tools"
# Install Node.js
curl -sL https://rpm.nodesource.com/setup_10.x bash -
yum install nodejs -y
# Install pm2
npm install pm2 -g
2. Install the program
# Clone the source code
git clone https://gitee.com/quazero/cdn-manager.git
cd cdn-manager
# Install
npm install
# Set the environment
export NODE_ENV=production
3. Configure parameters
Note: This step explains the default configuration that only caches images. So we need to understand the basic configuration and then refer to the additional settings at the end of the article before running step 4.
This program mainly involves two configuration files: poster.json and config.production.json. The former is the configuration file for image processing, and the latter is the configuration file for the runtime environment.
The image configuration path is workspace/recipes/poster.json, and the default parameters are as follows:
{
"recipe": "poster",
"settings": {
"format": "jpg",
"quality": "90",
"height": "240",
"ratio": "16-9",
"resizeStyle": "entropy"
}
}
This means that the image will be processed into a ratio of 16:9, with a height of 240PX, and the format will be jpg with a quality of 90%.
The runtime environment configuration path is config/config.production.json, and the default parameters are as follows:
{
"server": {
"host": "127.0.0.1",
"port": 8001
},
"images": {
"remote": {
"enabled": true,
"path": "#"
}
}
}
This means that the server will run on port 8001, and the remote connection path will be set. For example, if my domain name is https://www.moerats.com, I will fill in that domain name after the parameter.
4. Run the program
# Go to the source code folder
cd /root/cdn-manager
# To avoid errors, it is recommended to install some modules here
npm install sharp farmhash
# Run
pm2 start index.js
Runtime environment configuration#
1. Caching settings
"caching": {
"ttl": 3600,
"expireAt": "0 5 0 * * *"
}
Explanation of the expireAt configuration:
"0 5 0 * * _" Refresh the cache at 12:05 AM every day
"0 30 11 _ _ 1-5" Refresh the cache at 11:30 AM from Monday to Friday
"0 15 14 1 _ _" Refresh the cache at 2:15 PM on the first day of each month
"0 22 _ _ 1-5" Refresh the cache at 10 PM from Monday to Friday
"_ 5 4 * * 0" Refresh the cache at 4:05 AM every Sunday
Explanation of the ttl configuration:
The unit is seconds, the default is 3600 seconds, which means the cache will expire after one hour. You can set the duration yourself.
/api/flush can be used to manually refresh the cache files.
2. Image configuration
"images": {
"directory": {
"enabled": true,
"path": "relative/path/to/your/images"
},
"remote": {
"enabled": true,
"path": "https://www.moerats.com/images"
},
"s3": {
"enabled": true,
"accessKey": "your-access-key",
"secretKey": "your-secret",
"bucketName": "your-bucket",
"region": "your-region",
"endpoint": "ams3.digitaloceanspaces.com"
}
}
There are three ways to use the images configuration. One is to directly serve images from the same host, serving local images. The second is to serve remote images. The last one is to use Amazon and Digital Ocean Space cloud storage.
directory serves local files by directly specifying the path as the folder address. For example, /www/moerats.com/picture, and then change the image address host to the CDN address.
remote serves remote files by directly setting the path to the remote URL. For example, https://www.moerats.com, and finally replace the address with the CDN address.
s3 is an expandable cloud storage that can directly cache images from various cloud storage platforms.
3. Assets configuration
"assets": {
"directory": {
"enabled": true,
"path": "/Users/absolute/path/to/your/assets"
}
}
This configuration method is similar to the image configuration. Except for replacing images with assets, the usage is the same. After setting this, all files except for jpg will be cached, including video files.
Image processing configuration#
{
"recipe": "poster",
"settings": {
"format": "jpg",
"quality": "90",
"height": "240",
"ratio": "16-9",
"resizeStyle": "entropy"
}
}
The recipe file is a pre-configured format processing file, which is established in the workspace/recipes folder.
The "recipe" parameter must be consistent with the file name. The options that can be set in settings are as follows:
blur: blur
filter: set the cropping processing algorithm
flip: flip
format: format
gravity: set the cropping area
ratio: ratio
rotate: rotation
width: width
height: height
resizeStyle cropping mode is recommended to be entropy, etc. For details, please refer to the dadi/cdn documentation.
After setting is complete, for example, if the recipe is poster, the access link will be cdnhost/poster/yourpath/1.jpg.
Domain name reverse proxy#
To access normally, you need to use domain name reverse proxy. Here, we will explain how to use Baota reverse proxy and Caddy reverse proxy. If you have Baota on your website, you can use Baota for reverse proxy. If not, it is recommended to use the second method, Caddy reverse proxy, which is quick to configure.
1. Baota reverse proxy
First, go to the Baota panel, then click on the website on the left, add a site, and then click on the added domain name. At this time, you will enter the site configuration, click on reverse proxy, and fill in the target URL as http://127.0.0.1:8001, and then enable the reverse proxy. As for enabling SSL, you can see it directly in the site configuration.
2. Caddy reverse proxy
Install Caddy:
wget -N --no-check-certificate https://raw.githubusercontent.com/ToyoDAdoubiBackup/doubi/master/caddy_install.sh && chmod +x caddy_install.sh && bash caddy_install.sh
# Backup address
wget -N --no-check-certificate https://www.moerats.com/usr/shell/Caddy/caddy_install.sh && chmod +x caddy_install.sh && bash caddy_install.sh
Configure Caddy:
# The following content is a whole, please copy and run it in SSH after modifying the domain name!
# HTTP access, this configuration will not automatically issue SSL
echo "xx.com {
gzip
proxy / http://127.0.0.1:8001
}" > /usr/local/caddy/Caddyfile
# HTTPS access, this configuration will automatically issue SSL, please resolve the domain name to the VPS server in advance
echo "xx.com {
gzip
tls [email protected]
proxy / http://127.0.0.1:8001
}" > /usr/local/caddy/Caddyfile
The tls parameter will automatically issue an ssl certificate for you. If you want to use your own ssl, change it to tls /root/xx.crt /root/xx.key. The latter is the path of the ssl certificate.
Start Caddy:
/etc/init.d/caddy start
Finally, after we have configured everything, we just need to replace the resource link domain name with the CDN address of the original website. Here is a general usage example:
Original image address: https://www.moerats.com/rats.jpg
CDN address: https://cdn.moerats.com/poster/rats.jpg, this link will automatically process the image
# If you don't want to process the image, you can directly replace the original domain name with the CDN address, like this:
https://cdn.moerats.com/rats.jpg
# For other resources, just replace the domain name with the CDN address directly