How to Reverse Proxy Google with Nginx

By /

Ever since Google was blocked, although I can get around the firewall and access it at home, using a VPN at work is inconvenient, so I could only look online for Google mirror sites (“mirror site” is the common term; a more accurate term is reverse proxy). But these mirror sites I found were often quite unstable, and after a while they would become inaccessible or even get blocked as well, so I started thinking about setting up a personal reverse proxy for my own use.

Requirements for Setting Up a Reverse Proxy

  • A VPS outside the firewall
    The VPS requirements are not high. Even a small BandwagonHOST instance with 128 MB of memory can handle both the setup and usage with no pressure at all.
  • A domain name
    There are many places online where you can apply for a free domain name, such as Freenom. You can register top-level domains with suffixes like .tk, .ml, .cf, and .ga for free, for up to one year, and renew them for free when they expire.
  • An SSL certificate for the domain
    Actually, a reverse proxy itself does not require an SSL certificate, but if you only use http without a certificate, the firewall can still detect the content being searched, making it very easy to be blocked or even have the IP banned. So if you want to reverse proxy Google, it is best to have an SSL certificate and access it via https.
    There are not many places online where you can apply for a free SSL certificate. I recommend StartSSL, which is also free for up to one year, with free renewal after expiration. The process of applying for an SSL certificate is rather complicated, so you can refer to this tutorial: Applying for and Using the New Free StartSSL SSL Certificate.

Install and Compile Nginx

Nginx itself supports reverse proxy functionality, but to reverse proxy Google more effectively, at least one third-party module is still needed. Adding third-party modules to Nginx is somewhat troublesome, because you need to download the source code and recompile it.
This time, when dealing with compiling Nginx, I referred to some online tutorials, but I did not follow them exactly. The steps are a bit more numerous, mainly because I wanted to preserve as much of the original distribution’s functionality as possible, in case I want to use Nginx for other purposes later.

All of the following commands were executed on Ubuntu. The commands for CentOs are different. The default user is root; for non-root users, many commands require sudo.

First install some packages that will be needed

# apt-get update
# apt-get install libpcre3 libpcre3-dev
# apt-get install zlib1g zlib1g-dev openssl libssl-dev
# apt-get install libxml2 libxml2-dev libxslt1-dev
# apt-get install libgd-dev libgeoip-dev
# apt-get install -y gcc g++ make automake

Install the distribution version of Nginx

# apt-get install nginx

Check the distribution version number and compile parameters

# nginx -V
nginx version: nginx/1.9.3 (Ubuntu)
built with OpenSSL 1.0.2d 9 Jul 2015
TLS SNI support enabled
configure arguments: --with-cc-opt='-g -O2 -fPIE -fstack-protector-strong -Wformat -Werror=format-security -D_FORTIFY_SOURCE=2' --with-ld-opt='-Wl,-Bsymbolic-functions -fPIE -pie -Wl,-z,relro -Wl,-z,now' --prefix=/usr/share/nginx --conf-path=/etc/nginx/nginx.conf --http-log-path=/var/log/nginx/access.log --error-log-path=/var/log/nginx/error.log --lock-path=/var/lock/nginx.lock --pid-path=/run/nginx.pid --http-client-body-temp-path=/var/lib/nginx/body --http-fastcgi-temp-path=/var/lib/nginx/fastcgi --http-proxy-temp-path=/var/lib/nginx/proxy --http-scgi-temp-path=/var/lib/nginx/scgi --http-uwsgi-temp-path=/var/lib/nginx/uwsgi --with-debug --with-pcre-jit --with-ipv6 --with-http_ssl_module --with-http_stub_status_module --with-http_realip_module --with-http_auth_request_module --with-http_addition_module --with-http_dav_module --with-http_geoip_module --with-http_gzip_static_module --with-http_image_filter_module --with-http_spdy_module --with-http_sub_module --with-http_xslt_module --with-mail --with-mail_ssl_module

From the information above, you can see that the currently installed version is 1.9.3. The long string following configure
arguments is the distribution’s compile parameters, which are very important and will be needed later.

Find a directory to store the source code, then download and extract the corresponding version of the Nginx source code in that directory: 

# wget http://nginx.org/download/nginx-1.9.3.tar.gz
# tar -zxvf nginx-1.9.3.tar.gz

To better replace information in the original webpage when using a reverse proxy, you need to add a third-party module during compilation: the substitutions extension
Download the substitutions source code 

# git clone https://github.com/yaoweibin/ngx_http_substitutions_filter_module

After downloading, there will be a
ngx_http_substitutions_filter_module directory
in the current directory

Another third-party module for conveniently configuring a reverse proxy for Google: ngx_http_google_filter_module 

git clone https://github.com/cuber/ngx_http_google_filter_module

After downloading, there will be an ngx_http_google_filter_module
directory in the current directory. This module is optional; after adding it, you can very easily set up a reverse proxy for google in the Nginx configuration file.

Enter the Nginx source directory and set the compilation parameters. In fact, you just need to add two --add-module=../xxx options to the original distribution’s compilation parameters to include these two third-party modules. 

# cd nginx-1.9.3
# ./configure 
> --with-cc-opt='-g -O2 -fPIE -fstack-protector-strong -Wformat -Werror=format-security -D_FORTIFY_SOURCE=2' --with-ld-opt='-Wl,-Bsymbolic-functions -fPIE -pie -Wl,-z,relro -Wl,-z,now' --prefix=/usr/share/nginx --conf-path=/etc/nginx/nginx.conf --http-log-path=/var/log/nginx/access.log --error-log-path=/var/log/nginx/error.log --lock-path=/var/lock/nginx.lock --pid-path=/run/nginx.pid --http-client-body-temp-path=/var/lib/nginx/body --http-fastcgi-temp-path=/var/lib/nginx/fastcgi --http-proxy-temp-path=/var/lib/nginx/proxy --http-scgi-temp-path=/var/lib/nginx/scgi --http-uwsgi-temp-path=/var/lib/nginx/uwsgi --with-debug --with-pcre-jit --with-ipv6 --with-http_ssl_module --with-http_stub_status_module --with-http_realip_module --with-http_auth_request_module --with-http_addition_module --with-http_dav_module --with-http_geoip_module --with-http_gzip_static_module --with-http_image_filter_module --with-http_spdy_module --with-http_sub_module --with-http_xslt_module --with-mail --with-mail_ssl_module 
> --add-module=../ngx_http_substitutions_filter_module 
> --add-module=../ngx_http_google_filter_module

After setting this up, it will begin checking these compile parameters and the environment. If some packages listed above were not installed, or if a newer version requires other packages, an error will be reported. For example, if libgeoip-dev is missing, the following error will be shown: 

./configure: error: the GeoIP module requires the GeoIP library.

If this kind of error appears, you can Google the error message, find the missing package and install it, and then run the ./configure ... command again to set the compile parameters.

If the check passes, the following information will be displayed at the end:

Configuration summary
  + using system PCRE library
  + using system OpenSSL library
  + md5: using OpenSSL library
  + sha1: using OpenSSL library
  + using system zlib library

  nginx path prefix: "/usr/share/nginx"
  nginx binary file: "/usr/share/nginx/sbin/nginx"
  nginx configuration prefix: "/etc/nginx"
  nginx configuration file: "/etc/nginx/nginx.conf"
  nginx pid file: "/run/nginx.pid"
  nginx error log file: "/var/log/nginx/error.log"
  nginx http access log file: "/var/log/nginx/access.log"
  nginx http client request body temporary files: "/var/lib/nginx/body"
  nginx http proxy temporary files: "/var/lib/nginx/proxy"
  nginx http fastcgi temporary files: "/var/lib/nginx/fastcgi"
  nginx http uwsgi temporary files: "/var/lib/nginx/uwsgi"
  nginx http scgi temporary files: "/var/lib/nginx/scgi"

Then compile Nginx

# make 
# make install

Replace the compiled file in the distribution’s installation directory

# cp -rf objs/nginx /usr/sbin/nginx

You can use the following commands to stop/start Nginx

# systemctl stop nginx
# systemctl start nginx

You can use the following command to check the running status

# systemctl status nginx

Nginx Settings

The Nginx configuration file is /etc/nginx/nginx.conf. Here are my settings for reverse proxying Google: 

# Configure upstream with Google's IPs. The IPs can be obtained with the nslookup www.google.com command,
# Running nslookup several times will return multiple IPs, which helps avoid triggering Google's anti-bot detection.
upstream www.google.com {
    server 172.217.0.4:443 weight=1;
    server 172.217.1.36:443 weight=1;
    server 216.58.193.196:443 weight=1;
    server 216.58.216.4:443 weight=1;
    server 216.58.216.36:443 weight=1;
    server 216.58.219.36:443 weight=1;
    server 74.125.25.99:443 weight=1;
    server 74.125.25.103:443 weight=1;
    server 74.125.25.104:443 weight=1;
    server 74.125.25.105:443 weight=1;
    server 74.125.25.106:443 weight=1;
    server 74.125.25.147:443 weight=1;
}

# Force HTTP access to redirect to HTTPS here; replace <domain.name> with your own domain.
server { 
    listen 80;
    server_name <domain.name>;
    # http to https
    location / {
          rewrite ^/(.*)$ https://<domain.name>$1 permanent;
    }
}

# HTTPS settings
server {
    listen       443 ssl;
    server_name  <domain.name>;
    resolver 8.8.8.8;

    # SSL certificate settings; replace <path to ssl.xxx> with the path to your own certificate
    ssl on;
    ssl_certificate <path to ssl.crt>;
    ssl_certificate_key <path to ssl.key>;

    # Prevent web crawlers
    #forbid spider
    if ($http_user_agent ~* "qihoobot|Baiduspider|Googlebot|Googlebot-Mobile|Googlebot-Image|Mediapartners-Google|Adsbot-Google|Feedfetcher-Google|Yahoo! Slurp|Yahoo! Slurp China|YoudaoBot|Sosospider|Sogou spider|Sogou web spider|MSNBot|ia_archiver|Tomato Bot") 
    { 
        return 403; 
    }

    # Block access via other domain names or direct IP access; only the specified domain is allowed
    #forbid illegal domain
    if ( $host != "<domain.name>" ) {
        return 403; 
    }

    access_log  off;
    error_log   on;
    error_log  /var/log/nginx/google-proxy-error.log;

    # If the ngx_http_google_filter_module module was added during compilation, the location configuration is very simple
    location / {
        google on;
    }
}

If the ngx_http_google_filter_module
module was not added during compilation, the location configuration can refer to the following: 

location / {
    proxy_redirect off;
    proxy_cookie_domain google.com <domain.name>; 
    proxy_pass https://www.google.com;
    proxy_connect_timeout 60s;
    proxy_read_timeout 5400s;
    proxy_send_timeout 5400s;

    proxy_set_header Host "www.google.com";
    proxy_set_header User-Agent $http_user_agent;
    proxy_set_header Referer https://www.google.com;
    proxy_set_header Accept-Encoding "";
    proxy_set_header X-Real-IP $remote_addr; 
    proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; 
    proxy_set_header X-Forwarded-Proto https;
    proxy_set_header Accept-Language "zh-CN";
    proxy_set_header Cookie "PREF=ID=047808f19f6de346:U=0f62f33dd8549d11:FF=2:LD=en-US:NW=1:TM=1325338577:LM=1332142444:GM=1:SG=2:S=rE0SyJh2W1IQ-Maw";

    subs_filter https://www.google.com.hk <domain.name>;
    subs_filter https://www.google.com <domain.name>;
    #subs_filter_types text/css text/xml text/javascript;

    sub_filter_once off; 
}

After modifying the configuration file, you can use nginx -t to check whether the configuration file is correct: 

# nginx -t
nginx: the configuration file /etc/nginx/nginx.conf syntax is ok
nginx: configuration file /etc/nginx/nginx.conf test is successful

Seeing the above output indicates that the configuration is correct.

After modifying the configuration file, you need to restart Nginx:

# systemctl restart nginx

Leave a Comment

Your email address will not be published. Required fields are marked *

中文 EN
🚀

RedGate VPN

免费节点太挤太慢？
升级高速稳定专线

立即体验 →
⚡

告别卡顿

RedGate VPN
全球高速节点

免费下载 →
Scroll to Top