A few days ago, shadowsocks proposed the SIP004 draft, aiming to replace the original insecure stream cipher + OTA with AEAD algorithms, and to deprecate One-Time Auth
(OTA).
The introduction of the new protocol is an extremely, extremely significant improvement for shadowsocks,
so I wrote this blog post to explain to friends who can’t read English friends why “OTA
was deprecated so quickly” and “why the new protocol should be used.”
1. What OTA Is
OTA (One Time Auth, one-time authentication) was an experimental feature previously added to shadowsocks to enhance security and defend against
CCA (Chosen-ciphertext Attack, chosen-ciphertext attacks).
I think a lot of people have heard of this thing — even if you don’t know what OTA is, you’ve probably at least seen the “One-Time Auth” toggle
in various shadowsocks
client forks, right? Although the name really is a bit confusing, to be fair (lol).
So next, let me explain why the OTA feature was added in the first place.
2. Weaknesses of the Original Protocol
This vulnerability in the original shadowsocks protocol was actually pointed out as early as 2015 by @breakwa11. At the time, it happened to coincide with @clowwindy being summoned for tea, and this issue
became a huge controversy with nonstop arguing. Only after quite some time did serious technical discussion begin.
If you want to learn about what happened back then, you can take a look at this issue. Here I’ll briefly summarize the vulnerability that was raised at the time.
2.1 The shadowsocks Protocol
The TCP handshake packet format (after encryption) of the original shadowsocks
protocol is as follows:
+-------+----------+
| IV | Payload |
+-------+----------+
| Fixed | Variable |
+-------+----------+
Here, the IV (Initialization Vector, initialization vector) is a fixed-length input value generated using a random number generator. By introducing an IV, the same plaintext and the same key can produce different ciphertexts, making it difficult for an attacker to crack ciphertexts encrypted with the same key.
The shadowsocks server uses this IV together with the pre-shared key (a pre-shared key, usually the password set by the user) to decrypt the payload in the TCP
packet.
The format of the decrypted content is as follows:
+--------------+---------------------+------------------+----------+
| Address Type | Destination Address | Destination Port | Data |
+--------------+---------------------+------------------+----------+
| 1 | Variable | 2 | Variable |
+--------------+---------------------+------------------+----------+
Here, Address Type (ATYP)
is the address type, occupying one byte, and it has three possible values: 01, 03, 04, corresponding respectively to IPv4, hostname, and IPv6 address types. These are all standards defined in RFC1928; if you’re interested, you can take a look.
After the handshake is completed, the shadowsocks relay operates in stream mode, and all subsequent
TCP
packets will no longer carry the IV, but will instead use the IV negotiated during the handshake.
Now that we’ve covered the original shadowsocks protocol, let’s talk about its shortcomings.
2.2 Defects of the Original Protocol
As shown in the table above, the IV field in the TCP
handshake packet of the original shadowsocks protocol is
fixed-length. Different encryption algorithms use different IV lengths; for commonly used algorithms such as rc4-md5 and the aes family, this length is 16 bytes. Detailed information about each encryption algorithm can be found in the official documentation –
Cipher.
To determine whether data is valid, the server checks the byte in the packet that represents the address information to see whether it is one of the three possible values mentioned above. If it is, it tries to parse the following address and port and establish a connection; if not, it immediately closes the connection.
It is precisely this behavior of the shadowsocks server that makes active probing possible.
2.2.1 Principle of Active Probing
This method was provided by @breakwa11
Generally speaking, “the byte that indicates the address type” is sent after being encrypted, so a third party cannot modify it precisely. But unfortunately, all shadowsocks
encryption methods are stream ciphers (stream encryption), and the characteristic of this encryption method is that “the plaintext stream corresponds one-to-one with the keystream”.
Put simply, if a certain position in the ciphertext is modified accordingly (depending on the encryption mode, this may affect the decryption of later ciphertext blocks, or it may not, but that property is not important here), then if the plaintext pattern is known in advance, although the content still cannot be decrypted and recovered, specific bytes in the ciphertext can be modified to achieve the effect of modifying the decrypted plaintext.
Based on this characteristic of stream ciphers, malicious actors can actively probe shadowsocks
servers by forging TCP
packets. An attacker only needs to brute-force modifications to the byte immediately following the IV in the encrypted packet (if the cipher used has an IV length of
16 bytes, then modify the 17th
byte), exhaustively trying all 2^8 = 256 possibilities. If, in one to three of those cases, the server being tested does not immediately close the connection, then it can be determined that the port open on that machine is running a
shadowsocks service.
Perhaps this kind of active probing is already being used by the GFW
on a large scale—who knows? The stock shadowsocks
proxy you are using could be blocked at any time.
2.2.2 Defending Against Active Probing
After discussion, the above vulnerability was proven to indeed exist, so now most shadowsocks
forks have added countermeasures against this probing method (e.g. shadowsocks-libev v2.5.5+), namely “random timeout resistance” instead of immediately closing the connection. Combined with mechanisms such as automatic blacklisting, this can effectively reduce the risk of being detected.
But this approach is not a long-term solution after all, so what should be done? 
3. OTA Makes a Grand Entrance
The reason active probing succeeds in the above situation is that the server does not verify the packets it receives. No matter what random sender a packet comes from, and regardless of whether it has been maliciously tampered with, the original
shadowsocks server will react in the same way.
At this point, @madeye (the current
shadowsocks maintainer) proposed One Time Auth,
or “one-time authentication,” adding packet verification to the original shadowsocks
protocol.
3.1 OTA Protocol
This is what a shadowsocks handshake packet looks like after OTA is enabled (before encryption):
+------+---------------------+------------------+-----------+
| ATYP | Destination Address | Destination Port | HMAC-SHA1 |
+------+---------------------+------------------+-----------+
| 1 | Variable | 2 | 10 |
+------+---------------------+------------------+-----------+
As you can see, it adds an HMAC-SHA1 field. This field is generated by applying the HMAC-SHA1 algorithm to everything except DATA (using IV + PSK as the
key). In addition, a flag bit is added to the packet header’s ATYP
to indicate whether OTA is enabled (ATYP & 0x10 == 0x10).
+----------+-----------+----------+----
| DATA.LEN | HMAC-SHA1 | DATA | ...
+----------+-----------+----------+----
| 2 | 10 | Variable | ...
+----------+-----------+----------+----
After the handshake is completed, subsequent TCP
packets all have DATA.LEN (packet length) and HMAC-SHA1 fields added to the original protocol packets. In this way, the server can perform integrity checks on the packets and thus identify tampered packets.
3.2 Flaws of OTA
OTA improves security, can defend against
CCA, and also solves the problem of packets in the original protocol being easily tampered with. Sounds great, doesn’t it?
However, in implementations of this protocol, shadowsocks-libev
and most other branches assume that the first packet must contain the entire header with SHA1-MAC, otherwise the connection is terminated.
OK, here is another point where active probing can be carried out through server behavior. However, this kind of active probing can also be defended against using the “random timeout resistance” mentioned above. The truly scary part is below:
This method was provided by @breakwa11
Do you still remember the characteristic of stream cipher mentioned above? Attackers can use the same trick to modify the DATA.LEN field in a packet, and then determine from the server’s response whether it is a
shadowsocks server.
For example, if an attacker maliciously crafts the ciphertext of the high-order byte of DATA.LEN so that after decryption the value of DATA.LEN becomes extremely large (while the size of the following DATA remains unchanged), the shadowsocks
server will continue waiting for the transmission of that data—which in fact does not exist—to complete until it times out. Therefore, by simply observing after sending the malicious packet whether the server “does not disconnect and waits at least
1 minute without any packets,” it is possible to determine whether that server has shadowsocks
enabled.
That’s right, this detection method is even more stealthy than detecting the original protocol, and it won’t even leave any suspicious traces on the server side. OTA
was originally intended to add authentication to the stream encryption of the original protocol to enhance security, but unexpectedly it introduced an even greater hidden danger. This is also why
shadowsocks-org hurriedly deprecated OTA.
4. New Protocol: AEAD
4.1 Summary and Analysis of the Flaws in Previous Protocols
The biggest flaw of the original shadowsocks
protocol is that it does not verify packet integrity. Combined with the characteristics of stream encryption, this allows attackers to modify ciphertext by brute force and conduct active probing based on server behavior.
Although the OTA
protocol verifies the integrity of DATA by appending an HMAC-SHA1 field to the end of the packet, the DATA.LEN field in the packet header, which indicates the length of DATA used to calculate offsets, is not verified. As a result, attackers can carry out more stealthy active probing by crafting ciphertext for the high-order bits of DATA.LEN.
Therefore, during the discussion of this new protocol draft, an important improvement from the shadowsocksR protocol was referenced ——
separately verifying DATA.LEN; see: ShadowsocksR Protocol Plugin Documentation.
4.2 What Is AEAD
In typical cryptographic applications, Confidentiality is achieved through encryption, while message authentication is implemented with
MAC (Message Authentication
Code). The way these two algorithms are combined has led to many security vulnerabilities. In the past, there were
3 methods:
- Encrypt-and-MAC (E&M)
- MAC-then-Encrypt (MtE) <- that is the OTA approach
- Encrypt-then-MAC (EtM) <- the new protocol’s approach
However, people later discovered that E&M and MtE both have security issues, so
starting in 2008,
the idea of “using a single algorithm to perform both encryption and authentication internally”
was gradually proposed, called AEAD
(Authenticated Encryption with Associated Data). Under the AEAD
concept, the cipher + MAC mode is replaced by a single AEAD algorithm.
The new protocol using the AEAD
algorithm is essentially a more complete stream cipher + authentication. Although it still uses stream encryption, it eliminates the possibility of ciphertext being tampered with as described above through a more robust packet integrity verification mechanism.
Note: As of the time this article was published, the new protocol uses
stream cipher + authentication, but AEAD
is designed in such a way that it can use block ciphers, so the statement above is not absolute.
To implement authenticated
encryption, the new protocol must split the TCP stream into different chunks
and verify them separately. If you are interested in the packet format definition of the new protocol, you can refer to the official documentation – AEAD; this article will not go into further detail.
4.3 AEAD Algorithms Supported by the New Protocol
At present, shadowsocks-libev already supports the following AEAD algorithms, and other branches are also catching up:
- AES-128-GCM
- AES-192-GCM
- AES-256-GCM
- ChaCha20-IETF-Poly1305
- XChaCha20-IETF-Poly1305
These new encryption algorithms are essentially stream encryption + authentication, and the previous pure stream encryption algorithms are not suitable for the new protocol.
4.4 Advantages and Disadvantages of the New Protocol
The new protocol using AEAD algorithms can solve all the problems of the Original/OTA
protocol described above, effectively defend against CCA
and man-in-the-middle attacks, and reduce the risk of active probing. The only drawback I can think of is probably performance, but how much impact can it really have? For benchmark
reference, see here.
shadowsocks
was never originally a project created for “speeding up the network”; its original purpose was to “bypass network censorship and provide secure encrypted access.” Should you continue using the old protocol that may very likely be blocked by the
GFW
, or choose the more secure new protocol instead? I believe every reader already has their own answer in mind 