IP/Network

 

 

 

 

IPSec - ESP

 

ESP stands for Encapsulating Security Payload. Like AH, ESP is also a component of IPSec algorithm. It provides all the security feature provided by AH and in addition it provides Data Confidentiality by performing data encription. In this case, even if you can capture the transmitted packets using sniffer or wireshark, you cannot figure out what is the real contents of the packet because the real contents is Encrypted/Ciphered. It  is a protocol used in IPSec (Internet Protocol Security) to provide confidentiality, integrity, and authentication of data transmitted over an IP network. ESP operates at the IP layer and encapsulates the original IP packet with a new header that provides security services.

 

 

 

Two types of ESP mode

 

There are two different ways of implementing the ESP as illustrated below. One is Transport Mode and the other is Tunnel Mode. In Transport Mode, only one IP header from the original packet is used for ESP packet and only transport layer header and Payload (Data) get encrypted. In Tunnel Mode, the completely new IP header used for ESP packet and the original IP header is treated as part of encrypted data. In case of Tunnel Model, you may use different versions of IP header for original data and ESP packet. For example, you can use IPv6 for the original data and use IPv4 for ESP packet.

 

In transport mode, the original IP header is left intact and only the payload (the data portion of the IP packet) is encrypted and protected by ESP. The ESP header is added between the original IP header and the payload, and contains the security parameters, such as the Security Parameters Index (SPI) and the Sequence Number. The ESP trailer is also added to the end of the packet, and contains the Integrity Check Value (ICV), which provides integrity and authentication for the packet.

Transport mode is typically used to secure traffic between two hosts or end systems, where the original IP header needs to be retained for routing purposes.

 

In tunnel mode, a new IP header is added to the packet, and both the new header and the original IP header are protected by ESP. The original IP packet becomes the payload of the new IP packet, and is encrypted and protected by ESP. The ESP header and trailer are added after the new IP header and before the original IP packet, respectively.

Tunnel mode is typically used to secure traffic between two networks or between a host and a network, where the original IP header needs to be hidden or modified for security or routing purposes.

 

 

Since we use two different IP headers in Tunnel Mode, we often use different name for the original IP and ESP Packet IP as shown below.

 

 

Followings are the overal packet structure of ESP packets .

 

< Based on RFC 4303 Figure 1. Top-Level Format of an ESP Packet >

 

< Based on RFC 4303 Figure 2. Substructure of Payload Data >

 

 

Followings shows the ESP packet structure in more details comparing Transport Mode and Tunnel Mode.

 

 

Followings are description of each field of the ESP packet.

  • Security Parameters Index (SPI): this is used to identify the security association (SA) that is used to protect the packet.
  • Sequence Number: this is used to provide anti-replay protection by ensuring that packets are received in the correct order and have not been duplicated or replayed.
  • Payload Data: the encrypted payload of the original IP packet.
  • Padding: optional padding that is added to the packet to meet cryptographic requirements.
  • Next Header: a 1-byte field that specifies the protocol of the next header after the ESP packet.
  • ICV(Authentication Data): a message authentication code (MAC) that is used to provide integrity and authentication for the packet.
  • Padding: optional padding that is added to the packet to meet cryptographic requirements.

 

 

< Transport Mode Example >

 

Following is an example of Transport Mode ESP, showing the association between Wireshark log and ESP packet. As you see here, some parameters showing up in wireshark does not completely aligned with ESP packet structure. For example, some field (e.g, Authentication Data) is located at the end of the packet, but shown at early part of the decoding. (I know... it is very confusing and messy diagram, but I think it would be worth tracing one by one with your finger or pencil at least once).

 

 

< Tunnel Mode Example >

 

Following is an example of Tunnel Mode ESP, showing the association between Wireshark log and ESP packet. As you see here, some parameters showing up in wireshark does not completely aligned with ESP packet structure. For example, some field (e.g, Authentication Data) is located at the end of the packet, but shown at early part of the decoding. (I know... it is very confusing and messy diagram, but I think it would be worth tracing one by one with your finger or pencil at least once).

 

 

 

 

Does ESP encrypt TCP / UDP port number as well ?

 

Yes, the TCP and UDP port numbers are encrypted along with the rest of the IP packet payload. This means that even if an attacker intercepts an ESP-encrypted packet, they will not be able to determine the TCP or UDP port numbers being used.

 

This is because ESP encrypts the entire IP packet payload, including the transport layer protocol header (such as TCP or UDP), as well as any application-layer data being transmitted. The only parts of the packet that are not encrypted are the IP header fields, such as the source and destination IP addresses.

 

While ESP provides confidentiality for the IP packet payload, it can make it difficult to perform certain network functions such as traffic filtering, which typically relies on the use of TCP and UDP port numbers. To address this, some network security devices can be configured to inspect ESP-encrypted packets and extract the port numbers for filtering purposes. However, this requires additional processing and can be resource-intensive.

 

NOTE : How can some network security device can inspect ESP-encrypted packets and extract the port numbers ?

Those security devices can use a technique called decryption offloading.

    In decryption offloading, the encrypted packets are first intercepted by a device, such as a firewall or an intrusion prevention system (IPS), which is capable of decrypting ESP-encrypted packets. The device then decrypts the packet and inspects the transport layer header, which contains the TCP or UDP port numbers, as well as any other information needed for filtering or analysis.

     

    Once the device has inspected the packet, it can either allow the packet to continue its journey to its destination or block it, depending on the configured security policies.

     

    One common method for implementing decryption offloading is to use a dedicated appliance or hardware module that is specifically designed for high-performance packet processing and decryption. This device typically uses specialized hardware and software to decrypt the packets quickly and efficiently, without introducing significant latency or performance degradation.

     

    Another method for implementing decryption offloading is to use a software-based solution that runs on a server or virtual machine. In this case, the server or virtual machine must have sufficient processing power and memory to handle the additional overhead of decryption and packet processing.

 

NOTE : Do we need to provide IP sec key to such a device to decode the encrypted packet ?  

    Yes, to decrypt Encapsulating Security Payload (ESP) encrypted packets, the network security device needs to have access to the IPSec key used to encrypt the packets. This key is shared between the endpoints of the IPSec tunnel and is used to encrypt and decrypt the packets.

     

    To allow a network security device to decrypt ESP-encrypted packets, the IPSec key must be shared with the device. This can be done in a few ways, depending on the specific implementation of the IPSec tunnel and the security policies in place:

     

    Pre-shared key (PSK): In a PSK configuration, both endpoints of the IPSec tunnel use the same shared secret key to encrypt and decrypt packets. This key can be manually configured on both endpoints and on the network security device to allow decryption of packets.

     

    Public key infrastructure (PKI): In a PKI configuration, each endpoint of the IPSec tunnel has its own public-private key pair. The public keys are exchanged and trusted by a trusted third party, such as a certificate authority (CA). The network security device can be configured to use the trusted public keys to decrypt packets.

     

    Key management protocol: Some IPSec implementations use a key management protocol, such as Internet Key Exchange (IKE), to automatically exchange and manage IPSec keys between endpoints. In this case, the network security device can be configured to participate in the key exchange and obtain the necessary keys for decryption.

 

 

 

ESP vs AH

 

ESP (Encapsulating Security Payload) and AH (Authentication Header) are two protocols used in IPSec (Internet Protocol Security) to provide security services for IP packets. While both protocols provide confidentiality, integrity, and authentication for IP packets, they differ in the way they achieve these goals and the level of protection they provide.

 

The main differences between ESP and AH are as follows:

  • Confidentiality: ESP provides confidentiality by encrypting the payload of the IP packet, whereas AH does not provide encryption and therefore does not provide confidentiality.
  • Authentication: Both ESP and AH provide authentication, but they use different methods. ESP uses digital signatures to authenticate the packet, while AH uses a message authentication code (MAC).
  • Integrity: Both ESP and AH provide integrity by ensuring that the packet has not been modified during transmission. However, ESP provides integrity protection for both the IP header and payload, while AH provides integrity protection only for the IP header.
  • Protocol compatibility: AH is incompatible with NAT (Network Address Translation), while ESP is compatible with NAT.

In short, ESP is the more commonly used protocol in IPSec because it provides both confidentiality and integrity protection for IP packets, and is compatible with NAT. AH is primarily used in situations where confidentiality is not required, but authentication and integrity protection are necessary.