IPC in PHP with Sockets, Named Pipes, Shared Memory and Message Queues.

In my last blog, I wrote about how to execute code in an app that is running on a completely different PHP version and Composer dependencies, check it out [here] if you like.

In that article, I loaded PHP code into a file with the tmpfile() command and executed it by running the php -f ... command. While this approach works great, I was intrigued to look into options that do not use the file system for data exchange.

Communication and data transfer between two separate processes on the same system is called inter-process communication or IPC. Many different technologies can be used to achieve IPC and it doesn’t have to involve passing data through files in a filesystem.

In this article, I’ll do a brief introduction of four ways of IPC and how to use them with PHP: sockets, named pipes, shared memory and message queues.

What are sockets?

A socket is a connection point for communication and data transfer, also known as an endpoint. Sockets are one method of inter-process communication that allows exchanging data between processes even if they’re apps written in completely different languages.

To connect to a socket and start communication, one must know where to connect. Defining an address where the communication will take place can be done in different ways.

In this article, I’ll focus on the internet protocol (IP) and unix domain socket (UDS). Also known as the AF_INET (IPv4 or Internet Protocol version 4) and AF_UNIX (Unix Domain Sockets) address families.

Socket endpoints

The way that internet protocol (IP) and unix domain sockets (UDS) define their address namespace is completely different. With IP, you’ll use an IP address and a port to define an endpoint. With UDS, files are used to define a path to an endpoint to connect to.

Since unix domain sockets use files as sockets and do not route any traffic through the network layer as IP does, UDS has better performance than an IP-based socket.

Although the socket files that UDS use are called files, they are merely used as an endpoint and no actual data ever passes through the file or file system. Instead, data is routed through the kernel of the operating system.

There are many things in UNIX-based operating systems that use files to define access points, such as UDS, named pipes (FIFO), shared memory, and message queues. All of which I’m covering here in this article.

Socket types

Now that we know how to find the address to our socket and where to connect, we need to decide on what socket type to use.

There are three socket types: stream, datagram and raw. These define how data is sent over the socket and what protocol is being used to send messages or streams of data. I’ll introduce the first two, streams and datagrams.

Streams are typically used with the TCP protocol and datagrams are typically used with the UDP protocol. In the next two chapters, I’ll demonstrate how they can be used in PHP with some sample code.

Datagrams with UDP over IP and UDS

Datagrams are data messages with clearly defined boundaries. That means, in contrast to streams, each message is a self-contained unit independent of data sent before or after. Each datagram must be read in its whole or the data will be lost, this is because UDP does not support reading data in chunks like TCP does.

Communication with UDP through a socket can only go one way. So, if the reader needs to send a message back to the sender it will need to open a new socket for communication back to the sender. But, it does support multicast and broadcast which can make transferring a message to multiple clients at the same time more effective and easier than using TCP.

UDP doesn’t need to first establish a connection to send data like the TCP protocol and is thus easier to set up. Compared to TCP, UDP sends packets with lower bandwidth overhead and latency which makes it useful for live streaming and real-time applications.

The syntax for UDP over IP (UDP/IP) is udp:// and for UDP over UDS (UDP/UDS) it’s udg://. Note that the listener must be listening before any data can be sent over the socket.

// //////////// //
// listener.php //
// //////////// //

// UDP/UDS [Datagram]
$file    = '/tmp/ipc-socket.sock';
@unlink($file);
$message = '';
$socket  = socket_create(AF_UNIX, SOCK_DGRAM, 0);
$bind    = socket_bind($socket, $file); // creates the .sock file
$bytes   = socket_recvfrom($socket, $message, 1024, 0, $file);
echo $message . PHP_EOL;

// UDP/IP [Datagram]
$address = '127.0.0.1';
$port    = 9999;
$message = '';
$socket  = socket_create(AF_INET, SOCK_DGRAM, SOL_UDP);
$bind    = socket_bind($socket, $address, $port);
$bytes   = socket_recvfrom($socket, $message, 1024, 0, $address, $port);
$close   = socket_close($socket); // close connection
echo $message . PHP_EOL;

// ////////// //
// sender.php //
// ////////// //

// UDP/UDS [Datagram]
$file    = '/tmp/ipc-socket.sock';
$message = 'Hello from Sender: UDP/UDS [Datagram]';
$socket  = socket_create(AF_UNIX, SOCK_DGRAM, 0);
$written = socket_sendto($socket, $message, mb_strlen($message), 0, $file);
sleep(1); // wait for UDP/IP listener

// UDP/IP [Datagram]
$address = '127.0.0.1';
$port    = 9999;
$message = 'Hello from Sender: UDP/IP [Datagram]';
$socket  = socket_create(AF_INET, SOCK_DGRAM, SOL_UDP);
$written = socket_sendto($socket, $message, mb_strlen($message), 0, $address, $port);
socket_close($socket); // close connection

Streams with TCP over IP and UDS

Streams are different from datagrams in that they don’t have clear message boundaries, it’s just a stream of bytes. Since it’s a continuous stream, there are no limits on how much data can be transferred, but the buffer and window size can be adjusted to achieve better throughput. If the size of the data being transferred is large, increasing the buffer size will improve throughput and reduce CPU usage.

Unlike datagrams, streams can be read in chunks. This also means that if one chunk is lost then the whole stream of data would become useless. To compensate for this, TCP has advanced functionality for checking that packets were delivered successfully and in the correct order. It also has mechanisms for retransmission of failed packets.

TCP is therefore a better choice than UDP when data must be guaranteed to arrive and also when transferring large data.

The syntax for TCP over IP (TCP/IP) is tcp:// followed by an ip and a port, for TCP over UDS (TCP/UDS) it’s unix://.

// ////////// //
// server.php //
// ////////// //

// TCP/UDS [Stream]
$file      = '/tmp/ipc-socket.sock';
@unlink($file);
$message   = 'Hello from Server: TCP/UDS [Stream]';
$socket    = socket_create(AF_UNIX, SOCK_STREAM, 0);
$bind      = socket_bind($socket, $file);
$listen    = socket_listen($socket);
$client1   = socket_accept($socket);
$written   = socket_write($client1, $message);
$input     = socket_read($client1, 1024);
$closeChnl = socket_close($client1);
$closeSock = socket_close($socket);
echo $input . PHP_EOL;

// TCP/IP [Stream]
$address   = '127.0.0.1';
$port      = 9999;
$message   = 'Hello from Server: TCP/IP [Stream]';
$socket    = socket_create(AF_INET, SOCK_STREAM, SOL_TCP);
$bind      = socket_bind($socket, $address, $port);
$listen    = socket_listen($socket);
$client1   = socket_accept($socket);
$written   = socket_write($client1, $message);
$input     = socket_read($client1, 1024);
$closeChnl = socket_close($client1);
$closeSock = socket_close($socket);
echo $input . PHP_EOL;

// ////////// //
// client.php //
// ////////// //
// TCP/UDS [Stream]
$file    = '/tmp/ipc-socket.sock';
$message = 'Hello from Client: TCP/UDS [Stream]';
$socket  = socket_create(AF_UNIX, SOCK_STREAM, 0);
$connect = socket_connect($socket, $file);
$written = socket_write($socket, $message);
$input   = socket_read($socket, 1024);
$close   = socket_close($socket);
echo $input . PHP_EOL;

// TCP/IP [Stream]
$address = '127.0.0.1';
$port    = 9999;
$message = 'Hello from Client: TCP/IP [Stream]';
$socket  = socket_create(AF_INET, SOCK_STREAM, SOL_TCP);
$connect = socket_connect($socket, $address, $port);
$written = socket_write($socket, $message);
$input   = socket_read($socket, 1024);
$close   = socket_close($socket);
echo $input . PHP_EOL;

Named Pipes (FIFO)

If you’ve ever used the command line to type some commands, you’ve most probably piped something from one command to another. Named pipes are similar to these types of pipes, however, they can pass data between two unrelated processes.

One process writes to the named pipe and another process reads from it. A named pipe is created by making a FIFO (first in, first out) file. This is a special type of file and is only used as an access point for communication (sounds familiar by now) and no data is ever passed through it.

The data can only flow one way, meaning that named pipes are completely unidirectional. Here’s some sample code on how they can be used in PHP.

// ////////// //
// writer.php //
// ////////// //
$file  = "/tmp/ipc-named-pipe";
@unlink($file);
posix_mkfifo($file, 0600);
$pipe  = fopen($file, 'w');
$write = fwrite($pipe, 'Hello from Sender: Named Pipe');
unlink($file); // delete pipe

// ////////// //
// reader.php //
// ////////// //
$file  = "/tmp/ipc-named-pipe";
$pipe  = fopen($file, 'r');
$input = fread($pipe, 1024);
echo $input . PHP_EOL;

Shared Memory

The kernel in the operating system is in charge of allocating memory to all processes making sure that the memory addresses don’t overlap each other. Memory owned by processes is only allowed to be accessed by the process that owns it, which is quite obvious for security and stability reasons.

That is different from shared memory where you allocate a piece of memory that doesn’t belong to any particular process. This enables multiple processes to read and write to one shared piece of memory.

In this code example, I’m using the System V extension in PHP, but I’ve also attached an example of how the same thing can be achieved with PHP’s shmop_ functions. Note that the status of shared memory can be checked with the command: ipcs -m

// ///////////////// //
// writer.php (SysV) //
// ///////////////// //
$file   = "/tmp/ipc-shared-memory";
@unlink($file);
@touch($file);
$key    = ftok($file, 'T');
$memory = shm_attach($key, 1024);
$put    = shm_put_var($memory, 1, 'Hello from Writer: Shared Memory');
$detach = shm_detach($memory);

// ///////////////// //
// reader.php (SysV) //
// ///////////////// //
$file   = "/tmp/ipc-shared-memory";
$key    = ftok($file, 'T');
$memory = shm_attach($key, 1024);
$input  = shm_get_var($memory, 1);
$remove = shm_remove($memory);
$detach = shm_detach($memory);
@unlink($file);
echo $input . PHP_EOL;

// ////////////////// //
// writer.php (Shmop) //
// ////////////////// //
$file   = "/tmp/ipc-shared-memory";
@unlink($file);
@touch($file);
$key    = ftok($file, 'T');
$memory = shmop_open($key, 'c', 0644, 1024);
$put    = shmop_write($memory, 'Hello from Writer: Shared Memory', 0);

// ////////////////// //
// reader.php (Shmop) //
// ////////////////// //
$file   = "/tmp/ipc-shared-memory";
$key    = ftok($file, 'T');
$memory = shmop_open($key, 'c', 0644, 1024);
$size   = shmop_size($memory);
$input  = shmop_read($memory, 0, $size);
shmop_delete($memory);
echo $input . PHP_EOL;

Message Queues

A message queue is another type of inter-process communication and the last one we’re going to cover in this article. It’s a list of messages stored in the kernel of the operating system. Two independent processes can push and receive messages from this queue to exchange data with each other.

This, just like shared memory, uses a file as a base to create a key, or identifier, with the ftok command. This generated key is used to identify the queue that messages are shared on. Like other IPC methods that use files as identifiers and connection points, no data is ever passed through this file and it’s simply used as a base for the key identifier.

Here’s a code example of how message queues can be used in PHP, the System V extension will be needed to run this code. Message queue limits can be checked with the command: ipcs -l

// ////////// //
// writer.php //
// ////////// //
$file   = "/tmp/ipc-message-queues";
@unlink($file);
@touch($file);
$key     = ftok($file, 'T');
$queue   = msg_get_queue($key, 0666);
$error   = null;
$message = 'Hello from Writer: Message Queue'; // If serialize is set to false, data must be either: string, int, float or bool.
msg_send($queue, 1, $message, true, false, $error);

// ////////// //
// reader.php //
// ////////// //
$file   = "/tmp/ipc-message-queues";
$key     = ftok($file, 'T');
$queue   = msg_get_queue($key, 0666);
$msgType = null;
$input   = '';
$error   = null;
msg_receive($queue, 0, $msgType, 1024, $input, true, 0, $error);
msg_remove_queue($queue);
echo $input . PHP_EOL;

Conclusion

I think that IPC is a really interesting topic and opens up so many new doors for how you can split and divide your platform into various functional separate units or apps that communicate with each other.

It might make sense to write a daemon in C or Golang but a different language might be used for a CLI tool that is communicating with this app. Or, if you have multiple separate microservices written in different or the same language that need to exchange data, IPC is there to help you out.

I introduced a few different methods of achieving inter-process communication in this blog article and they all have their strengths and weaknesses. UDP is really easy to set up and supports multicast and broadcast. While TCP is more reliable when it comes to packet delivery and large data transfers, it has a larger overhead and takes more effort to set up and connections have to be handled.

Named pipes are reliable and performant when it comes to data delivery but they’re unidirectional and might not fit every use case. With shared memory, it’s important to handle race conditions and make sure that two processes are not writing to the same piece of shared memory at the same time. Shared memory is therefore often used with semaphores or other locking mechanisms.

Lastly, message queues are similar to datagrams except that any party involved in the data exchange can push and read messages. It’s therefore important to know who should push and who should read which messages.

That’s it for this time, I hope that this will be useful for someone out there wondering about what different methods of IPC are available and how to use them with PHP.

Until next time, have a good one!