Create a pip package for pipe_io_server

This commit is contained in:
Kristóf Tóth 2019-08-02 15:32:38 +02:00
parent 499ce8af76
commit c39527d99e
19 changed files with 98 additions and 0 deletions

9
LICENSE Normal file
View File

@ -0,0 +1,9 @@
The MIT License (MIT)
Copyright (c) 2019 Kristóf Tóth
Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

1
MANIFEST.in Normal file
View File

@ -0,0 +1 @@
include pipe_io_server/clients/**/*

64
README.md Normal file
View File

@ -0,0 +1,64 @@
# pipe_io_server
A trivial to use IPC solution based on POSIX named pipes and newlines.
## Why?
When it comes to communication between processes, we often find that
all we need is the simple abstraction of a text based message queue with
a blocking `send()` and `recv()` to connect 2 processes.
We could use sockets for instance, but that requires writing lots of
boilerplate. This can especially be a hassle if you want to support many
programming languages, as you would have to develop and maintain your
own bindings implementing your custom messaging protocol across all of them
(turning the abstraction of a byte stream into a queue of messages).
You could also use a message broker like RabbitMQ or Redis, but that would
force you to run a daemon in the background.
You could choose something like ZeroMQ, but all abstractions come with costs
(i.e how would you send a message to a ZMQ socket from a bash shell, or an
environment where ZMQ bindings are not easily available?).
What if you didn't have to maintain _any_ language bindings to
integrate with different languages?
What if you could rely on an API so trivial, that virtually every programming
language already provides it?
Some developers like doing IPC by writing and reading lines of text to and from named pipes.
It turns out that since a named pipe is like a file without `seek()`
(`stdin` or `stdout` _are_ pipes), you can use the readline and print utilities
of programming languages to use them.
Opening a pipe will block until the other side opens it and reading from it
also blocks until data is available. Just what we want!
The only issue is that writing a server for pipe based IO is not that
trivial if you wish to avoid blocking your main thread:
you have to start a thread for each pipe, avoid losing messages
when clients are opening and closing your pipes, deal with race conditions and so on.
It is especially hard to properly stop such a server: deadlocks are very easy to encounter
when juggling around with threads blocking on pipe operations.
This gets even more difficult if you wish your server to be resilient against irregular
client behaviour, such as the deletion of a named pipe while a thread is blocking on it.
This may sound discouraging, but this is the exact reason why I've written this package:
so you won't have to.
And solving all these issues are worth it in the end: writing clients is as trivial as it gets.
This package provides robust, asynchronous servers capable of IO over a text-based,
newline delimited protocol using named pipes.
This makes it extremely easy to integrate any external process as a plugin
into your application: clients can just open file descriptors for reading/writing
without worrying about anything else.
## Examples
A working client written in just a 4 lines of bash (`in` and `out` are the paths of the pipes):
```
while IFS= read -r message; do
printf "Received: ${message}"
printf "some response\n" > "out"
done < "in"
```
Some more examples:
- `echo_server.py` showcases a simple echo server
- `pipe_io_server/clients` contains examples on how to use pipes in different languages
- `pipe_io_server/test_pipe_io_server.py` contains a relatively comprehensive suite of unit tests

24
setup.py Normal file
View File

@ -0,0 +1,24 @@
from setuptools import setup
with open('README.md', 'r') as ifile:
long_description = ifile.read()
setup(
name='pipe_io_server',
version='1.0.0',
author='Kristóf Tóth',
author_email='mrtoth@strongds.hu',
description='A trivial to use IPC solution based on pipes and newlines',
long_description=long_description,
long_description_content_type='text/markdown',
url='https://git.strongds.hu/mrtoth/pipe-io-server',
packages=['pipe_io_server'],
package_dir={'pipe_io_server': 'pipe_io_server'},
include_package_data=True,
classifiers=[
'Programming Language :: Python :: 3',
'License :: OSI Approved :: MIT License',
'Operating System :: POSIX'
'Operating System :: Unix'
],
)