python3-threaded_servers

2017-11-01 04:36 UTC
  • Xyne

Metadata

Description:

Threaded server modules (ThreadedHTTPSServer, ThreadedMulticastServer, Quickserve, Pacserve).

Latest Version:

2016.10.17

Source Code:

src/

Architecture:

  • any

Dependencies:

  • python3

Optional Dependencies:

  • pyalpm: required for the Pacserve module

Arch Repositories:

  • [xyne-any]
  • [xyne-i686]
  • [xyne-x86_64]

AUR Page:

python3-threaded_servers

Tags:

About

ThreadedServers is a collection of Python 3 modules for implementing various threaded servers.

Modules

ThreadedHTTPSServer.py
This modules subclasses the standard HTTP server and request handlers in http.server to provide threading support, HTTP Digest authorization, and HTTPS (i.e. SSL) support for both server and client certificates. The handlers also provide several convenience functions for transferring files (with multipart/byteranges support) and UTF-8 encoded content (HTML, JSON, plaintext).
ThreadedMulticastServer.py
A simple threaded subclass of socketserver.UDPServer for running a multicast server. The module includes a convenience function for sending multicast datagrams and a very basic example request handler.
Quickserve.py
The main backend file of quickserve.
Pacserve.py
The main backend file of pacserve.

Caveats

These modules are provided without any warranty whatsoever. The API may change unexpectedly from one release to another even if it tends to be stable. If you wish to use these modules in your own project, you should keep local copies of working versions to avoid migration issues.

Certificate Generation

Here are some references for creating self-signed certificates:

Generate a key and self-signed certificate for the server:

openssl req -newkey rsa:1024 -x509 -nodes -days 365 -keyout server.pem -out server.pem

Generate a certificate for the client using the server certificate created above (replace "My Certificate" with the name that you want to use):

openssl pkcs12 -export -out client.pfx -in server.pem -name "My Certificate"

This PKCS#12 certificate can then be used by clients to authenticate themselves when connecting to the server. For example, to import it in Firefox, "Edit" -> "Preferences" -> "Advanced" -> "View Certificate" -> "Your Certificates" -> "Import".

Convert the PKCS#12 certificate to "pem" format:

openssl pkcs12 -in client.pfx -out client.pem -nodes

ThreadedServers.HTTPS Help Message

$ python3 -m ThreadedServers.HTTPS --help

usage: HTTPS.py [-h] [-a <interface|address>] [-p <port>] [--ipv6]
                [--auth <string> <string>] [--authfile <filepath>] [--ssl]
                [--certfile <filepath>] [--keyfile <filepath>] [--req-cert]
                [--ca-certs <filepath>]

Run a basic HTTP(S) server with HTTP digest authentication support.

optional arguments:
  -h, --help            show this help message and exit

Server Address and Port:
  Configure the server's listening address and port.

  -a <interface|address>, --address <interface|address>
                        Bind the server to this address. By default the server
                        will listen on all interfaces.
  -p <port>, --port <port>
                        Set the server port (default: 8000)
  --ipv6                Use IPv6.

HTTP Authentication:
  HTTP digest authentication via a username and password.

  --auth <string> <string>
                        HTTP digest username and password. Multiple pairs may
                        be passed.
  --authfile <filepath>
                        The path to a file containing alternating lines of
                        usernames and passwords.

SSL (HTTPS):
  Options for wrapping sockets in SSL for encrypted connections. Simply
  enabling SSL does not guarantee a secure connection and it is the user's
  responsibility to check that the implementation is correct and secure and
  that the server is properly configured. You can find information about
  generating self-signed certificates in the OpenSSL FAQ:
  http://www.openssl.org/support/faq.html

  --ssl                 Enable SSL (HTTPS).
  --certfile <filepath>
                        The path to the server's certificate.
  --keyfile <filepath>  The path to the server's key.
  --req-cert            Require a certificate from the client.
  --ca-certs <filepath>
                        Set the path to a file containing concatenated CA
                        certificates for verifying the client certificate.
                        This defaults to the server's own certificate.

ThreadedServers.MulticastQuickserve Help Message

$ python3 -m ThreadedServers.MulticastQuickserve --help

usage: MulticastQuickserve.py [-h] [--root <directory path>] [-f <filepath>]
                              [--filter <ix><regex>] [--filterlist <filepath>]
                              [--show-hidden]
                              [--tar {none,gz,bz2,xz} [{none,gz,bz2,xz} ...]]
                              [--upload <filepath>] [--allow-overwrite]
                              [--motd <filepath>] [--index <filename>]
                              [--peer <scheme>://<host>:<port>/]
                              [--list-remote] [-a <interface|address>]
                              [-p <port>] [--ipv6] [--auth <string> <string>]
                              [--authfile <filepath>] [--ssl]
                              [--certfile <filepath>] [--keyfile <filepath>]
                              [--req-cert] [--ca-certs <filepath>]
                              [--multicast]
                              [--multicast-address <interface|address>]
                              [--multicast-port <port>]
                              [--multicast-group <group>]
                              [--multicast-interval <seconds>]
                              [--multicast-interface <interface|address>]
                              [<filepath> [<filepath> ...]]

MulticastQuickserve.py - Quickserve with p2p support.

positional arguments:
  <filepath>            The files and directories to share. These will appear
                        with the same name in server root. Use the filelist
                        option for more advanced features.

optional arguments:
  -h, --help            show this help message and exit

File Download Options:
  --root <directory path>
                        If given then the directory will be treated as the
                        root of the server and all other paths will be
                        ignored. This is useful for testing static websites.
                        Similar and more complicated effects can be achieved
                        using a JSON filelist.
  -f <filepath>, --filelist <filepath>
                        A file to specify what to share on the server. If it
                        is a flat plaintext file then each line will be
                        treated as though it had been passed on the command
                        line. If it is a JSON file then it should be a map of
                        server paths to either single files or lists of
                        directories. The contents of each directory in the
                        list will appear as a single directory on the server.
  --filter <ix><regex>  Regular expressions to filter paths that appear on the
                        server. These will be applied in order when
                        determining which files to share.
  --filterlist <filepath>
                        A file consisting of filter expressions on each line.
                        The file will be reloaded if it is modified.
  --show-hidden         Share hidden files and directories.
  --tar {none,gz,bz2,xz} [{none,gz,bz2,xz} ...]
                        Enable directories to be transfered as optionally
                        compressed tar archives. This option accepts the
                        compression types to enable.

File Upload Options:
  --upload <filepath>   Enable uploads and save uploaded files in given
                        directory.
  --allow-overwrite     Allow uploaded files to overwrite existing files in
                        upload directory.

Content Options:
  --motd <filepath>     The MOTD message to display on the server. The file
                        will be reloaded if it is updated.
  --index <filename>    The name of the index page to display (if present)
                        when a directory is requested.

MulticastQuickserve Options:
  --peer <scheme>://<host>:<port>/
                        Static peers. Pass the option multiple times if
                        necessary. Example: "http://10.0.0.2:8000/"
  --list-remote         Include remote files in directory listings.

Server Address and Port:
  Configure the server's listening address and port.

  -a <interface|address>, --address <interface|address>
                        Bind the server to this address. By default the server
                        will listen on all interfaces.
  -p <port>, --port <port>
                        Set the server port (default: 8000)
  --ipv6                Use IPv6.

HTTP Authentication:
  HTTP digest authentication via a username and password.

  --auth <string> <string>
                        HTTP digest username and password. Multiple pairs may
                        be passed.
  --authfile <filepath>
                        The path to a file containing alternating lines of
                        usernames and passwords.

SSL (HTTPS):
  Options for wrapping sockets in SSL for encrypted connections. Simply
  enabling SSL does not guarantee a secure connection and it is the user's
  responsibility to check that the implementation is correct and secure and
  that the server is properly configured. You can find information about
  generating self-signed certificates in the OpenSSL FAQ:
  http://www.openssl.org/support/faq.html

  --ssl                 Enable SSL (HTTPS).
  --certfile <filepath>
                        The path to the server's certificate.
  --keyfile <filepath>  The path to the server's key.
  --req-cert            Require a certificate from the client.
  --ca-certs <filepath>
                        Set the path to a file containing concatenated CA
                        certificates for verifying the client certificate.
                        This defaults to the server's own certificate.

Multicast Options:
  Options that affect the behavior of the multicast (sub)server system.

  --multicast           Use multicasting to announce presence and detect other
                        servers.
  --multicast-address <interface|address>
                        The address to which to bind the multicast server
                        socket. Default: 0.0.0.0.
  --multicast-port <port>
                        The multicast port. Default: 15680.
  --multicast-group <group>
                        The multicast group. Default: 224.3.45.66.
  --multicast-interval <seconds>
                        The multicast announcement interval. Default: 300.
  --multicast-interface <interface|address>
                        The interface or address through which to announce
                        presence with multicast packets. If not given, all
                        interfaces are used.

ThreadedServers.Pacserve Help Message

$ python3 -m ThreadedServers.Pacserve --help

usage: Pacserve.py [-h] [--pacman-conf <filepath>] [--trust-pacserve-peers]
                   [--root <directory path>] [-f <filepath>]
                   [--filter <ix><regex>] [--filterlist <filepath>]
                   [--show-hidden]
                   [--tar {none,gz,bz2,xz} [{none,gz,bz2,xz} ...]]
                   [--upload <filepath>] [--allow-overwrite]
                   [--motd <filepath>] [--index <filename>]
                   [--peer <scheme>://<host>:<port>/] [--list-remote]
                   [-a <interface|address>] [-p <port>] [--ipv6]
                   [--auth <string> <string>] [--authfile <filepath>] [--ssl]
                   [--certfile <filepath>] [--keyfile <filepath>] [--req-cert]
                   [--ca-certs <filepath>] [--multicast]
                   [--multicast-address <interface|address>]
                   [--multicast-port <port>] [--multicast-group <group>]
                   [--multicast-interval <seconds>]
                   [--multicast-interface <interface|address>]
                   [<filepath> [<filepath> ...]]

Pacserve.py - share Pacman packages over your LAN and beyond

positional arguments:
  <filepath>            Additional files and directories to share. These will
                        appear with the same name in server root. Use the
                        filelist option for more advanced features.

optional arguments:
  -h, --help            show this help message and exit

Pacserve Options:
  --pacman-conf <filepath>
                        The Pacman configuration file to use. Default:
                        /etc/pacman.conf
  --trust-pacserve-peers
                        Allow the server to redirect database, signature and
                        other non-package request to its peers instead of
                        immediately redirecting to a mirror. This can be
                        useful for some setups but you should only use it if
                        you trust the peers or know exactly what you are
                        doing.

File Download Options:
  --root <directory path>
                        If given then the directory will be treated as the
                        root of the server and all other paths will be
                        ignored. This is useful for testing static websites.
                        Similar and more complicated effects can be achieved
                        using a JSON filelist.
  -f <filepath>, --filelist <filepath>
                        A file to specify what to share on the server. If it
                        is a flat plaintext file then each line will be
                        treated as though it had been passed on the command
                        line. If it is a JSON file then it should be a map of
                        server paths to either single files or lists of
                        directories. The contents of each directory in the
                        list will appear as a single directory on the server.
  --filter <ix><regex>  Regular expressions to filter paths that appear on the
                        server. These will be applied in order when
                        determining which files to share.
  --filterlist <filepath>
                        A file consisting of filter expressions on each line.
                        The file will be reloaded if it is modified.
  --show-hidden         Share hidden files and directories.
  --tar {none,gz,bz2,xz} [{none,gz,bz2,xz} ...]
                        Enable directories to be transfered as optionally
                        compressed tar archives. This option accepts the
                        compression types to enable.

File Upload Options:
  --upload <filepath>   Enable uploads and save uploaded files in given
                        directory.
  --allow-overwrite     Allow uploaded files to overwrite existing files in
                        upload directory.

Content Options:
  --motd <filepath>     The MOTD message to display on the server. The file
                        will be reloaded if it is updated.
  --index <filename>    The name of the index page to display (if present)
                        when a directory is requested.

MulticastQuickserve Options:
  --peer <scheme>://<host>:<port>/
                        Static peers. Pass the option multiple times if
                        necessary. Example: "http://10.0.0.2:8000/"
  --list-remote         Include remote files in directory listings.

Server Address and Port:
  Configure the server's listening address and port.

  -a <interface|address>, --address <interface|address>
                        Bind the server to this address. By default the server
                        will listen on all interfaces.
  -p <port>, --port <port>
                        Set the server port (default: 15678)
  --ipv6                Use IPv6.

HTTP Authentication:
  HTTP digest authentication via a username and password.

  --auth <string> <string>
                        HTTP digest username and password. Multiple pairs may
                        be passed.
  --authfile <filepath>
                        The path to a file containing alternating lines of
                        usernames and passwords.

SSL (HTTPS):
  Options for wrapping sockets in SSL for encrypted connections. Simply
  enabling SSL does not guarantee a secure connection and it is the user's
  responsibility to check that the implementation is correct and secure and
  that the server is properly configured. You can find information about
  generating self-signed certificates in the OpenSSL FAQ:
  http://www.openssl.org/support/faq.html

  --ssl                 Enable SSL (HTTPS).
  --certfile <filepath>
                        The path to the server's certificate.
  --keyfile <filepath>  The path to the server's key.
  --req-cert            Require a certificate from the client.
  --ca-certs <filepath>
                        Set the path to a file containing concatenated CA
                        certificates for verifying the client certificate.
                        This defaults to the server's own certificate.

Multicast Options:
  Options that affect the behavior of the multicast (sub)server system.

  --multicast           Use multicasting to announce presence and detect other
                        servers.
  --multicast-address <interface|address>
                        The address to which to bind the multicast server
                        socket. Default: 0.0.0.0.
  --multicast-port <port>
                        The multicast port. Default: 15679.
  --multicast-group <group>
                        The multicast group. Default: 224.3.45.67.
  --multicast-interval <seconds>
                        The multicast announcement interval. Default: 300.
  --multicast-interface <interface|address>
                        The interface or address through which to announce
                        presence with multicast packets. If not given, all
                        interfaces are used.

CHANGELOG

2014-01-05

  • log file transfer sizes and local filepaths

2013-11-26

  • added new "multiple" attribute to file upload input tags
  • changed query strings for displaying custom pages (upload, peers)
  • added space before closing slash in empty XML tags
  • fixed bug in index page handling

2013-09-28

  • show all bound interfaces and addresses in info message

2013-07-01

  • support for multicasting via selected interfaces
  • accept interface names as binding address arguments

TODO

TODO

Figure out how to properly force all threads to exit.

  • finalize quickserve regex prefixes
  • add support for pseudo-directory to host autogenerated files (e.g. CSS)
  • create common base class for all servers
  • maybe cache generated pages locally (with MemoizeDB?)
  • add support for peerlist files
  • (re-?)add support for peer filters
  • optionally display peer files in local directory listings
  • client certificates for secure p2p networks
  • search function
  • peer exchange
  • logging options
  • per-directory authorizations

Page Generators

Create "get_page" methods to retrieve the page pre-str for other purposes, and move the str to the send_page method when necessary for consistency.

Contact
echo xyne.archlinux.ca | sed 's/\./@/'
Feeds
Blog News
Validation
XHTML 1.0 Strict CSS level 3 Atom 1.0