Under the Hood¶

WebServer¶

For now, yopypi uses a single threaded webserver provided by Bottle. I currently find this approach safe enough and do not have any plans to provide a more robust web server.

However, in a large environment where multiple requests are made, the web server (Bottle in this case) may take longer or fail to respond.

Daemon/Service¶

Bottle doesn’t provide a clear way of running the server as a daemon. So we are using a Python daemon library called Supay.

The way it is configured is as follows:

• It will write stderr, stdout and stdin to ~/.yopypi/yopypi.log
• Will create a PID file (a text file with the process number) in /var/run/yopypi

Although is a robust enough module, it still requires some troubleshooting every now and then. In the case that you have the webserver running and somehow the process is hanging, Supay. will try to stop it. If somehow it couldn’t, a simple ps aux | grep yopypi should do the work.

Service Status¶

Probably one of the most important aspects of the balancer is the service status check that it performs at every request.

This has to be done fast, yet it needs to be effective. After trying several approaches, like ping or using urllib2.open(url) where way to slow.

The check is performed via the socket module and we set a timeout for the connection of 0.8 seconds. This is more than enough time to reach PYPI and see if it is up and running or not.

This is a snippet of how we perform the check with added comments:

socket.setdefaulttimeout(0.8)  # sets the timeout
try:
    s = socket.socket()        # New instance of socket
    s.connect((host, port))    # Connects to PYPI on port 80
    s.send('X')                # Sends a packet
    s.close()                  # Closes the socket
    return True
[...]                          # If port 80 takes
                               # more than 0.8 seconds or
                               # doesn't allow our packet
                               # then it is down and we return False

Avoiding running as root¶

We try to avoid yopypi to be run as root, and we take a few provitions to accomplishing this:

• Log file is stored at ~/.yopypi/yopypi.log
• PID file is stored where the command line tools lives: os.path.dirname(os.path.abspath(__file__))
• Default standard port is 8080. Any port under 1000 needs root privileges.

Any change to the above will require you to have super user privileges.