Installing Python 3.7 on Linode Server

This has been fun.

I installed a “bleeding edge” version of Python (3.7) on the server we’re running Glitch on and it’s been ah… fun?

Forget what exactly I did, but one of the mistakes with “symlinking” the new python version.

It was a mess. Ended up restoring a backup and starting over.

It’s been a while so started with:

sudo apt-get update
sudo apt-get upgrade
sudo apt-get dist-upgrade

Now bash is nearly disabled returning error:

cannot create temp file for here-document: No space left on device
-bash: cannot create temp file for here-document: No space left on device

Temp file not very big

du -hs /tmp

Two gigs of logs:

sudo du -hs /var/log
2.0G    /var/log

The syslog and daemon.log are pretty big.

Let’s clear those out.

sudo -su root
> /var/log/syslog
> /var/log/daemon.log
rm /var/log*.gz
$ sudo du -hs /var/log
75M /var/log

That’s better. Now.

git clone https://github.com/python/cpython .
fatal: unable to access 'https://github.com/python/cpython/': Could not resolve host: github.com

Tried this SO solution without success.

Fortunately the restart server solution solved the problem.

So cd into the cpython repository clone and:

./configure
make
sudo make altinstall

And after lots and lots of chugging:

$ python3
Python 3.7.0a0 (default, Dec 24 2016, 17:37:03) 
[GCC 4.9.2] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> 

Good!

cd ../
$ sudo rm -rf cpython/

Make a nice new Virtual Environment:

$ python3 -m venv env

Activate it:

$ source env/bin/activate

Pull in the latest code changes for the app.

$ git pull
$ pip install -r requirements.txt 
pip is configured with locations that require TLS/SSL, however the ssl module in Python is not available.
Collecting DateTime (from -r requirements.txt (line 1))
  Could not fetch URL https://pypi.python.org/simple/datetime/: There was a problem confirming the ssl certificate: Can't connect to HTTPS URL because the SSL module is not available. - skipping
  Could not find a version that satisfies the requirement DateTime (from -r requirements.txt (line 1)) (from versions: )
No matching distribution found for DateTime (from -r requirements.txt (line 1))

Same error as yesterday. Okay. Restore backup again and let’s start with fresh server. Rrrr. How annoying.

Already went through most of the setup last night before I realized we had backups enabled. New ssh key added to github and repository cloned. Finishing pip install.

Will need to install and configure something to send mail with. Sendmail? I think that’s fairly simple for just sending system email.

We’re using two servers in this version of the app. One for just generating the audio and track details (json) streams using a Python library called aiohttp.

$ sudo apt-get install tmux
$ tmux new -s renderer

Meanwhile in pip install land:

Collecting resampy>=0.1.0 (from librosa>=0.4.3->amen->-r requirements.txt (line 4))
  Using cached resampy-0.1.4.tar.gz
    Complete output from command python setup.py egg_info:
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "/tmp/pip-build-vxlym7db/resampy/setup.py", line 6, in <module>
        import numpy as np
    ModuleNotFoundError: No module named 'numpy'

Hmmm.

$ python
Python 3.7.0a0 (default, Dec 24 2016, 05:54:40) 
[GCC 4.9.2] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import numpy as np
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ModuleNotFoundError: No module named 'numpy'
>>> exit()

Let’s try installing manually.

$ pip install numpy

That gets us a little further.

Running setup.py install for scipy ... error
 Note: if you need reliable uninstall behavior, then install
    with pip instead of using `setup.py install`:

      - `pip install .`       (from a git repo or downloaded source
                               release)
      - `pip install scipy`   (last SciPy release on PyPI)

More yak shaving. Final error in the terminal output: python3 raise NotFoundError('no lapack/blas resources found') numpy.distutils.system_info.NotFoundError: no lapack/blas resources found.
SO? No. We’re not on Windows. Ah!:

 sudo apt-get install libblas-dev liblapack-dev libatlas-base-dev gfortran

I remember Chris Angelico telling me that a lot of SciPy is written in Fortran, which is the “best language for mathematics”. Success!

Now we need PostgreSQL:

sudo apt-get install postgresql-9.4 postgresql-client-9.4

Confirm:

$ ps -ef | grep postgre                                                                              
postgres  2195     1  0 14:33 ?        00:00:00 /usr/lib/postgresql/9.4/bin/postgres -D /var/lib/postgresql/9.4/main -c config_file=/etc/postgresql/9.4/main/postgresql.conf
postgres  2197  2195  0 14:33 ?        00:00:00 postgres: checkpointer process                                                                                              
postgres  2198  2195  0 14:33 ?        00:00:00 postgres: writer process                                                                                                    
postgres  2199  2195  0 14:33 ?        00:00:00 postgres: wal writer process                                                                                                
postgres  2200  2195  0 14:33 ?        00:00:00 postgres: autovacuum launcher process                                                                                       
postgres  2201  2195  0 14:33 ?        00:00:00 postgres: stats collector process                                                                                           
mikekil+  2237 32658  0 14:33 pts/2    00:00:00 grep postgre
mikekilmer@indu:~/appension$ sudo systemctl status postgresql.service
● postgresql.service - PostgreSQL RDBMS
   Loaded: loaded (/lib/systemd/system/postgresql.service; enabled)
   Active: active (exited) since Sat 2016-12-24 14:33:04 EST; 56s ago
 Main PID: 2046 (code=exited, status=0/SUCCESS)
   CGroup: /system.slice/postgresql.service

Dec 24 14:33:04 indu systemd[1]: Started PostgreSQL RDBMS.
Dec 24 14:33:08 indu systemd[1]: Started PostgreSQL RDBMS.

And we need to import the data dump from the other server, along with the audio files. From local dev computer rsync -avP glitchbox:/appension/ on audio directory and backupsql export from live server. Create a psql user for myself:

sudo -u postgres createuser mikekilmer
$ psql
psql: FATAL:  database "mikekilmer" does not exist

What’s going on in there?

mikekilmer@indu:~/appension$ sudo -u postgres -i
postgres@indu:~$ psql
psql (9.4.9)
Type "help" for help.

postgres=# \l
                                  List of databases
   Name    |  Owner   | Encoding |   Collate   |    Ctype    |   Access privileges   
-----------+----------+----------+-------------+-------------+-----------------------
 postgres  | postgres | UTF8     | en_US.UTF-8 | en_US.UTF-8 | 
 template0 | postgres | UTF8     | en_US.UTF-8 | en_US.UTF-8 | =c/postgres          +
           |          |          |             |             | postgres=CTc/postgres
 template1 | postgres | UTF8     | en_US.UTF-8 | en_US.UTF-8 | =c/postgres          +
           |          |          |             |             | postgres=CTc/postgres
(3 rows)

postgres=# 

Now I think there are two ways of creating the database for my account. The explicit way, within the psql terminal, which I think you can do without sudo:

$ psql -d template1
#template1=# \l
                                  List of databases
    Name    |  Owner   | Encoding |   Collate   |    Ctype    |   Access privileges   
------------+----------+----------+-------------+-------------+-----------------------
 postgres   | postgres | UTF8     | en_US.UTF-8 | en_US.UTF-8 | 
 template0  | postgres | UTF8     | en_US.UTF-8 | en_US.UTF-8 | =c/postgres          +
            |          |          |             |             | postgres=CTc/postgres
 template1  | postgres | UTF8     | en_US.UTF-8 | en_US.UTF-8 | =c/postgres          +
            |          |          |             |             | postgres=CTc/postgres
(3 rows)
template1=# CREATE DATABASE gogasca WITH OWNER gogasca ENCODING 'UTF8';

Or from bash, but still as user postgres: (sudo -i -u postgres). running createdb mikekilmer.
This is a bit of code for ~/.bashrc to activate a Virtual Environment whenever you CD into a directory in which one has been created.

checkdir() {
   [ -n "$VIRTUAL_ENV" ] && ! [[ `pwd` =~ `dirname $VIRTUAL_ENV`* ]] && echo Deactivating venv $VIRTUAL_ENV... && deactivate
   [ -z "$VIRTUAL_ENV" -a -d env ] && echo Activating venv `pwd`/env... && source env/bin/activate
  }
PROMPT_COMMAND=checkdir