Peter Sobot’s Introducing Forever.FM article links to his gist.github page and a module he made that interfaces with the Lame MP3 encoder.
Well I finally discovered that he actually has a WWW.github page where the code for Forever.FM is posted. Wow. I guess I should have asked.
Anyway – today I’m going to get THAT BEAST running on my OSX server and then possibly deconstructing it in developing the Major Glitch library.
First you’ll need to git clone https://github.com/psobot/foreverfm.git.
Then there is a huge list of dependencies that needed to be installed, many of which makeup this long list:
pip freeze
CoffeeScript==1.0.10
DateTime==4.0.1
PIL==1.1.7
PyExecJS==1.0.4
PyMySQL==0.6.2
PyYAML==3.11
TornadIO2==0.0.4
Twisted==14.0.2
argparse==1.2.1
backports.ssl-match-hostname==3.4.0.2
certifi==14.05.14
cryptography==0.5.4
docutils==0.12
fudge==1.0.3
gnureadline==6.3.3
httplib2==0.9
initgroups==2.13.0
iotop==0.6
ipython==2.2.0
keyring==4.0
lazr.uri==1.0.3
nose==1.3.4
numpy==1.8.2
oauth==1.0.1
persistent==4.0.8
pyOpenSSL==0.14
pyScss==1.2.1
pycparser==2.10
pycrypto==2.6.1
pyechonest==4.2.22
pyserial==2.7
pytz==2014.7
remix==1.5.1psobot
requests==2.4.1
simplejson==3.6.3
six==1.8.0
soundcloud==0.4.1
tornado==4.0.2
wadllib==1.3.2
wsgiref==0.1.2
There are a few packages Sobot has listed in the requirements.txt file, which do not seem to be necessary (at least to get to the working version I have), one of which is Zope, which seems to have installed a ton of other little packages. Many of these I was able to uninstall with the command pip freeze | grep 'zope' | xargs pip uninstall -y.
The rest of the Zope packages include the following and then some:
AccessControl==3.0.9
Acquisition==4.0
DocumentTemplate==2.13.2
Persistence==2.13.2
Products.BTreeFolder2==2.13.4
Products.ExternalMethod==2.13.0
Products.MIMETools==2.13.0
Products.MailHost==2.13.1
Products.OFSP==2.13.2
Products.PythonScripts==2.13.2
Products.StandardCacheManagers==2.13.1
Products.ZCTextIndex==2.13.5
Products.ZCatalog==3.0.2
ExtensionClass==4.0
Missing==3.0
MultiMapping==2.13.0
Persistence==2.13.2
Record==3.0
RestrictedPython==3.6.0
cffi==0.8.6
mechanize==0.2.5
tempstorage==2.12.2
transaction==1.4.3Within the remix directory you will have to cd to the remix directory and update git dependencies – something along the lines of git submodule foreach git pull, which will install his version of pyechonest containing the LocalAudioStream class.
Update: the process for adding the submodule is actually:
git submodule sync
git submodule init
git submodule update
If you get an error like permission denied (public key)
You’ll have to edit .gitmodule and .git/config to change the git@github.com:psobot/pyechonest to https://github.com/psobot/pyechonest as per SO. If you get “fatal: Needed a single revision” error, have to remove the empty submodule folder (rm -rf pyechonest), then run git submodule update.
You’ll need to install a version of MySQL that’s accessible to your OS (I have MAMP, but wasn’t sure how to make it’s version of MySQL play with Python). I’m running OS X 10.6 so Server version: 5.5.40 MySQL, which is here via the MySql site.
Also need to have one MySQL database named foreverfm, which you can create from the command line by first following the details in the ReadMe.txt file to get MySQL into the environment (I made aliases in ~/.bash_profile), then start it up with mysql -u root -p. After that make the database with command CREATE DATABASE foreverfm and there’s a schema.sql file containing the command to make the necessary tracks table: CREATE TABLE `tracks` (...table details here...) ENGINE=InnoDB DEFAULT CHARSET=utf8;. I think you could pull this script with the SOURCE command, but I wasn’t sure so just added each line into the mysql session in the terminal.
Note: Leaving in the following paragraph, but actually, though nginx is used in the live implementation of ForeverFM, it’s not necessary to get the module running. Also I removed all the social media calls from the index.html template.
Then you have to get nginx (“Engine x”) working. Hmmmpf. For this you will need to have XCode or at least (my option) the GCC compiler. I followed these instructions, but you may have to change the version of PCRE (Perl Compatible Regular Expressions) to a version that still exists. Following those instructions, you may also have to create the directory /usr/local/src. (Do not do as I did and accidentally run sudo chown -R usr! If you do fix with Applejack utility OR enable logging in as Root and run chmod u+s /usr/bin/sudo.)
I had to modify liveyamlfile.py a little bit:
File "/Users/mikekilmer/Envs/GLITCH/glitcher/forever/liveyamlfile.py", line 96, in __getattribute__
fmod_time = os.stat(self.__file)[9]
OSError: [Errno 2] No such file or directory: 'Users/mikekilmer/Envs/GLITCH/glitcher/api_keys.yml'
Because the OS is looking for a "/". So added a couple of workarounds.
try:
target_file = open(self.__file)
except IOError:
target_file = open("/" + self.__file) # add "/" cause of error on osx
try:
fmod_time = os.stat(self.__file)[9]
except OSError:
fmod_time = os.stat("/" + self.__file)[9] # added "/" cause error on osx
Also OS and MySQLdb do not get along well, so am using PyMySQL, which is similar enough that all I needed to do is change one import statement and the call to the .connect function in the database.py file. Note, though that the module is actually called by pymysql (all lower case).
So you wanna make sure MySQL server is running, then call the forever script from the directory in which the forever directory resides:
You need to call server.py like this python -m forever.server, using the module-name flag, otherwise the __init__ won’t run and initiate the CustomLogging.
python -m forever.server
When it’s working the output will look like:
/Users/mikekilmer/Envs/GLITCH/lib/python2.7/site-packages/scss/__init__.py:86: RuntimeWarning: Scanning acceleration disabled (_speedups not found)!
RuntimeWarning
Starting ForeverFM...
Initializing read queue to hold 240.00 seconds of audio.
Grabbing fresh tracklist from SoundCloud...
Info generator waiting on first frame...
Waiting for a new track.
Starting up tornadio server on port '8193'
Entering IOLoop...
Got 400 tracks in 135.31ms.
Solving TSP on 279 tracks...
Solved TSP in 16508.61ms.
Calling ffmpeg: en-ffmpeg -i /Users/mikekilmer/Envs/GLITCH/glitcher/cache/169217324.mp3 -acodec copy -f null -
Calling ffmpeg: en-ffmpeg -i /Users/mikekilmer/Envs/GLITCH/glitcher/cache/169217324.mp3 -ac 2 -ar 44100 -f s16le -acodec pcm_s16le pipe:1
Fetching analysis...
At this point you can point the browser to http://localhost:8193/ (replace ‘8193’ with the port number specified in your terminal output by: Starting up tornadio server on port ‘8193’, and you’ll see a web page, but no dynamic audio or images yet.
It will look like:
It took me like a day, which included studying NGINX, Node, TornadIO2 and Socket.IO before I realized the front end was actually working. Being a person of little patience, I haven’t confirmed this. but you may beed to actually press the “play” button, then wait a few minutes (maybe go get a drink of water). Actually I think I had left it running by mistake later accidentally noticed that the web page had the track details on it. Safe to say it took a little while to buffer and show us:
Generated waveform in 59.13ms.
Adding track info. Currently holding info for 3 tracks.
Current delta: 2 samples.And…
Queue All drifting by 107.33 ms. Compensating...
Dropping frames! Queue All is starving!
Committing suicide.
Could not broadcast due to:
Traceback (most recent call last):
File "/Users/mikekilmer/Envs/GLITCH/glitcher/forever/server.py", line 183, in stream_frames
cls.relays.broadcast()
File "forever/listeners.py", line 71, in broadcast
sys.exit(RESTART_EXIT_CODE)
SystemExit: 123
200 GET / (127.0.0.1) 48.30ms
200 GET /static/assets/raf-1410822034_front-1412132118.js (127.0.0.1) 51.35ms
200 GET /all.json (127.0.0.1) 0.72ms
200 GET /timing.json (127.0.0.1) 0.75ms
200 GET /socket.io/1/?t=1412180366058 (127.0.0.1) 1.20ms
Added new debug listener at 127.0.0.1.
Removing action that ended at 1412132871 (now is 1412133298).
Calling ffmpeg: en-ffmpeg -i /Users/mikekilmer/Envs/GLITCH/glitcher/cache/170121125.mp3 -ac 2 -ar 44100 -f s16le -acodec pcm_s16le pipe:1
Fetching analysis...
Removing action that ended at 1412133530 (now is 1412134033).
Removed 1 actions.
And in the browser:
Since two years ago is “ancient history” (in ways) for technology, some of his methods will end up being replaced. For example TornadIO2 is no longer supported. So now it’s time to begin deconstructing. What a fun (and daunting) adventure. Man, there’s some smart people out there. Wish me luck.