Archive for : December, 2012

Bitly Data API

Over my Christmas vacation I decided to take on a small programing challenge.  I had stumbled upon the Bitly api a few days earlier and noticed that it provides some information about location of links, etc. so I decided to give it a go!

After looking over the api and data returned, I decided that it would be cool to build a map that displays trending locations on Bitly.  My first task was to see if I could find a python api client library for bitly, since python was going to be my server language of choice.  After looking over the apis listed here, I found that all of the apis were centered mainly around shortening urls and I needed more than that! So I picked up my favorite python rest library (restlib) and started working. I only needed a few api calls, but I figured it would be nice to build a simple library out of it so that I could reuse it.

I started out by creating a Connection class that would contain all of the methods to connect to the api.
[python]
class Connection():
“””Class that represents a connection to the bitly api.”””
access_token=””
_baseUrl=””
def __init__(self,access_token):
“””Creates a connection object with the specified access_token. This token should come from an oauth2 process.”””
self.access_token = access_token
self._baseUrl = “https://api-ssl.bitly.com/v3/”
[/python]

This gives me a simple framework that I can use to create a “connection” to the api, and ask for specific information, without trying to track an access_token though different requests. My first method was to find “hot phrases” as defined by the bitly api /v3/realtime/hot_phrases. This returns a list of trending phrases right now, and the links that are connected to them.

[python]
def realtime_hot_phrases(self):
“””Lists the current realtime hot phrases.”””
url = self._fullUrl(“realtime/hot_phrases”)
params={“access_token”:self.access_token}
result = restclient.GET(url,params)
result = json.JSONDecoder().decode(result)
self._checkStatus(result)
return result[“data”][“phrases”]
[/python]

In order to make my life easier I created a couple of private helper methods to make my life easier. The first is, “_fullUrl”, it takes in the url component that this particular method uses, and returns the full url. This just makes it easier to manage the urls, and I didn’t have to duplicate string concatenation all over the code.

The second is, “_checkStatus”, this method automatically checks the result returned by the http call for any errors that may have been returned by the api.

[python]
def _checkStatus(self,results):
“””Checks the status of a return, and raises Exceptions as appropriate.”””
try:
if results.has_key(“status_code”):
if results[“status_code”] == 200:
return
else:
raise BitlyError(“Status Code is not 200 (%s) (%s)”%(str(results[“status_code”]),str(results[“status_txt”])))
else:
raise BitlyError(“No status code found.”)
except Exception as e:
raise e
[/python]

This function automatically looks for the status code attribute of the result object, if it exists, and it is 200 the function will just return. If the status code is not 200, the a BitlyError is raised with details about the status that was returned. One thing that would make the library a bit more usable would be to add more status code errors, and raise errors with more detailed information about what happened.

With that done, you can easily get the hot links from bitly like this:

[python]
bit = Connection(“”)
print bit.realtime_hot_phrases()
[/python]

I added a few more methods for things like:

  • link_countries – To get the countries that a link is related to
  • link_locations – To get locations (a bit more specific than country) that a link is being used from
  • link_history – Gets the current logged in user’s link history
  • realtime_bursting_phrases – Gets  a list of phrases that are bursting in popularity among links right now

You can download the library here if you want to try it out.

I will add another post later about the final product LinkMap!

Migrating to DreamHost

Just finished moving my site from godaddy.com to dreamhost.com.  So far the transfer has been pretty painless, but all I did was migrate the hosting!

After getting setup, I started working on a project to put some Bitly API data on a map using Django/Python.  I quickly discovered that the version of python on DreamHost was too old, so I set out to install my own.  I found these instructions to be the best: http://www.ricksresources.com/2012/07/install-python-27-dreamhost/

Once you can SSH into your DH server, do the following:

[bash]
[paramaribo]$ mkdir python
[paramaribo]$ cd python
[paramaribo]$ wget http://python.org/ftp/python/2.7.3/Python-2.7.3.tar.bz2
[paramaribo]$ tar xjf Python-2.7.3.tar.bz2
[/bash]

That will download python for you, and extract it.  Now you will need to build Python:

[bash]
[paramaribo]$ cd Python-2.7.3
[paramaribo]$ ./configure –prefix=$HOME/python
[paramaribo]$ make
[/bash]

After running this, you may see some text like this:

[bash]
Python build finished, but the necessary bits to build these modules were not found:
_bsddb _tkinter bsddb185
bz2 dl imageop
sunaudiodev
[/bash]

These Python libraries probably won’t be needed, so don’t worry that they didn’t build. Now run the following to install Python:

[bash]
[paramaribo]$ make install
[/bash]

Once that is done, python should be operational! Run the following to add your new python to your PATH so that this python will always run:

[bash]
[paramaribo]$ echo ‘export PATH=$HOME/python/bin:$PATH’ >> ~/.bash_profile
[/bash]

Restart your SSH session, and try running “python” at the prompt, and you should have python 2.7.3 all up and running! The next step is to set up a virtual environment and get some stuff running.

A python virtual environment lets you have isolated versions of python running with different installed libraries. This makes it really easy to setup different applications on the same server even when they need different versions of python, or different libraries installed.

[bash]
[paramaribo]$ wget http://pypi.python.org/packages/source/v/virtualenv/virtualenv-1.8.4.tar.gz
[paramaribo]$ tar xzf virtualenv-1.8.4.tar.gz
[paramaribo]$ cd virtualenv-1.8.4
[paramaribo]$ python setup.py install
[/bash]

I keep all of my environments in an env folder in my user’s root directory:

[bash]
[paramaribo]$ mkdir env
[paramaribo]$ virtualenv env/linkmap.moravec.net
[bash]

Now, switch to your new environment, install Django, and mysql drivers:

[bash]
[paramaribo]$ source ~/env/linkmap.moravec.net/bin/activate
(linkmap.moravec.net)[paramaribo]$ pip install Django
Downloading/unpacking Django
Downloading Django-1.4.3.tar.gz (7.7MB): 7.7MB downloaded
Running setup.py egg_info for package Django

Installing collected packages: Django
Running setup.py install for Django
changing mode of build/scripts-2.7/django-admin.py from 664 to 775

changing mode of /home/cmoravec/env/linkmap.moravec.net/bin/django-admin.py to 775
Successfully installed Django
Cleaning up…
(linkmap.moravec.net)[paramaribo]$ pip install MySQL-python
Downloading/unpacking MySQL-python
Downloading MySQL-python-1.2.4c1.zip (113kB): 113kB downloaded
Running setup.py egg_info for package MySQL-python
Downloading http://pypi.python.org/packages/source/d/distribute/distribute-0.6.28.tar.gz
Extracting in /tmp/tmpZZ7doS
Now working in /tmp/tmpZZ7doS/distribute-0.6.28
Building a Distribute egg in /home/cmoravec/env/linkmap.moravec.net/build/MySQL-python
/home/cmoravec/env/linkmap.moravec.net/build/MySQL-python/distribute-0.6.28-py2.7.egg

Installing collected packages: MySQL-python
Running setup.py install for MySQL-python
building ‘_mysql’ extension
gcc -pthread -fno-strict-aliasing -g -O2 -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -Dversion_info=(1,2,4,’rc’,5) -D__version__=1.2.4c1 -I/usr/include/mysql -I/home/cmoravec/python/include/python2.7 -c _mysql.c -o build/temp.linux-x86_64-2.7/_mysql.o -DBIG_JOINS=1 -fPIC
gcc -pthread -shared build/temp.linux-x86_64-2.7/_mysql.o -L/usr/lib/mysql -lmysqlclient_r -o build/lib.linux-x86_64-2.7/_mysql.so

Successfully installed MySQL-python
[/bash]

Once that is done, you should be good to go!

For details, try: http://www.ricksresources.com/2012/07/writing-wsgi-apps-on-dreamhost-shared-hosting/