Tag : python

5 ways to make a GUID or UUID in ArcGIS


Globally Unique Identifiers or Universally Unique Identifiers: Humanity’s attempt to make enough unique values to assign one to everything on the Earth. These funny values are the subject of many debates on when to use them and when not to, should we use a GUID or an Integer? What is faster? Can a user remember a GUID (no, no they can not)? etc.

Side note: GUIDs and UUIDs are the same thing, a GUID is just Microsoft’s implementation of a UUID!  For a quick primer on GUIDs and UUIDs I suggest the Wikipedia pages.

We may not agree on when we should use them, or how to pronounce them (GUID seems most commonly pronounced GOO-id, but I don’t say it that way…) we can all agree that they are becoming more a part of our every day lives when dealing with data. As Data Geeks we often want to shield our users from these internal values, but we need them, and generating these values is the subject of much consternation but ultimately we need not be afraid, they are just 128-bit integers and there are simple ways to generate them in most programming languages!

Enough with the discussion, lets see how to generate these things! I picked 5 common languages that I have needed to generate GUIDs in for one reason or another, feel free to skip down to the one that matters to you: ArcMap/Pro (Python), Javascript, SQL, .Net, Swift.

ArcMap/Pro (Python):

Python has this handy library called uuid, we can easily convert this to a string using: "{" + str(uuid.uuid4()).upper() + "}"

Bonus: You can use the above method in a Calculate Field Code Block and simply call the function in the Value box.  For examples on using Esri’s Calculate Field tool, check out: http://resources.arcgis.com/en/help/main/10.2/index.html#//00170000004s000000

For a complete working example, check out this fiddle: http://pythonfiddle.com/makerandomguid


Javascript can be a bit harder to generate a GUID in, however if we are using Esri’s Javascript library, then we have access to the Dojo library already, and we might as well use it! We can require['dojox/uuid/generateRandomUuid'] and from there convert to upper case and add brackets: "{"+uuid().toUpperCase()+"}" Putting it all together:

For a complete working example, check out this fiddle: https://jsfiddle.net/cmq3u7jn/


It seems that automating and building things in SQL is sometimes forgotten, but SQL provides a very powerful platform to store an analyze data, and there is always room to balance what you process in the database vs what you process on the client.  For example, when doing an insert via SQL you can easily generate the GUID that you need.  This example is for SQL Server, but Oracle is pretty close (try using sys_guid).

For a complete working example, check out this fiddle: http://sqlfiddle.com/#!6/9eecb7/1474/0


In a typically simple yet complicated fashion, .Net (c#) makes it pretty easy to generate a guid, it comes down to one line:  System.Guid.NewGuid().ToString("B").ToUpper();

For a complete working example, check out this fiddle: https://dotnetfiddle.net/TdPCoD


Last but not least, Swift.  Apple’s newest excursion into building new programming languages provides a pretty clean and neat syntax and is pretty fun to use!  Generating GUIDs in Swift is as easy as I would expect:  "{"+NSUUID().UUIDString+"}"

For a complete working example, check out this fiddle: http://swiftstub.com/15398662/?v=gm

What to do about “[Shape_STLength__]” Errors?

Oh boy….
Database error: The field is not nullable. [Shape_STLength__]. The geometry is not Z-aware.

That field shouldn’t be there, you can safely remove it. I wrote a quick script to do so here: https://gist.github.com/morehavoc/3e86f535bf204b040ee2

Now for the full story…

It took me a bit to figure this one out, I thought “Hmm, that looks like an Esri field, why would I be populating the Length from my javascript app, shouldn’t the server take care of that for me?” As it turns out I was wrong, that is not an Esri field… but in my case it was caused by Esri.

For me the field was in a feature class that I had created via a Python script using another feature class as a template; that was the issue. Esri, being so helpful, decided to convert the Shape.STLength() field into Shape_STLength__ and to top it off, make it required as well. The field Shape_STLength__ has no use, other than to cause trouble. The Shape.STLength() field is not actually a real field, it is a alias created by Esri to get the length of the geometry, handy right!

Once I figured out that was the issue, I was able to remove all of the Shape_STLength__ and Shape_STArea__ fields using a python script (making sure to shut down my services first) and all was well once again.

You can check out the python script here: https://gist.github.com/morehavoc/3e86f535bf204b040ee2


One line if statements in Python

Every once in a while it is kind of nice to put a one-liner if statement in a Python program.  I don’t use these very often as they can make the code harder to read.  However I do think they have their place.  Mainly I use these when I need set the value of a variable based on a condition.  So lets say I start with a variable called dummy, and set it to None.


dummy = None


And I want to set dummy to something, but only if dummy is still none, so:


dummy = (something if dummy is None else dummy)


This is the same as writing:

if dummy is None:
dummy = something
dummy = dummy

The one line expression is a much simpler way to express this condition.

Installing MYSQL-python on Mac OS X

I like to do my web development on my MacBook; I find it much easier than working on my Windows machine.  Sometimes things don’t go so well when installing things on my Mac, at least not as easy as they are when installing on Linux.  One of these such things is installing MYSQL-python into a virtual-env for use in my Django apps.  When I run it usually get something like this:


(thegame)strix:online_app cmoravec$ pip install mysql-python

File "setup_posix.py", line 25, in mysql_config

raise EnvironmentError("%s not found" % (mysql_config.path,))

EnvironmentError: mysql_config not found


The answer is actually pretty easy, before running pip (or permanently if you wanted) run:


export PATH=$PATH:/usr/local/mysql/bin


This assumes that your mysql is installed in /usr/local/mysql/bin.  I installed MySQL here to get the binaries and headers so that I could install things that depend on them, I actually use MAMP to run my database.  I understand that if you use the bitnami MAMP then you don’t have to go through this trouble, oh well, Live and Learn!

The most usefully link for information about this is from stackoverflow.

Bitly Data API

Over my Christmas vacation I decided to take on a small programing challenge.  I had stumbled upon the Bitly api a few days earlier and noticed that it provides some information about location of links, etc. so I decided to give it a go!

After looking over the api and data returned, I decided that it would be cool to build a map that displays trending locations on Bitly.  My first task was to see if I could find a python api client library for bitly, since python was going to be my server language of choice.  After looking over the apis listed here, I found that all of the apis were centered mainly around shortening urls and I needed more than that! So I picked up my favorite python rest library (restlib) and started working. I only needed a few api calls, but I figured it would be nice to build a simple library out of it so that I could reuse it.

I started out by creating a Connection class that would contain all of the methods to connect to the api.
class Connection():
“””Class that represents a connection to the bitly api.”””
def __init__(self,access_token):
“””Creates a connection object with the specified access_token. This token should come from an oauth2 process.”””
self.access_token = access_token
self._baseUrl = “https://api-ssl.bitly.com/v3/”

This gives me a simple framework that I can use to create a “connection” to the api, and ask for specific information, without trying to track an access_token though different requests. My first method was to find “hot phrases” as defined by the bitly api /v3/realtime/hot_phrases. This returns a list of trending phrases right now, and the links that are connected to them.

def realtime_hot_phrases(self):
“””Lists the current realtime hot phrases.”””
url = self._fullUrl(“realtime/hot_phrases”)
result = restclient.GET(url,params)
result = json.JSONDecoder().decode(result)
return result[“data”][“phrases”]

In order to make my life easier I created a couple of private helper methods to make my life easier. The first is, “_fullUrl”, it takes in the url component that this particular method uses, and returns the full url. This just makes it easier to manage the urls, and I didn’t have to duplicate string concatenation all over the code.

The second is, “_checkStatus”, this method automatically checks the result returned by the http call for any errors that may have been returned by the api.

def _checkStatus(self,results):
“””Checks the status of a return, and raises Exceptions as appropriate.”””
if results.has_key(“status_code”):
if results[“status_code”] == 200:
raise BitlyError(“Status Code is not 200 (%s) (%s)”%(str(results[“status_code”]),str(results[“status_txt”])))
raise BitlyError(“No status code found.”)
except Exception as e:
raise e

This function automatically looks for the status code attribute of the result object, if it exists, and it is 200 the function will just return. If the status code is not 200, the a BitlyError is raised with details about the status that was returned. One thing that would make the library a bit more usable would be to add more status code errors, and raise errors with more detailed information about what happened.

With that done, you can easily get the hot links from bitly like this:

bit = Connection(“”)
print bit.realtime_hot_phrases()

I added a few more methods for things like:

  • link_countries – To get the countries that a link is related to
  • link_locations – To get locations (a bit more specific than country) that a link is being used from
  • link_history – Gets the current logged in user’s link history
  • realtime_bursting_phrases – Gets  a list of phrases that are bursting in popularity among links right now

You can download the library here if you want to try it out.

I will add another post later about the final product LinkMap!