— Jose Gonzalez

Archive
Projects

Average sentiment score

Twitter simultaneously allows people to be heard and to hear, easily and in real time, bringing with it potentially fascinating and ground breaking insights for anybody trying to take the public’s pulse on a hot issue.  It is almost mandatory that so much benefit comes with its challenges, namely how to process the humongous amounts of data available every minute. Never one to shy away from some super number crunching, this post is the first one of a series on Twitter.

The chart above represents more than 200,000 mentions of the twitter account belonging to Mexico’s President Enrique Peña Nieto during the past 20 days. It is intended as a proxy of how popular the President of Mexico is on Twitter. Every tweet was analyzed and scored as either positive or negative. Then the total number of positive/negative tweets per hour was recorded and divided by the total number of tweets in that hour. The results show, that on average, Peña Nieto has had a positive sentiment score (0.54) but there are severe hourly negative spikes.

If you would like to know more about how the calculations were elaborated, continue reading

Read More

Poverty and crime in Mexico

How safe is Mexico? That is a frequent question people ask me. There is even a  website about it  (see this blog post). So I’ve decided it was time to go one step further and make an interactive map (click here for full screen) of poverty and crime in Mexico.

The objective was to see which municipalities have higher crime rates and visually check if municipalities with higher crime rates also have high poverty rates. While unfortunately,  the most recent poverty data at the municipality level is for 2010 while the crime data, at the same level, is only available for 2011, 2012, 2013, I was still able to glean some interesting insights.

The interactive map´s main take away is that high rates of selected crimes are concentrated in just a few municipalities. Moreover, there appears to be no direct link to high poverty rates.

To do the analysis I used R, QGIS and TileMill, all that code is freely available in my github account. If you are interested in learning more about how I made the map, keep reading!

Read More
Command Line Terminal

Source: Coursera | Startup Engineering

In the past I’ve struggled to install python modules and updates with the terminal on Mac. Recently, I started to use the terminal to connect to AWS and found it quite powerful, although there is certainly a learning curve. Perhaps, one of the best ways to master its use is by learning it the hard way. Check this website that also includes quite powerful training on Python and other languages. Learn to code the hard way: the command line.

Read More

I just converted the Mexico City’.KML file to .shp format suitable for ArcGIS. To get the file click here

Mexico City's Metro Map

Read More

After struggling for a while to highlight Python code without having to install a package or taking screenshots I found this awesome tool:     Online syntax highlighting for “Python”. It is also very useful to highlight code that is freely available in the web. Hope to see R extensions soon.

Another option for WordPress.com is to use the this wrapper. As an example check this Python code to calculate the mean center and deviational ellipses in ArcGIS. The only problem is with the indenting for the loop it has some trouble recognizing the format.

#Calculate the mean center and deviational ellipse for several variables programatically
#Author Jose Gonzalez
import arcpy
from arcpy import env
from datetime import datetime

try:
# Set environment settings
env.workspace = "U:/advGIS/Lab9/Lab9.gdb/Layers"
print "Working"
print datetime.now()
# Local variables:
inFeatures = "master2006cut"
# Process: Select
arcpy.Select_analysis(inFeatures, "Penninsula", ""ENTIDAD" = 4 OR "ENTIDAD" =23 OR "ENTIDAD" =31")
for i in ["VTURNOUT", "PCTVPAN", "PCTVAPM", "PCTVPBT"]:
# Process: Mean Center
arcpy.MeanCenter_stats(inFeatures, i + "_MeanCenter", i, "", "")

# Process: Directional Distribution (Standard Deviational Ellipse)
arcpy.DirectionalDistribution_stats(inFeatures, i + "_DevEllipse", "1_STANDARD_DEVIATION", i, "")

print inFeatures
print datetime.now()
except Exception, e:
# If an error occurred, print line number and error message
import traceback, sys
tb = sys.exc_info()[2]
print "Line %i" % tb.tb_lineno
print e.message

Read More