Friday, February 10, 2017

President Trump and the Demise of Net Neutrailty...

Regardless of which political echo chamber you prefer, your social media news feed is likely overtaken with stories and opinions related to President Trump's first few weeks in office. But barely noticed has been, perhaps, the single most consequential and important public policy development of all... the end of the Internet as we know it.

I speak, of course, of Net Neutrality. Readers of The Nerfherder are well aware that I've been trying to raise awareness about this issue for 15 years (!) now.

Here's what you need to know.  The Internet that you've known your entire life has always been neutral. We've had Net Neutrality all this time. It's known as the First Amendment of the Internet, and it's a legal principle that states that all data must be treated equally in terms of how it's routed across the Internet's infrastructure. Whether you're trying to reach Google's website, or a college student's humble little blog, you're going to get there, and in a way that's the same regardless of how well capitalized the destination site happens to be.

That's all about to change, and with hardly anyone saying boo about it.

The telecoms have long wanted to do away with this system. They've lobbied heavily to be able to create an "EZ-Pass toll lane" for Internet traffic, whereby companies and individuals would have to pay untold fortunes in order to have people reach their websites faster, and every other website would be relegated to the sidelines. Scholars have long pointed out the immediate economic impact this would have on startups, entrepreneurship, and innovation - not to mention how it would greatly impair the ability of individuals to express their free speech in a manner where people might actually be able to view what they post.

If you don't have the resources of Google or Microsoft to pay off the telecoms, good luck to you.

During the transition period, President-elect Trump signaled his policy preference against Net Neutrality by naming Jeff Eisenach and Mark Jamison to oversee hiring and policy at the FCC (the agency that sets Net Neutrality policy). These men have histories of lobbying on behalf of the large telecoms and favoring more mergers within the telecom sector (reducing competition).

Then, once in office, President Trump appointed Ajit Pai as the new Chairman of the FCC. In just his first two weeks in office, as reported by the New York Times, Pai has aggressively been assaulting various Net Neutrality rules - stopping nine companies from providing discounted high-speed internet service to low-income individuals, not allowing a scheduled vote to take place that would have overhauled the pay-TV settop box market, and, perhaps most importantly, closed an investigation into whether AT&T, Verizon, and T-Mobile violated the law by giving preferential treatment to some websites and web services over others. This is the definition of violating Net Neutrality, and the FCC Chairman signaled this week that when telecoms do so, there will be no consequences.

Somehow, some way, this issue has become very partisan over the last decade. It shouldn't be.  Republicans ought to support an Internet which values free speech over censorship, which values entrepreneurship and innovation, and encourages a thriving competitive free market. As soon as you get rid of Net Neutrality, the small handful of giant telecoms will have the ability to decide what Internet content people will likely see and which web services people will likely use.

Unfortunately, the telecoms have lobbied for 20 years and have contributed millions of dollars to members of Congress (from both parties) in order to frame the issue as one where "government shouldn't regulate the Internet". But here's the problem... regardless of which side of the debate you support, regulation is the end result. If you are against Net Neutrality, the giant telecoms will regulate how websites operate and what content people will be able to access; meanwhile, if you are pro-neutrality, you're regulating the telcoms and giving a structural advantage to the websites. Either way, you're advocating for the selection of certain winners and losers. I would argue that the "free market" isn't necessarily being served by either, so the question is would you rather have a free marketplace for the telecoms or for cyberspace? Where is competition most likely to occur?

During the presidential transition period, I gave an interview urging people not to be alarmists, though when asked if Net Neutrality was under threat and should supporters be worried, I replied absolutely. The actions of the new FCC Chairman have now made it likely that the end of the Internet as we know it is only weeks away.


Tuesday, January 31, 2017

I just got my Raspberry Pi. Now what?

I recently bought the Raspberry Pi 3 Model B. When it showed up at my doorstep I had no idea how to get started. Most online tutorials jumped straight into beginner projects, but you need to set up and configure the Pi before you can do anything with it. So here goes.

The Raspberry Pi literally comes as just a motherboard, so most likely you'll want to get it as part of a Raspberry Pi "kit" that includes a charger, cables, protective case, and SD Card. Don't spend more than $50 total.

It will look like this...

Now, before we can do anything with the Pi, we have to give it an operating system. For Windows users, download the free Win32DiskImager, and unzip it to your computer. Then, download Raspbian which is the official operating system for Raspberry Pi, and unzip that to your computer as well.

The next step is to open that Disk Imager program, click the blue folder icon, go find your Raspbian image file, and click to "Write" it to your SD card (which you should've put in your computer's SD card reader).

Believe it or not, you're almost finished. The only remaining task is to put that SD card into your Raspberry Pi and turn the Pi on.

Here is definitely something important to be aware of, though.  The first time you break out your Raspberry Pi and want to install your Raspbian operating system, you MUST connect your Pi to a monitor, mouse, and keyboard.  You'll only need to do this the first time you use it, but it's an unavoidable step.  You can't use a laptop either.  Without realizing this, after getting my Pi in the mail, I then had to wait a few days, bring the Pi into my office at work, take over a colleague's USB mouse and keyboard for an hour, and finally (and this was the real challenge) I had to find a monitor that not only used an HDMI cable but also had the right adapter for that HDMI cable (the adapter with the cord didn't work for some still-unknown reason).

But here's the good news... even though hooking the Pi up to a mouse, keyboard and monitor the first time can be a pain, once they're all connected and you simply power it on, the operating system automatically loads, and just like that you have a new pocket-sized computer.

You're totally finished and ready to experiment with all of those beginner Raspberry Pi projects that you see all over the Web.

One last thing, which is optional but highly recommended.  As of now, your Pi is 100% ready to use, but you still need it connected to a keyboard, mouse, and monitor whenever you want to do something with it.  Why not make it "headless"?

To make your Pi headless, you just need to set up some type of Remote Desktop program so that your laptop or main computer can take control of your Pi when you want it to and give you an interface.

I recommend the VNC Viewer.  While your Pi is still hooked up to a monitor, boot it up. What's nice is that the VNC Server software is already built in to Raspbian, so all you need to do is click on Menu > Preferences > Raspberry Pi Configuration > Interfaces, and then click to Enable VNC.  You should also double-click on the VNC Server icon on the top-right of the desktop screen and take note of your IP address. Reboot your Pi. You can now disconnect the mouse, keyboard, and monitor.

Your Pi is headless, so when you want to control it from your laptop or main computer, download and install the VNC Viewer software linked to above, then simply click to Create New Connection and enter your Raspberry Pi's IP address.  It will display a nice Raspbian interface for your Pi so you'll never need to connect those external pieces of hardware again.

You're off to the races!  My first few projects... 1) Turn the Pi into a Web server, 2) turn it into a Minecraft gaming server, 3) turn it into a RetroPi gaming console, and 4) turn it into a Kodi device for use with my TV.


The Privacy Paradox Podcast...

This morning I heard about The Privacy Paradox on NPR radio. It's an "interactive podcast", which as best as I can tell means that after each new podcast episode is released, the listener is directed to complete a series of challenges, and then the next podcast episode will review the listeners' collective results.

In this case, there will be 5 challenges that you can complete with the goal of "taking back your digital identity".  The challenges include finding out what your phone is tracking about you, discovering how algorithms learn about and then sell your identity, reclaiming your privacy, and more.

I don't know if this is already the case, but the thought occurs to me that for a podcast to truly be interactive, rather than having pre-determined content, users' results could help decide what the next round of content ought to be.  For instance, the podcast's guest speakers could be selected based on the privacy findings of the users' first challenge on smartphone surveillance.

Another intriguing aspect of this project is the Privacy Personality Quiz to see if you are a Shrugger, Realist, or Believer.

For the record, I was notified that I'm a Realist.

This seems like a useful project that also seems like it could be both entertaining and fun.  Who wants to join me and then have a discussion?  You can choose whether it would be private or public, of course.


Friday, January 27, 2017

How to Install Minecraft Mods...

When you're ready to step your game up to the next level in Minecraft, it may be time to experiment with "mods".  Mods are not part of the official Minecraft game, but are instead created by other players.  They can make it possible to build unique items, interact with new kinds of creatures like dinosaurs, and even change the game's graphics to 3D. Basically, anything is possible through mods, including changing the basic rules of the Minecraft game itself.

Go explore what's out there at,, and  Here are the steps for installing new mods...

1.  Download and install Minecraft Forge.  This is an API that lets you install mods for Minecraft.  Go to the Minecraft Forge downloads page, click on the most version of Minecraft you have (for example, 1.11.2), and on the next page click "Installer-win".

2.  Run the installer file that you just downloaded.  Make sure that "Install Client" is selected and click "OK".

3.  Now start up Minecraft.  You should notice that in the bottom-left corner, in the drop-down "Profile" menu, you will see a new profile labelled "forge".  Select it and click to "Play".  After a little bit of updating, the Minecraft Game Menu will appear and display a new button labelled "Mods".

4.  At this point, Forge is successfully installed and you are ready to play mods.  However you still need to go get a few.  You can find some you like from  For this example, I'm going to get the JurassiCraft Mod to play in a world with dinosaurs.

5. First download the LLibrary jar file (this won't be necessary for all, or even most, mods, but it is required for JurassiCraft), then download the JurassiCraft jar file (the file links are towards the bottom of these two pages).  Save them both into your Minecraft "mods" folder.

  • If you don't know where your "mods" folder is, do the following...
  • Start Minecraft as if you were about to play a new game (in other words, open the "launcher").
  • In the bottom-left corner, underneath your profile name, click the button labelled "Edit Profile".
  • Take a look  in the field labelled "Game Directory". That is the location of your saves folder. Write it down or remember it. For this example, you can see that my folder is located in "C:\Users\RobbieD\AppData\Roaming\.minecraft".

6. You're finished! When you launch Minecraft and click your new "Mods" button, you should see your new mod (ex. - "JurassiCraft") appear on the left. Play a new game with this mod and you'll have lots of new non-standard features to the game.  Like dinosaurs.


Monday, January 23, 2017

How to Install Minecraft Maps...

When you want to play Minecraft but are ready to branch out beyond the game-generated landscapes, it's time to explore other maps. One of the reasons Minecraft has endless possibilities is because whenever someone creates their world, they can then share it and let the rest of us download it for ourselves. This is called a Minecraft Map.

There are thousands of maps that you can download, install, and play - for free. To get a sense of what types of maps are out there, go explore

Here is the process:

1. Find a map that you want and download it. For this tutorial, I am going to download the map titled, "Escaping!", which is one of the most popular "puzzle" maps. Click where it says "Download Maps". In the pop-up window that will appear, click to Save the zip file.

2. You will need to unzip those new files into your Minecraft "saves" folder, so first you need to locate it on your hard drive. To locate your "saves" folder do the following...
  • Start Minecraft as if you were about to play a new game (in other words, open the "launcher").
  • In the bottom-left corner, underneath your profile name, click the button labelled "Edit Profile".
  • Take a look  in the field labelled "Game Directory". That is the location of your saves folder. Write it down or remember it. For this example, you can see that my folder is located in "C:\Users\RobbieD\AppData\Roaming\.minecraft".

3. Now that you know the location of your "saves" folder, go to Windows Explorer, open your "Downloads" folder, and double-click the map you downloaded earlier called "Escaping_v1.1" (or something close to that).

4. From the top menu in Windows Explorer, click "Extract", then "Extract All". Then "Browse" to find your "saves" folder, and click "Extract".

That should be it!  Just restart Minecraft, start a new Singleplayer game, and "Escaping!" (or whatever Map you chose) will display in the list of worlds for you to play. And best of all, you can repeat these same steps for hundreds of different Maps.


Saturday, December 31, 2016

Best Internet Politics Quick-Reads of 2016...

Make yourself more interesting at your New Year's Eve party with these...


Tuesday, November 08, 2016

Election Day Data: Twitter Sentiment

Academics and researchers will be looking this Election Day for any data that might indicate to what extent social media can help predict electoral outcomes. Towards this end, I've created my own software that analyzes Twitter sentiment for each presidential candidate.

Here's how it works. Written in Python, it uses the Twitter Streaming API to collect thousands of live tweets per minute about either Hillary Clinton or Donald Trump. The software then pipes all of those tweets through IBM's Artificial Intelligence engine (the Alchemy API) to determine, for each tweet, if the sentiment is Positive, Negative, or Neutral about that candidate.

Take this data only for what it is and avoid reading too much into it. For instance, it may say more about Twitter users than about the political candidates; it also may say more about the quality of IBM's language analysis algorithms than about actual sentiment; and more weaknesses abound.

That said, here's the data...

Of people tweeting about Hillary Clinton on this morning of Election Day, the sentiment is:

  • 19.5% Positive
  • 42.3% Negative
  • 38.3% Neutral
 Notably, that is a wider spread between her positives and negatives than she's had in the past (and not a promising one).


Election Day Data: The Betting Markets

To save some data for posterity, here is what the political betting markets have had to say about today's presidential election.

As of this morning (Election Day)...

(if you're wondering why these numbers add up to more than 100, remember these are betting markets, not percentages.)

For comparison, public polling currently shows Clinton leading Trump by only a 45.5% - 42.2% margin.

Also, here is a a chart illustrating how the betting markets have fluctuated for the two candidates over the past 90 days leading up to the election...

More to come.


Wednesday, September 21, 2016

What Ted Cruz Doesn't Understand About 'Controlling the Internet'...

Senator Ted Cruz is once again threatening a shutdown of the United States government - this time for the reason that the U.S. shouldn't "give foreign governments greater control over the Internet".

What is he talking about?

On October 1st, the Internet Corporation for Assigned Names and Numbers (ICANN), is scheduled to be turned over to an international consortium group for management. Cruz has made bold statements about how this is a national security issue that will lead to widespread censorship and enable countries like Russia and China to make decisions about what can and cannot be posted online.

This is completely ridiculous, and here's why. ICANN doesn't manage the Internet. All ICANN does is manage the Internet's domain name system. When you want to buy a domain name like "", they simply keep track of the fact that now you are the owner of that domain name. They have no effect whatsoever on how the Internet functions beyond maintaining a list linking IP addresses to domain names.

Cruz has Internet experts and computer scientists scratching their heads in bewilderment as to how little he understands the technical dimensions of this issue. Even Tim Berners-Lee, the creator of the World Wide Web itself, wrote an op-ed today stating, in no uncertain terms, just how wrong he is.

Here's the bottom line - ICANN and the domain name system play no role in Internet censorship. When the Internet is censored, it is accomplished by national firewalls (See Russia, China, etc.), through ISPs, and, as Cory Doctorow points out, massive armies of trolls for hire and other astroturfers, as well as copyright abuse.

So where is Cruz even getting these ideas? My guess is that he's reading headlines about how "Internet Governance" is about to shift out of U.S. control. What he doesn't realize is that the phrase "Internet Governance" has a special meaning in academia, referring solely to governance of the domain name system. It appears nobody informed him of this, and he made a common assumption based on the phrasing, which happens to be completely wrong.


Tuesday, September 20, 2016

Do Algorithms Have a 'White Guy' Problem?

Algorithms are coming to dominate many aspects of our lives and most people are completely unaware of it. They are being used to determine which job applicants actually get called for an interview, which people are awarded a mortgage or business loan, which stores will open in a community, which news headlines and opinions we're likely to see on social media, and much more.

The question that has made headlines recently is whether these algorithms are biased and discriminatory. Some like ProPublica's Julie Angwin and Microsoft's Kate Crawford have argued that algorithms and Artificial Intelligence have a "white guy problem" - meaning that, since algorithms learn by being fed certain data, and they build models of the world based on that data, the people doing the feeding are tremendously important because as they make selections informed by their own biases, the resulting software comes to incorporate those same biases.

They cite these recent examples:
  • "Predictive policing" in places like New York is used to forecast where crime is most likely to occur and directs police to focus on  those neighborhoods. Crawford states that, 
At the very least, this software risks perpetuating an already vicious cycle, in which the police increase their presence in the same places they are already policing (or overpolicing), thus ensuring that more arrests come from those areas. In the United States, this could result in more surveillance in traditionally poorer, nonwhite neighborhoods, while wealthy, whiter neighborhoods are scrutinized even less. Predictive programs are only as good as the data they are trained on, and that data has a complex history. 

From my own anecdotal perspective, I remember once when my wife and I were sitting next to each other on the couch - both using Facebook on our laptops - and she was shown ads for diaper coupons while I was shown ads for getting a second job; nevermind that I'm the stay-at-home parent and she's the one who works full-time. How's that for gender stereotypes being ingrained in an algorithm?

While these biases clearly exist, it's more difficult to prescribe realistic remedies. Some ideas thrown out there include making the algorithms more transparent and accountable - the idea of "algorithmic auditing". Also, since algorithms reflect the values of its creators, there's the idea that we must address not only the people who design them, but also the people who sit on the company boards, and which ethical perspectives are to be included.

It seems to me that the real danger isn't that algorithms are being used in the areas that they are, but that they are being used in ways that overlook the merits of the individual in favor of larger group associations. For instance, credit scores (long determined by algorithms) are based on whether each individual pays their bills on time, their individual history, their individual access to more credit as needed, etc. While still flawed, at least these credit scores are based on what you do as a person, so you have some level of control over your score.

By contrast, these other algorithms in the news recently are more problematic because a person's "scores" are completely dependent on what neighborhood you live in and what racial/ethnic/religious group you were born into. In other words, unlike credit scores, these algorithms are explicitly designed and programmed to make judgments about you based on group stereotypes rather than on the actions you take and decisions you make as an individual.

All of us would like to be defined by what we do as individuals, not by what other people "similarly situated" to us do. There should be universal condemnation of a system in which an upstanding and responsible person can't get a mortgage or business loan solely because their neighbor defaulted on theirs.

The good news is that since algorithms are programmed by people, people can program them differently. If values like meritocracy and individualism are important then they can become the constitutionally defining principles of algorithms going forward. Right now, though, they are a problem, reinforcing the worst stereotypes and acting as an obstacle to individual advancement. As Alistair Croll has famously said, "this is our generation's civil right's issue, and we don't know it".


Sunday, July 31, 2016

Do the Political Party Conventions Matter? Not according to the Betting Markets...

Think what you will about the party conventions that just wrapped up, but one thing worth pointing out is that, despite the so-called boost in polling that candidates receive coming out of their convention, temporary as it may be, this boost doesn't really exist if you look at a different metric than polls - the political betting markets.

Political betting markets have often proved to be a better predictor of electoral outcomes than public opinion polls, particularly the further away the analysis is done from Election Day.  So with the Republican and now Democratic conventions just completed, this is what the betting markets have had to say regarding who will win the presidential election...

Click image to enlarge

This is a 30-day snapshot with the conventions occurring during the second half of the chart.  Do you notice those drastic swings for and against Hillary Clinton and Donald Trump?  If not, you're not alone.  There is virtually no discernible change for either candidate.

For the record, as of July 31st, the betting markets favor Hillary Clinton winning the presidency over Donald Trump by a 68 to 35 margin (these are not percentages but rather how many cents it would cost per dollar to "Buy Yes" for each candidate).

So do the conventions matter?  According to the betting markets, not in the slightest.


Thursday, July 28, 2016

Basic Minecraft Programming Examples...

In the last post we reviewed how to set up Python for Minecraft.  Once you have it up and running, jump right in and try these two simple scripts that not only demonstrate what Python-for-Minecraft programs look like, but are also both really useful when you're actually playing.

Example 1:  Teleport to your starting location (or any location)

As most Minecrcaft newbies and veterans alike can tell you, sometimes you just want to go back to your "home base" (wherever in your world that might be) without needing to hike all the way there or even needing to figure out where it is.  Wouldn't it be helpful to have an easy way to teleport there whenever you wanted?

As soon as you start a new game, click on "F3" to see lots of in-game details display on the screen.  Pay particular attention to the three "Block" numbers (or XYZ numbers).  Then try this...

from mcpi.minecraft import Minecraft
mc = Minecraft.create()

# Replace these numbers with your three starting "Block" numbers

x = 20
y = 80
z = 16

mc.player.setTilePos(x, y, z)

Example 2:  Instantly build a shelter

Most players have to spend their first day gathering materials to build a shelter before nightfall.  But you can try this instead whenever you want to instantly build yourself a cobblestone shelter right next to your present location.  You can even build it in the sky!

# Get player's location and build a hollow cuboid

from mcpi.minecraft import Minecraft
mc = Minecraft.create()
pos = mc.player.getPos()

x = pos.x
y = pos.y
z = pos.z
width = 10
height = 5
length = 6
blockType = 4
air = 0

mc.setBlocks(x, y, z, x + width, y + height, z + length, blockType)
mc.setBlocks(x+1, y+1, z+1, x + width-1, y + height-1, z + length-1, air)

The result...


Wednesday, July 13, 2016

How to Set Up Python for Minecraft...

Most of the online tutorials on this topic seem weak, so here is my step-by-step beginner's tutorial on how to set up your Windows PC so that you can start writing Python programs for Minecraft.

Step #1:  Download and install Python (use any version after 3.5.0).  During installation, make sure to check the box to "Add Python to Path".

Step #2:  Download and install the Minecraft Python API and Spigot.  Go to this link and download the file: "Minecraft".  Extract all of the files, then double-click the file: "Install_API".

Step #3:  Run the Spigot Server.  Open the "Minecraft Tools" folder (that you extracted in Step #2), then double-click the file: "Start_Server".  Without closing the Spigot window, open Minecraft and choose "Multiplayer", then "Add Server".  In the server address box, type "localhost", then "Done".  Double-click the server name listed and your game should start.
  • If you have a compatibility issue when you double-click the server name, exit back to the Minecraft launcher and create a new profile. Edit the settings for that profile by selecting an older version release (for example, 1.9.4) 
You're now totally set up.  In the future, when you want to write the code for your programs, use IDLE (a text editor installed when you installed Python).  In order to execute your finished programs, make sure to run the Spigot server and Minecraft, and simply click "F5" within IDLE.

Some actual programming examples are coming soon.


Thursday, July 07, 2016

Using Tableau for Political Visualizations...

If you're curious about how Data Science is being used to analyze political data, look no further than Tableau - an easy-to-use tool for converting analytics into fabulous visualizations.

Recently, Tableau held a Politics Viz Contest to showcase some of the work their users are generating.  The contest winner, Robert Rouse, used data from a Pew Research Center report to visualize increasing political polarization in the U.S. over the past few decades (Click on the image to enlarge and see the details):

Or here is one that I personally found interesting - a visualization of how each state has voted in partisan terms since 1964:

Or this alternative visualization of state-by-state partisanship in every election since 1916:

Really, if the conjunction of Data Science and Politics is your jam, then it's worth checking out the entire Tableau gallery to see what people are doing in the field.


Tuesday, June 28, 2016

Brexit Voters Were Not Actually Googling "What is the EU?" After They Voted...

The day after the historic "Brexit" vote, where British citizens voted in a public referendum to leave the European Union, a headline from the Washington Post circulated wildly on social media:  "The British Are Frantically Googling What the E.U. Is, Hours After Voting To Leave It".

This was used to ridicule the validity of the results, insinuating that British voters had no idea what they were even voting for or against.  However, the headline is completely disingenuous.

Here's why.  The Washington Post reporter used Google Trends as the source of his information, and noted that the number of searches for the phrase "What happens if we leave the EU" had more than tripled in the hours after polls had closed.  But Google Trends reports search numbers in relative terms and within the date range and context of other trends.

So, for example, the "250% spike" in searches for the E.U. "in the past hour" only means that the number of searches more than tripled relative to the number of searches in the previous hours.  More importantly, it says absolutely nothing about the total number of searches actually made.

What this means is that a very small number of people conducting their searches can lead to Google Trends (or, more accurately, a journalist using Google Trends) completely misrepresenting how many people are doing those searches.  For instance, if 10 people had searched that phrase in the previous hour, and now 25 people searched that phrase in the current hour, then Google Trends would note that it was a 250% increase.  Although, clearly, 25 people hardly represent the entire U.K. with its tens of millions of people.

And, in fact, this is what happened the other night.  As Remy Smith uncovered, in the month leading up to the vote, that phrase was receiving about 261 searches per day.  Even after tripling, that still means that fewer than 1,000 individuals actually googled “What is the EU” in response to the “Leave” victory. And that is hardly enough to conclude that large swaths of the population were generally uninformed.

As if any more context were needed, consider that more people googled "Game of Thrones" that night more than "What is the EU?".  By a lot.


Thursday, May 05, 2016

Results for the Presidential 'Facebook Primary'...

Now that Donald Trump is officially the presumptive Republican nominee and Hillary Clinton is basically, if not yet officially, the same on the Democratic side, it might be instructive to save, for posterity's sake, the results of Nate Silver's "Facebook Primary" data.

First, let's look at the Democratic campaign.  In terms of raw Facebook "likes", Bernie Sanders beats Hillary Clinton nationwide by a nearly 3-to-1 margin.  Yet Hillary Clinton is currently leading Bernie Sanders in the popular vote by more than 3 million votes and a 56%-42% margin.

Number of raw Facebook "likes" for each candidate, by county (Sanders in purple, Clinton in green)...

Another interesting metric is which candidates perform best on Facebook by county relative to their national share of "likes".  In other words, this map shows each candidates' strongholds of support (Sanders in purple, Clinton in green)...

Next, let's look at the Republican campaign, where, in terms of raw Facebook "likes", Donald Trump has more than all other candidates combined (Trump in orange, Cruz in red, and no other candidate registers enough likes to be on the map)...

And, again, here's a map showing each candidates' strongholds of support (Trump in orange, Cruz in red, Kasich in purple)...

The million-dollar question, of course, is to what extent Facebook can be a predictor of electoral success.  Based on the numbers, it hasn't been a very good predictor in the Democratic race, but has been a good one in the Republican race.  That's a ratio of 1:2, or a 50% rate of accuracy.  You don't need to be a statistician to see how 50% accuracy in predicting elections essentially means that it's not accurate at all.


Monday, March 21, 2016

CUNY's Women in Technology Initiative...

We all hope that on some rare occasion we will be fortunate enough to work on something more meaningful, more impactful, than the day-to-day routine aspects of our jobs.  This past January I was fortunate enough to be given the opportunity to co-teach a pilot course with Professor Zachary Dodds that aimed to tackle the challenge of why there are so few female students in Computer Science, and then set out to do something about it.  This effort has now been officially launched as the Women in Technology and Entrepreneurship in New York Initiative.

Please check out the good work the initiative is pursuing at the link above, and check out the video embedded here - you'll not only watch video testimonials from our former students (who represent themselves extremely well, if I may say so), but you'll also catch a fleeting glimpse of yours truly giving a lecture.

It is an honor to be part of this program, and it's the type of thing that I'm grateful that my kids will one day see that I was a part of.


Thursday, March 17, 2016

Who is Anonymous Fighting For in Their War Against Donald Trump?

The hacker group Anonymous has declared "total war" on Donald Trump, urging hackers to expose personal information about him and his staff and to disable various websites associated with him including,,,, etc.

Even though #OpTrump mentions April 1st as its target date, the collective has already posted unverified personal information about the candidate and his staff, including social security numbers.  Trump was also previously targeted by Anonymous last December following his comments about banning Muslims from entering the country.

The reason for the "call-to-arms", as stated in the Anonymous YouTube video, is that “Your inconsistent and hateful campaign has not only shocked the United States of America, you have shocked the entire planet with your appalling actions and ideals".

This action shouldn't surprise anyone because this is basically what Anonymous does.  What's interesting is that this time, rather than go after targets affiliated with the Ku Klux Klan or ISIS, or even Mastercard or Visa, as they have done in the past, they're going after an American presidential candidate who's currently in the process of winning the nomination based on the popular vote.

You don't need to be a Trump supporter to take issue with the fact that, whereas in the past Anonymous has chosen institutional targets like multi-national corporations or terrorist organizations, here they are going after an individual who is being democratically elected.  Rather than being an agent of The People, as they like to perceive themselves, they are actually fighting against what The People are voting for (on the Republican side).

Is Anonymous exposing itself as ideologically liberal, or perhaps even outright partisan?  If they seek to fight and resist the valid results of democratic elections then Anonymous is now showing itself to be anti-democratic in addition to anti-capitalist and anti-free speech (at least, anti-any-speech-that-they-disagree-with).  If that's the case, can someone please explain who exactly they are fighting for?


Thursday, March 03, 2016

The Python Elections Library and Other Programming Tools for Election Analysis...

During this campaign season, many political scientists are conducting research experiments that use online datasets as their primary units of analysis.  For instance, in this blog's previous post, real-time Twitter data was collected and then analyzed to get a sense of popular sentiment about Chris Christie's endorsement of Donald Trump.  Scraping data from social media sites is an increasingly common focus of statistical research.

To this end, there are numerous programming tools available to political scientists.  Let me start by saying that, personally, my language of choice for tasks of this sort has long been Java, although this year I've migrated over to Python.  Meanwhile, Web APIs have been around for years so that researchers can easily access the data on Twitter, Facebook, Google, Reddit, etc. Also, for this 2016 election cycle there are a few notable new additions to our collective toolkit that, combined with some useful already-existing tools, really improve a researcher's capabilities.  Here's what I'm using...

  • Tweepy - still the best Python-based library for accessing the Twitter API.
  • Alchemy and NLTK - the most common APIs for language sentiment analysis.
  • The Python Elections Library - a pay-for library that provides access to all federal, state, and local election results, as well as delegate estimates for the presidential nomination contests.
  • Elex - the Associated Press' brand new command-line tool for accessing current election data.
  • The Watson Emotion Analysis API - IBM's brand new API that was just released in Beta. Whereas Alchemy analyzes language sentiment as positive, negative, or neutral, the Emotion Analysis API detects joy, fear, anger, sadness, and disgust, and rates them by order of magnitude.
  • Matplotlib - the common Python library for visualizing the data with charts and graphs (although I'm currently searching for a better visualization tool, if you have recommendations).

These tools are a great starting point for collecting and analyzing social media data.  Now if only you didn't have to pay a fee for some of these large datasets, or for a sentiment analyses of them, then we'd really be cooking.


Friday, February 26, 2016

What Do Twitter Users Think of Trump Being Endorsed by Christie?

Just an hour or so ago, New Jersey Governor Chris Christie publicly endorsed Donald Trump for president.  This is turning a lot of heads in pundit circles, however I wanted to just do a quick analysis of what people were saying about the endorsement on Twitter.

Running a python script of my own creation, I scraped 949 tweets referring to Trump over the course of a single minute (approximately an hour after the public endorsement).  So this is neither a scientific nor a meaningfully large sample size by any means; rather, I just wanted to get a quick snapshot.  All 949 tweets were then run through IBM's Alchemy engine for sentiment analysis so that we could get a sense of whether tweets about Trump were singing his praises, condemning him, or staking out a more neutral stance.  Here are the data collected:

  • Number of tweets analyzed:  949
  • Number of positive tweets:  254
  • Number of negative tweets:  343
  • Number of neutral tweets:  305

Which can be visualized as follows:

Just some real-time API-driven sentiment for your consumption  :-)