Marc Neuwirth's Blog

Creating the Earth with D3.js

Marc — Sun, 24 Jun 2012 15:41:21 +0000

There is no way around it, creating data visualizations with D3 is just plain fun. Lately, I have especially enjoyed building map visualizations, but I hadn’t figured out an excuse to use the azimuthal projection on a project.

I still haven’t, but this Earth is pretty cool looking:

Click and drag to pan, use scroll wheel to zoom Fullscreen

There are really three parts working here:

The actual map, which is an orthographic azimuthal projection of the countries of the world onto the outside of a sphere.
The stars and outer space, which is a equidistant azimuthal projection onto the inside of a larger sphere
A giant circle with the same diameter as the map sphere for background gradient and a Gaussian blur (SVG Filter)

Map

The map itself is pretty simple, each country is a GeoJSON ‘feature’ polygon which is just a list of lat/lon points that form the borders of the country:

{
  "type":"Feature",
  "properties":{
    "name":"Cyprus"
  },
  "geometry":{
    "type":"Polygon",
    "coordinates":[
      [
        [33.973617,35.058506],
        [34.004881,34.978098],
        [32.979827,34.571869],
        [32.490296,34.701655],
        [32.256667,35.103232],
        [32.73178,35.140026],
        [32.919572,35.087833],
        [33.190977,35.173125],
        [33.383833,35.162712],
        [33.455922,35.101424],
        [33.475817,35.000345],
        [33.525685,35.038688],
        [33.675392,35.017863],
        [33.86644,35.093595],
        [33.973617,35.058506]
      ]
    ]
  },
  "id":"CYP"
}

Each country is mapped to a spot on the page according to the projection, and the paths are clipped so that only the ones on the visible side of the sphere are shown.

Stars

Outer space is the inside of a sphere that is three times the size of the Earth itself. It is also another map projection, but instead of countries, there are 300 randomly placed points of varying sizes. These points are supposed to look like stars and create the backdrop for the Earth.

Polish

In order to make this look more like the Earth, there is a single SVG circle element behind all of the map data. The circle uses a SVG linearGradient and filter for a little bit of a 3D spherical look and the fuzzily defined edge caused by the atmosphere.

Moving Around

D3 makes panning and zooming pretty easy to figure out with maps, however it is mostly tuned for the more often used projections like Albers and Mercator, which use the translate property instead of the origin to control the focal point. That works great for those projections because it allows the scrollwheel to zoom in on the mouse pointer as it does in Google Maps.

Unfortunately, that paradigm doesn’t work as well for this azimuthal projection, so we need to detach the pan and zoom events from affecting each other. This isn’t currently an option in D3, so I have a pull request with a fix that will hopefully be merged soon. Until then, I am using a custom version with my change already merged.

Now that panning and zooming are separated, the move function will calculate the origin and scale based on the mouse event and change the projections for the map and outer space accordingly. Then, the map will be redrawn using the new projections. This creates a smooth experience that allows your to pan and zoom around the Earth.

Feel free to play with the demo above and look through the code below or fork it on GitHub.

Solving Problems Subscribing to Facebook Real-time Updates

Marc — Thu, 08 Dec 2011 06:11:43 +0000

Facebook’s Real-time Updates documentation is not very explicit on how to subscribe to a feed, especially when all I am looking for is an example. So after figuring out what OAuthException (#15) "This method must be called with an app access_token" was trying to tell me and fixing the problem, I figured I would share my PHP example here. Hopefully, it will be helpful to you.* There are two parts that you need to set up:

Setup an endpoint for the subscription
Post to Facebook and tell them you are subscribing

Get an access_token
POST to Facebook with cURL

1. Setup an Endpoint

This part is actually pretty easy because Facebook actually gives you example code (yay!). Put that file on a web server somewhere with an external ip so that Facebook can reach it. Replace 'abc' in define('VERIFY_TOKEN', 'abc');. Save this value for later.

2. Post to Facebook

Now comes the part that is not well documented. In order to subscribe you need to first specifically get an access_token, then POST to Facebook.

A. Get an `access_token`

$app_id = APP_ID;
$secret = APPLICATION_SECRET;
$token_url =    "https://graph.facebook.com/oauth/access_token?" .
        "client_id=" . $app_id .
        "&client_secret=" . $secret .
        "&grant_type=client_credentials";
$response = @file_get_contents($token_url);
$params = null;
parse_str($response, $params);

B. POST to Facebook with cURL

Remember when we defined the VERIFY_TOKEN earlier? Replace VERIFY_TOKEN with that value in this code, and ENDPOINT_URLwith the url of the file from part 1.

$base_url = "https://graph.facebook.com/$app_id/subscriptions";
$url = $base_url . "?access_token=" . $params['access_token'];
$fields_string = '';
$fields = array(
            'callback_url=ENDPOINT_URL',
            'object=user',
            'verify_token=VERIFY_TOKEN',
            'fields=feed'
        );
$fields_string = implode("&", $fields);
//open connection
$ch = curl_init();
//set the url, number of POST vars, POST data
curl_setopt($ch,CURLOPT_URL,$url);
curl_setopt($ch,CURLOPT_POST,count($fields));
curl_setopt($ch,CURLOPT_POSTFIELDS,$fields_string);
//execute post
$result = curl_exec($ch);
//close connection
curl_close($ch);

Once you visit this page successfully, Facebook will call the callback_url defined, verify that the VERIFY_TOKEN in both parts match, and then start sending Real-time Updates to your endpoint.

That was easy.

*By the time you find this post and need help, you can either learn from my example code or, more likely, bang your head against a wall once you realize Facebook has changed its API once again.

Building the Hancock Tower in CSS

Marc — Mon, 25 Jul 2011 14:14:41 +0000

CSS Hancock Tower Demo

Still feeling inspired by Building a CSS City, I recreated another Boston building using CSS transforms this weekend: the Hancock Tower. This one was a bit tougher than building the Prudential Tower because it is more than just a few different size boxes stacked on top of each other.

Creating the main body of the building wasn’t too hard, except for the inset parts on the thin sides. The base, on the other hand, was a pretty big pain. Fitting the sides together and sizing a top was a bit of a struggle (and probably mostly unnecessary because no one really notices that part of the building), but I think it turned out pretty well. The roof of the base flickers when spinning around the back of the building in Chrome for some reason, but appears perfectly in Safari.

After getting the building shape together I added the glass window texture and the darker row on the top of the building. Also, I think the coolest part of the Hancock Tower is how it reflects whatever is around it, so I added a cloud photo that rotates as the building moves around.

Originally I though there would be way less DOM elements on this model, but it turned out to be around the same as the Prudential Tower because of all the details:

Take a look at the Hancock Tower demo in a webkit browser (Safari looks better than Chrome if you have the option).

Building the Prudential Tower in CSS

Marc — Tue, 19 Jul 2011 13:45:13 +0000

CSS Prudential Tower Demo

Last week I saw an awesome post by Matteo Spinelli on Building a CSS City and loved the idea. Since I am somewhat obsessed with trying to build things with 3D transforms (even though they are very impractical at this point), I decided to attempt the Prudential Center. The concept was basically the same as his, except instead of building out I am building up.

The tower is built of 4 blocks stacked on top of each other: the spire, hub, neck and floors. Each block is essentially a cube of a certain size positioned using CSS Transforms and skinned with a texture. The proportions of the building are not quite right, but I couldn’t find them anywhere easily enough.

The markup is pretty simple:

And CSS for the main section of the building:

.block {
    margin: 0 auto;
    width:100px;
    height:500px;
    -webkit-transform-style:preserve-3d;
}
.block div {
    position:absolute;
    width:100px;
    height:500px;
    background: url('images/pru-side.jpg');
}
.floors .top {
    height: 100px;
}
.side1 {
    -webkit-transform:rotateY(90deg) translate3d(-100px,0,100px);
    -webkit-transform-origin:0 100% 0;
}
.side2 {
    -webkit-transform:rotateY(-90deg);
    -webkit-transform-origin:0 0 0;
}
.side3 {
    -webkit-transform:translate3d(0,0,100px);
}
.side4 {
    -webkit-transform: rotateY(180deg);
}
.top {
    -webkit-transform:rotateX(90deg) ;
    -webkit-transform-origin:100% 0;
}
.bottom {
    -webkit-transform: rotateX(90deg) rotateZ(90deg);
    -webkit-transform-origin:0 100%;
}

The other sections use similar transforms, just with different sizes and offsets to align them in the center. Take a look at the Final Demo in any webkit browser.

Revisiting the jQuery UI Time Slider

Marc — Mon, 23 May 2011 01:44:31 +0000

Last year I wrote a post on how to use the jQuery UI slider widget for time ranges. I got emailed a question about it two weeks ago and looking back at the code, I realized there were many things that could be done much better. So I decided to rewrite the code as a jQuery UI plugin wrapping the slider widget and put it on GitHub.

Hopefully the code is now cleaner, more efficient, and easier to understand. You can see the demo here

Slideshow with CSS 3D Transforms Part 2: Playing with Shapes

Marc — Fri, 08 Apr 2011 15:19:21 +0000

So there were a couple of issues in Part 1:

1) I was using ‘class’ as an element in the options object and threw an error in Safari.

2) The rest of the page besides the gallery looked really uninteresting.

3) A ring is the only shape that we could make.

Luckily, since then I have fixed all of those things plus added a couple more improvements. Obviously it is nice that this will work for people with Safari and the page is not as terrible looking, but the most interesting part is the new shapes.

While a ring was cool, I figured that now that we had the framework for generating a 3d slideshow gallery, it should be pretty simple to extend this to include a bunch of different shapes: Vertical, Cylinder, Ribbon, Double Helix, Coil and, of course, Ring.

I was able to create these 8 different shapes using 4 parameters. Flat controls whether each slide will be a different height or even with its neighbors. Vertical controls if the rotation is on the x-axis or y-axis. Rows creates multiple smaller circles based on the number. Helix controls how many streams there are. So the default ring woud be flat, 1 row, 1 helix and not vertical.

getShape: function (type) {
    switch (type) {
    case 'Vertical':
        return {
            flat: true,
            rows: 1,
            helix: 1,
            vertical: true
        };
    case 'Cylinder':
        return {
            flat: true,
            rows: 3,
            helix: 1,
            vertical: false
        };
    case 'Tall Cylinder':
        return {
            flat: true,
            rows: 6,
            helix: 1,
            vertical: false
        };
    case 'Ribbon':
        return {
            flat: false,
            rows: 2,
            helix: 1,
            vertical: false
        };
    case 'Double Helix':
        return {
            flat: false,
            rows: 1,
            helix: 2,
            vertical: false
        };
    case 'Coil':
        return {
            flat: false,
            rows: 2,
            helix: 0,
            vertical: false
        };
    case 'Big Coil':
        return {
            flat: false,
            rows: 1,
            helix: 0,
            vertical: false
        };
    case 'Ring':
        return {
            flat: true,
            row: 1,
            helix: 1,
            vertical: false
        };
    default:
        return {};
    }
}

For each of those settings, some work is happening in the transform function to change the rotation value or the amount the picture is translated in either the Z or Y direction:

var rotate = ((length / rows) - count) * 360 / (length / rows),
    translateZ = length / rows * width,
    translateY = 0;
if (!flat) {
    if (helix > 1) {
        translateY = count % helix * 2;
        translateZ /= 2;
    } else {
        translateY = count / 4;
    }
} else {
    if (rows > 1) {
        translateY = parseInt((count - 1) / (length / rows), 10) * 1.5;
    }
}
translateY *= height;
rotate = rotate % 360;
if (rotate > 180) {
    rotate -= 360;
}

Go over to the demo page and try out some of the different shapes with different numbers of pictures. It is interesting to see which handle many pictures really well. The Double Helix, Ring and Vertical are pretty unwieldy at around 100 pictures, but look cool with fewer. The Cylinder looks ok with less than 30 pictures, but I think it really shines with more. Try the Cylinder or Tall Cylinder with 120+ pictures and it looks awesome, everything is still easy to see and navigate. I like the Coil and Big Coil also, they look pretty good no matter how many pictures, but they take up a ton of room for the amount of pictures they show at one time. Ribbon isn’t all that exciting, but it looks nice enough and it doesn’t take up too much room.

One nice little addition I added in was the left and right arrow keys should now function to move you through the photos as you would expect. This makes navigation a little bit easier and is also a nice way to move through some of these new shapes in a way that wasn’t really necessary with just the Ring.

Make sure you take a look at the finished demo (or the original demo) and get the source on github. Which shapes do you like the best?

Enabling 3d transforms in Chrome:
Type in about:flags in your address bar, enable GPU Accelerated Compositing and restart your browser.
If it is still not working, navigate to about:gpu. It appears that this won’t work on some Windows XP machines saying: “Hardware Acceleration Unavailable NVIDIA drivers older than 257.21 on Windows XP are assumed to be buggy”

Creating a Slideshow using CSS3 3D Transforms

Marc — Tue, 05 Apr 2011 13:41:40 +0000

This weekend I was creating a couple fun ideas with Zoomooz.js, an awesome jQuery plugin for zooming to elements on a page, and was bummed to see that it couldn’t handle 3D transforms yet. After seeing Paul Hayes 3D Sphere and the Morphing Power Cubes, I spent a bunch of time playing around with 3D CSS transforms without coming up with anything too practical. But, after some of my Zoomooz experiments, I thought it would be interesting to see if it would be possible to create a scalable 3D slideshow in the model of the Morphing Power Cubes’ ring setup. Unfortunately this only works in webkit browsers because that is the only one that has 3D transform support as of this moment, so I am using Modernizr to test if your browser can preform the 3D Transforms.

So starting with the Power Cube’s ring CSS, I stripped out all of the styles I wasn’t going to need and started creating the JavaScript function that would create the ring dynamically based on how many elements were in the list:

Main CSS:

#container {
	-webkit-perspective: 1000;
	-webkit-perspective-origin: 50% 40%;
	-webkit-transform-style: preserve-3d;
}
#stage {
	width: 100%;
	margin-top: 200px;
	padding-bottom: 100px;
	-webkit-transform-style: preserve-3d;
}
#shape {
	margin: 0 auto;
	height: 240px;
	width: 240px;
	list-style: none;
	-webkit-transform-style: preserve-3d;
	-webkit-transition-duration: 1s;
}

Transform JavaScript:

transform: function () {
    var length = children.length,
        count = length;
    while (count) {
        var child = $(children[length - count]),
			ratio = count / length,
			width = child.width() / 6.3,
			rotate = (length - count) * 360 / length,
			translate = length * width;
        if (rotate > 180) {
			rotate -= 360;
        }
        child
		.css(transformClass + 'transform', 'rotateY( ' + rotate + 'deg ) translateZ(' + translate + 'px)')
		.attr('data-angle', rotate)
		.attr('data-original-angle', rotate)
		.attr('data-translate', translate)
		.addClass(addClass);
        count--;
    }
}

I calculated the angle at which each side should be rotated between -180 and 180 degrees, so that the ring will rotate towards the shortest path on the initial load. Next I estimated the radius of the circle by multiplying the length of the side by the number of sides and divide by approximately 2pi ~= 6.3. I set the angle and translation in the z direction (radius) so that the ring will be created given sides of equal widths. I also saved the original angle, current relative angle, and radius as data attributes for later use.

Now I have the ring and it is looking pretty good, but I want to be able to select pictures and have the ring spin to the selected picture. Instead of changing the angles of the actual sides and rotating that way, it is actually easier to just rotate the outer unordered list element to make it appear that the ring is rotating. Using a jQuery click event handler, I am going to rotate the outer element to the opposite angle of the side I want to see facing forward.

setElementTransform: function (elem, clicked) {
	var	deg = -1 * setup.getElementTransform(clicked),
		value = 'RotateY(' + deg + 'deg)';
	$(elem).css(transformClass + 'transform', value);
}

Next, I needed to recalculate the relative angles for each side so that the ring will always take the shortest path when spinning to the next selected side. So I adjusted the relative angle by +360 or -360 if the difference between the current angle and the slide angle is more than 180 degrees.

setAngles: function (current) {
	var length = children.length,
		currentAngle = parseInt(current.attr('data-angle'), 10),
		count = length;
	while (count) {
		var child = $(children[length - count]),
			rotate = parseInt(child.attr('data-angle'), 10),
			origRotate = parseInt(child.attr('data-original-angle'), 10),
			translate = parseInt(child.attr('data-translate'), 10);
		if (rotate - currentAngle > 180) {
			rotate -= 360;
		} else if (rotate - currentAngle < -180) {
			rotate += 360;
		}
		child
			.css(transformClass + 'transform', 'rotateY( ' + origRotate + 'deg ) translateZ(' + translate + 'px)')
			.attr('data-angle', rotate);
		count--;
	}
}

Last, I changed the translation in the z direction on the selected slide to be 1.6 times normal so that it comes closer to screen and stands out from the rest of the sides. That seems to be about as big as you can make it, without having the picture go past the screen.

I am using the flickr api to get the pictures used for the slideshow. This *should* work for any number of slides (besides 1 I guess), however I think it looks best with less than 50. My browser really starts choking after around 400 slides, but that seems to be pretty impractical anyway.

Again, at this time, 3D transforms only work in webkit browsers, although hopefully they will become more standard soon. In Chrome you may need to update and/or enable ‘GPU Accelerated Compositing’ on the about:flags page for this to work. If it is still not working check the about:gpu for more information. I am currently on Google Chrome 11.0.696.34 beta on Windows 7 and I think it looks pretty awesome, so hopefully it does for you too. My girlfriend was not as thrilled about it as I was until I changed the search term to be puppies; now she loves it.

Take a look at the finished demo or get the source on github

Continued in Part 2

Anything you can do, Google does better

Marc — Thu, 13 Jan 2011 00:11:05 +0000

Google does a lot of things better than I can. It always knows the best way between two points while I often get lost, it can do unit conversions on the fly while I’m stuck looking for the other side of the measuring cup, and of course it can find pretty much anything as long as you know how to ask. One of the last things I have going for me is the ability to see and process information on the fly. Luckily I can sit on the bus, look at a newspaper and figure out this Sudoku puzzle using just my eyes, my brain and a pen. Oh, Google does that better than I do now too? Great.

Google Goggles is an image recognition app that is used to search based on a picture taken on your phone. It is used to take a picture of a landmark, barcode, print ads or many other things and figure out what exactly it is that is staring you in the face. Ideally it would be useful if you have no idea what something is, but more often we have some idea and just type it in and search the old fashion way.

So what does that have to do with my Sudoku puzzle? Well, now with Google Goggles you can just take a picture of a Sudoku board with your phone and Google will actually solve it for you. Fast.

Since Sudoku is essentially a logic puzzle with very strict parameters, they are well suited to be solved by computer algorithms. Programs to solve Sudoku’s have been around for a while, so actually solving the puzzle isn’t terribly impressive; it’s the way that it happens that is fantastic. It takes about 30 seconds total for Goggles to take the picture, scan it in, figure out it is a Sudoku puzzle and solve for the answer. There is something just mind blowing about holding your phone’s camera up to a puzzle, taking a picture and watching the answer appear. And it is not just the easy newspaper Sudoku that it can handle; I decided to give it a real test with a couple of theHardest Sudoku’s according to Wikipedia, and Goggles sliced through them like butter.

I guess the next time I hit a snag in my morning Sudoku I can get help from Google in 30 seconds instead of glaring jealously at the guy across from me who just finished it. At least I still have crossword puzzles to do while Skynet puts the finishing touches on becoming self-aware…

A Free and Open Internet?

Marc — Wed, 22 Dec 2010 04:46:18 +0000

This is the moment we have all been waiting for —well, those of us who like the internet the way it is— Congress is going to enact Net Neutrality regulations.

Wait, what is Net Neutrality again?

Net Neutrality has two main principles:

1) Web information is distributed in an unbiased manner

2) Accordingly, it is accessible to everyone (with a computer) in a designated area for the same cost

Sounds like a great American idea, right? The new regulations would not allow Internet Service Providers to block Websites that offer competing services to their own nor speed up or slow down certain types or sources of Internet traffic. That’s great, because I’m way behind on my Hulu queue and if Comcast started slowing down their traffic I would never get through it.

However, according to the New York Times, these rules are more lenient for wireless carriers. Even though they cannot block Websites, they can still block or limit access to applications or services that directly compete with their own. Wait, what? That isn’t really Net Neutrality…

“Maybe you like Google Maps. Well, tough,” said Senator Al Franken on Saturday “If the F.C.C. passes this weak rule, Verizon will be able to cut off access to the Google Maps app on your phone and force you to use their own mapping program, Verizon Navigator, even if it is not as good. And even if they charge money, when Google Maps is free. If corporations are allowed to prioritize content on the Internet, or they are allowed to block applications you access on your iPhone, there is nothing to prevent those same corporations from censoring political speech.”

This is a nightmarish scenario for those of you who have ever used and paid for VZ Navigator; you can imagine how (literally) lost this world could be without our free Google Maps. But most importantly, telecom companies don’t have the people’s interest at heart; it’s their wallets they are thinking about. I think Steve Wozniak said it best: “Every time and in every way that the telecommunications careers have had power or control, we the people wind up getting screwed.”

More Capstone News

Marc — Tue, 04 May 2010 18:49:00 +0000

Yesterday’s Globe had a nice write up about Numbers Empower: http://www.boston.com/business/technology/articles/2010/05/03/app_can_help_fine_tune_your_electricity_usage/

Site redesign and some more posts coming soon.