Strawp.net

I’ve been using the original Facebook to Google Calendar script since it was first launched but for whatever reason the redesigns of Facebook seemed to run away from the original author and it stopped working properly so I decided to branch my own version.

v1.0.0 is largely untested and in “works for me” stage so try it out and give feedback!

Note that you may need to refresh the event page in order for the script to kick in because of how Facebook now loads pages in using AJAX.

As I’m sure you all noticed last week, the latest fun leaked data from a polical party came from the BNP who somehow had their entire membership list leaked onto the internet. For people like myself this presented two fun opportunities:

1. To do a little bit of geographical and statistical analysis on some odd data
2. To laugh at a bunch of hate-filled racists

There were quite a few nuggets of statistical analysis in the first couple of days: A proximity checker to see which of your neighbours members, the obligatory google maps mashups (since, sensibly taken down), a sort of heat map and the Grauniad did an excellent map broken into electoral wards. They were all pretty good, but they still rather suffered from the problem that you see “hot spots” in areas which are naturally population nodes. There was no accounting for population density.

Anyway, in a spare moment I took a copy of the database, cleaned up the postcode information a bit, ran it through a geocoder to get lat and long data and then ran that through a lookup for population density and then grouped the data by postcode area. What I now have is a count of each person in a postcode area, divided by the population density - this should then give a population normalised rank of how hate-filled post code areas are. Anyway, here’s the Top 40, Top Of The Pops style:

Creating pipes is a bit of a mental leap from traditional programming.

I noticed Yahoo Pipes pretty soon after it launched nearly two years ago but other than aggregating RSS feeds I couldn’t really get into it. The visual psuedo-flow-diagram programming didn’t gel with me at all and I just found myself thinking I could have more easily achieved the same results in less time in PHP.

Well, after chatting with the Yahoo Developers booth at FOWA the other week and this week failing to find a decent UK Xbox 360 release calendar in iCal format I decided to have another crack using the messy data on Gamestation’s website as a source. The result: Gamestation XBox 360 releases. I’ve also created a conversion script in PHP to make that data iCal compatible and therefore importable into pretty much any calendaring system you’d care to mention.

If you’re used to writing screen scraping software in a real programming language there are a few mental barriers you have to clear in order to get anywhere:

1. There is no regexp “match” function. You have to use substitute if you want to extract a term and make sure you “.*” at either end of it to remove unwanted text.
2. You have to program loops one after the other instead of nesting logic all inside one big loop. Makes it a very odd thing to read.
3. Multiline regexps are a headache to write because you have a single-line regexp input. I got around this by making a multi-line string single line with “s/[\r\n]/” before doing proper regexp operations
4. It’s really quite buggy. Under Firefox in windows drag and drop operations would stop working after a while and under Firefox in Ubuntu after a while the whole edit area would turn into a bit of a mess. Save frequently.

Anyway, it’s pretty good once you get your head around it and it outsources some of the heavy lifting involved in screen-scraping to Yahoo instead of your own site. I’d like to see the ability to export in iCal format as standard in future updates.

A few notes on my first Future of Web Apps (FOWA) conference in the ExCeL centre in London.

People

Everyone I met was very cool and friendly, from small startups just trying to get noticed to relative tech celebraties and the larger stands like the Microsoft one - everyone came across as genuinely interested in the kind of conversations that make most of my friends eyes glaze over. I was even happily talking about XBox Media Center versus Windows Media Center and why the Silverlight IDE sucks (but the runtime is very good) with the MS guys.

My God do we all look the same though - white middle class guys with short gelled hair wearing jeans and ironic T-shirts. I was finding at times I honestly couldn’t remember whether I’d spoken to someone or not.

Out of the list of speakers, I only really spoke to Blaine Cook (ex lead architect for Twitter, current architect for Fire Eagle) who I found myself sat next to for Diggnation Live. He was a really interesting and friendly guy.

Rumours/Tech

• Reading between the lines, the reason twitter IM isn’t coming back is because it’s Blaine’s area (he did a talk on Jabber) and not really a priority for the other members of the Twitter team. In terms of bandwidth and resource use Jabber should actually provide a lower overhead than traditional polling over HTTP of the Twitter service and could even allow much of the twitter service to be decentralised to other companies - e.g. to provide SMS gateway functionality again.
• In about 3 years Facebook should be very open. APIs for accessing/exporting your own friends to other services are on the horizon but not currently a priority.
• Mark Zuckerberg still delves into the Facebook source code to do bug fixes, despite being head of a staff of 700 employees.
• Despite being $1.5bn, Mark Zuckerberg still wears North Face fleeces. Who knows why.

Sites of note and people I met

• Josh from iPlatform - a technology for embedding existing apps and communities inside Facebook as native apps (pitched his business to Jason Calacanis and about 1500 people during the Dragon’s Den section - clearly has sizeable gonads)
• Zuz from Huddle - an online collaboration tool for businesses
• The guys from Soocial (pronounced “Social” - they’re Dutch). Soocial is a cloud computing app for your contacts. It takes your contacts from your phone, Gmail, Outlook or your Mac address book and sticks them all in one syncable and backed up place. I’ve signed up to this and it needs work but it has a lot of promise.
• The guys from Everycity (managed hosting) who I drunkenly found the last tube back with on Thursday.
• Some guy from GUIMagnets (whose business card doesn’t have a name on it) which is a selection of magnets that look like GUI elements for prototyping apps.
• Snaffleup, which is like Freecycle but more organised
• Mapstraction - an abstraction layer for creating embedded map applications with any of the popular mapping services
• GeoCommons - a resource and tool for finding location specific data and creating interactive maps with it
• Rummble - a mobile discovery service. If this takes off over the next few years with the uptake of phones like the iPhone and Android based handsets it will be very cool.
• Last but not least, TactileCRM which is an excellent, uncomplicated CRM system for businesses

There were countless other people I talked to as well who I didn’t get a business card or promotional item off, so I can’t remember their vowel-free unpronouncable domain names, sadly. It was really interesting just seeing what people are building.

The talks are being uploaded to the Carsonified events site and the hilarious episode of Diggnation Live will be up this Friday (17th).

They’ve clearly gotten a new marketing person over at Berocca in the past year. Having not really touched TV ads until now, they launched a campaign which is clearly targetted at the blogosphere which featured a slighty embarrassing rip-off of OK Go’s “Here It Goes Again” and now they’re buying blog posts by launching their “Blogger Relief” campaign. You can register your blog and if they like it they’ll send you a box of free stress relieving gizmos.

I don’t need to be paid off to thoroughly recommend Berocca - I’ve been addicted to the stuff for years and it’s saved my life countless times, however the odd cheap bribe never hurt anyone.

If that still doesn’t convince you, the prospect of luminescent orange pee after a glass always brightens up a dull day (and freaks out anyone else in the public toilet).

Just added a new repository to the site: PHP Libraries. Currently in there are working classes or functions for:

• Firefly Media Server - a class for direct access to the songs database
• Roku Client - a client for Roku music players like the Soundbridge. Allows remote scripting etc
• twitter - a bunch of functions for posting to, or getting info from a twitter account
• Delicious - a class for (currently only) getting delicious bookmarks. Will add other methods if/when they’re needed for other scripts.

And there’s a ping.fm class in there that I might work on if I can be bothered to get an App key for it.

Head to svn.strawp.net/lib to browse them, svn co http://svn.strawp.net/lib to check the lot out.

Update: Just added svn.strawp.net/scripts/ which currently just has my twitter command line client in it.

Well spotted by Phil who’s discovered that the new N95/N96 stream for the BBC iPlayer is already available and can be viewed or even saved to file just by pointing an RTSP compatible media player at a valid URL - just like when the iPhone H264 stream came out.

More details are on our lovely Beebhack wiki

I’ve just moved my site from the (once good value, now rubbish) ODSOL to Dreamhost, which is a blinding improvement.

I now have (about 1000 times) more web space and I’ll also be able to do nice things like host my own SVN projects. Yay!

Some of the links to other non-blog parts of the site might not work for a bit…

Somebody out there wants it, so here it is

Update: I’ve moved this to the SVN server. To get them:

1. svn co http://svn.strawp.net/scripts/tw
2. svn co http://svn.strawp.net/lib/twitter.php
3. chmod u+wrx tw
4. make sure TWITTER_EMAIL and TWITTER_PASSWORD are defined

“tw h” for help text.

I’ve been using my own PHP-based CLI Twitter client pretty much since signing up (the Twitter API is so simple). You can view your timeline, tweet, view replies and view and send direct messages all via the command line, and I’ve just added the now-famous Fail Whale image to the error output, see this screenshot for an example.

The source isn’t published at the moment, but if you want it I’ll clean it up (take out my hard-coded username and password) and upload it - just leave a comment.

The whale is based on this one which I turned into my own version reversed with added twitter birdies.