Hard News by Russell Brown

There's a very gradual process of opening up one particular type of digital content: geographic data. In that case, there's generally not even any digitisation work required, since it's already in digital form, but the government gave the rights to SOEs and commercial companies so of course they want to wring some income from it.

A couple of good, but only partially useful, examples are:

- You can get get the bulk of LINZ's data (including contours, roads, and even property boundaries), and then pass it on whoever you like as long as you keep a copyright notice on it. The bad news is that it still costs over $200 for the disc, and it's in their proprietary format that you'd have to spend a couple of weeks hacking to get into a useable form.

- The meshblock data (finest level of census data, about the size of a city block) will be available for free when it's released next month. However, that's just the numerical data, and to get the geographic boundaries of those blocks so that you can map them, you still have to pay $3000 or so.

All of this is so frustrating because geodata is among the most mash-up-able. The amount of productivity and creativity that could be unleashed if it were freely available is astonishing (mashing it up with ZoomIn or Google Maps to
create your own demographic or social analyses, for example), and when you consider that the US has made their equivalent data freely available for years, it's a huge shame.

What other publicly-funded data would you like to see made freely available?

Are there any other datasets that are being freed up?

Are there any local mashups (geographic or otherwise) that you want to pimp?

All of this is so frustrating because geodata is among the most mash-up-able. The amount of productivity and creativity that could be unleashed if it were freely available is astonishing (mashing it up with ZoomIn or Google Maps to create your own demographic or social analyses, for example), and when you consider that the US has made their equivalent data freely available for years, it's a huge shame.

I knew a little about this. That's fascinating. It also clarifies my mind about things that might be worth funding. How hard would is be to write a tool that converted LINZ data into a usable format?

I'm wary about new content creation being publicly funded (what do you choose?), but content tools are a different matter, and they're explicitly (if vaguely) referred to in the discussion document. We discussed the content tools idea in the group I was on, but I don't really know enough to know what's useful.

Anyway, I figured that I'd send this thread as a response to the discussion document (with all due credits, of course), so keep it coming.

Have a look at North Shore City's Geographical Information System. I don't know if it's still leading-edge, but it was when they launched it.

Perhaps they might be persuaded to permit some of the liberation Tom has in mind?

Hey all,

I am currently in the last few weeks of my Masters thesis in Design looking at weather representation and information design.

One large component of the thesis has been user driven content and in particular folksonomie. I find this stuff fascinating, it would be a goal to develop a truely ubiquitious weather service - but that's another story.

I found the bit in this report about folksonomies and central control interesting. What I found most interesting is how dismissive of this they are. I have no doubt (despite the merits of validity of data) that ontologies and folksonomies, as they are emerging now in the blogosphere etc are the voice of tomorrow.

Just as with the weather, the metservice will continue to provide the official word, but if you are like me, I have dozens of weather cams bookmarked that actually make my NZ weater picture, not the time delayed official picture.

It was like some of the large news stories of the last 2-3 years, officials have been watching TV to get the latest (although not offical or verifiable) information.

The immediacy of the medium (internet, wap etc) allows the official to be blurred, what is official information and is the official necessarily better than informal information.

Take this waterfront debacle, I say it lightly because as far as I see it, the media has done one hell of a job in disinformation (planting stadium Allliance from Germany on the waterfont for a representation), then all morning the talkback is filled with concerned of Riccarton who wonders why they are going to build that ugly thing on the waterfront. The official reaction has lagged behind the populous, I filled out the Auckland City official survey yesterday.

By the very nature of being official, information will mean inherent time lags, and that is fine. But to play such disregard for folksonomies (as disparaging term in itself) is too arrogant of such a report.

Also. of course there needs to be a sense of urgency in getting public content online, and there are some great examples of that out there. But again, they seem to be dismissive of what is actually already available online as in anyway actual real content. Just because it doesn't come from a ministry doesn't make it any less valid.

Public Address & all the other blogs, web cams, amateur weather stations, community sites etc etc etc, are all part of the online landscape that is NZ, and they provide so much for so many that the official will never be a part of.

All very well having the full historical official account of NZ broadcasting history online for example, interesting nice fluffy stuff, but it is the dynamic raw stuff that is being generated everyday that is really interesting. the good old high art & gatekeeping debate.

Nice to see this report, shame it's so dismissive of folksonomies, shame NZ braodband is so expensive and so slow, and shame that govt still sees itself as the director of this, shame it's 5 years late...

Still an interesting read, and something I will be commenting on, cheers.

ZUNE,

I bought a shuffle last year. I opened the box, plugged it in, added music, let it charge and that was it.
I got a great kick out of the disaster they had at endgadget.

itunes has never crashed my computer (pretty spectacular if it could under OSX), I never had to give my phone number to anyone... what a joke.

I agree the brown is funky, but the concept and software are tan at best, more a smokers stained off white. But this is why I am a Mac guy and the majority
of concened of Riccarton put up with crap. Walk into Harvey Norman and the Orwellian masses are driven past the macs like a conveyor belt to the PC
section, where mediocrity and frustration lives, why? God only knows?

Yikes, bit of a rant

cheers

Just as with the weather, the metservice will continue to provide the official word, but if you are like me, I have dozens of weather cams bookmarked that actually make my NZ weater picture, not the time delayed official picture.

The weather one's a really interesting example. Rick Huntington had a lot of visits to http://greylynnweather.net when the big winds hit Auckland last week - ironically, as he pointed out, he was sheltered from the weather, so there was nothing exciting going on.

There's a guide to NZ weather stations here

I've used the Wellington webcams quite a few times, to look anxiously at whether the fog has cleared by the airport, or to just have a giggle about the weather in Wellington ...

And I remember when Dave Winer turned Scripting News over to the unfolding events of the morning of 9/11, and readers started posted in NYC webcam links. I think it filled a need. (The New York Times, which performed poorly on the internet that day, the following year acknowledge Winer by become the first major newspaper website to publish RSS feeds.)

Google Earth is an interesting case too. They rely a lot on their user community for content. The day I visited this year, the guy at Google had just noticed in one of the user forums that someone had done a 3D overlay showing all the lighthouses in New Zealand. I imagine it's bloody excellent if you're interested in lighthouses.

Your National Library action point sums up the big problem libraries have in taking this kind of thing on: we're usually official and public representatives of large organisations (city councils, universities, central govt in NLNZ's case), but constrained from making any decisions on that organisation's behalf. The bigger the organisation represented, the more any action points like yours have to be driven through endless policy and strategy committees, with appropriate documentation etc. Which is why individuals and private companies are making all the running on this, and why librarians are generally left to bemoaning the fact that it ought to be us.

The universities are moving towards the principle of making publicly-funded research publicly accessible, and I'm involved in implementing one "institutional repository" for making that research avaiable. But as at the National Library, it's like wading through treacle - on the one hand, the thousands of potential authors who might contribute have to be individually satisfied that their intellectual property will be adequately safeguarded, and on the other university administrations have an interest in the legitimacy of what gets posted as official research output of the university, so ensuring the validity of what gets posted is an issue. To cap it off, academic publishers don't like having their own copyright breached, and can be expected to be grumpy about any examples they find. The owners of a private venture like YouTube can shrug their shoulders and just go for it and see what happens, but employees who do that kind of thing at a big organisation don't last long.

Hmm - on reading back through, it looks like yet another case of librarians whinging about how unfair life is. Wasn't intended as such - just wanted to point out the constraints we operate under are always going to leave us way behind outfits like Google or YouTube.

I think the government should be making as much information as possible accessible. The argument *for* charging for public sector data, of course, is that it costs money to collect/create, a minority will use it, so why should the general public purse subsidise that minority with free information (where have I heard that before?)

The US doesn't do this though - unclassified data has to be available on a copyright-free basis by law. So anyone can print their own marine charts using NOAA data, for instance - which makes US charts much cheaper than others.

What the government shouldn't be doing (as it's crap at it - see www.legislation.govt.nz) is actually hosting the content. Instead, they should leave it to other profit- and non-profit bodies to deal with that side. The approach with cadastral data is good (and I don't think $200 is unreasonable to produce and distribute media to a small group of people). Generally, if a format is documented and consistent, someone will create converters/importers pretty quickly.

First a disclaimer - as far as possible.

I work for the New Zealand Film Archive, a government-sounding, but actually independent, not-for-profit organisation which preserves all forms of New Zealand moving image material. Anything below which looks like, but is not explicitly stated as Film Archive policy is probably just me mouthing off and should not be quoted back as the offical word.

In my view, the document released yesterday continues the major problems with the Digital Strategy process to date. It does not really deal with the implications of its acknowledgement that many of the things that people want to access digitally are currently not digital. Moreover it reflects the fallacy of conflating "digital" with "on-line" or "web-based". It does not clearly commit to a role for the Government.

For preservation reasons, and for a wide variety of accessibility reasons, we would all be better off if NZ's full range of scientific, cultural, political and historical material was in digital form. The real challenge is how to achieve that and how to ensure that the resulting digital collections are as accessible as possible. The complexities of guardianship and ownership of this material go far beyond the scope of the response to date - funding the National Library and a number of other Government agencies to create or organise digital content.

We ought to be able to read the Government's digital content strategy and understand what it thinks its role is going to be - regulatory referee? funder? provider? cheerleader? territorial protector? If it feels an obligation to digitise elements of its own collections, does it feel a comparable obligation towards the other national collections - Hocken Library, Auckland Museum and so on? What does it think about copyright? What is the Government's interest in non-web applications of digital technology like large scale storage or preservation of cultural property?

I still don't know.

Frank

Good point about the digitalisation of cultural heritage. There is a lot of fluf in this document, but it is at least an acknowledgement of the difficult decisions and planning that is needed.

Russell,

interesting where one get's their collective knowledge these days. My weather sources are all over the place.
Since I'm on campus, my new Irish weather stone has become
http://www.physics.otago.ac.nz/eman/weather_station/current.php
I guess I could put my head out the window...

The NZ weather forum
http://www.weatherforum.org.nz/forum/viewforum.php?f=2
is a fine source of technical informal, observational and sometimes downright funny "pissing down here"

The weather forum was fascinating durning the big weather events we've had here this year, some people with some cool toys were giving us the latest readings and predictions, while the metservice was chugging along.

My news comes from many sources including these repositories of world newspaper front pages.

http://www.newspaperindex.com/
http://www.newseum.org/todaysfrontpages/

Google Earth is an interesting one. Turn it on to get some pretty fascinating imagery, and of course context to where things are happening, just don't rely on the community to get accuracy in their posts. I have seen labels for stadium and airports etc placed many many miles away in the sea, up buildings etc. Accuracy isn't really needed when you are hovering over San Francisco at 40km, but when you are apparently only hundreds of meters up and you have 6 different locations for Victoria's Secret (not that I really needed to know that).

As for the digital plan, kind of related. Did I really hear a spokesperson for TVNZ talking the other day about their upcoming digital channels claim that TVNZ is the world leader in independent jounralism and news innovation? has this person spent all of his/her life on South Georgia Island?

Russell, Rich:

<geekspeak>The LINZ data is in a form known as the "Bulk Data Extract": basically, it's just a dump from their relational database, rather in a GIS-ready form. That makes a lot of sense, in a way, since they can make it available relatively cheaply (yes, $270 isn't too bad) and it's the most general form. Some of the tables could be quite easily turned into something useable, but most layers would require a whole bunch of joins followed by some data-munging to create something viewable. See LINZ's BDE page for more details.</geekspeak>

The good news is that after you do that, you can pass it on as much as you like as long as you keep the copyright notice. You can even sell it for a profit, which is exactly what Eagle, Terralink etc do, and given the work required to do something useful with it, most users are happy to pay. To make it truly useful as a public resource, someone (a university? a bunch of public-spirited geo-geeks?) would have to buy it, do the conversions and host it. At 30GB it's too much to just upload as one big zip file, but with open-source tools like MapServer it should be possible to set it up as a web feature service. In other words, with the right XML query (defining the bounding box and layer that you're interested in) it would serve up the data you want in useable vector format (not just as a GIF).

Some people have already done some of this stuff. Ollivier & co hosts a bunch of free data together with some wry commentary on the availability thereof.

Since the data contains street addresses, this raises the possibility of rolling your own geocoder (i.e. converting 123 Thinggy street into a latitude & longitude). The lack of this and of fine-level postcodes has made it hard to do the sort of mashups you see in the US (such as mapping flats for rent by mashing up Google Maps and Craigslist). ZoomIn's API provides geocoding, but you can't use it for free very often. Google Maps has finally got around to having address searches for NZ, but I'm not sure how accurate they are.

The other limitation on mashups is the lack of local data in RSS format, or anything that doesn't require painful screen-scraping. For instance, talking of the weather (as someone was), the Greater Wellington council provides all sorts of near-real-time rainfall and other data. Unfortunately, it's all in pre-cooked graphs, so it's not (easily) machine-readable. On the other hand, GeoNet provides a very useful page that Matthew Walker was able to convert to an RSS feed of recent earthquakes. On the other hand, they do such a good job of mapping them themselves, it's hardly worth mashing.

What the government shouldn't be doing (as it's crap at it - see www.legislation.govt.nz) is actually hosting the content. Instead, they should leave it to other profit- and non-profit bodies to deal with that side.

I don't entirely agree. Perhaps they shouldn't be in the business of presenting it, but they should be able to host it in a raw, documented, machine readable form, and leave it up to the creativity of others to present it in exciting and informative ways.

I don't entirely agree. Perhaps they shouldn't be in the business of presenting it, but they should be able to host it in a raw, documented, machine readable form, and leave it up to the creativity of others to present it in exciting and informative ways.

I actually think they should do some presentation too, but allow for other uses.

When I was in this this year, I saw a quick demo of Many Eyes, IBM's new (still forthcoming?) public data visualisation service - as in free for public use. It looked like a bitchin' presentation layer: all the custom graphs and comparisons you could want.

The woman who presented it said they'd already had public-sector interest from Australia. Perhaps it's worth a look in New Zealand too.

Tom,

my fascination with data official and unofficial is in how can we make this data available to all, platform independent and context aware. Diving into the realm of Ontologies and how do data sets talk to each other is fascinating stuff being done by some very clever computer science geeks, providing heaps of opportunities of us otherwise enclined to play with.

I'm a designer with a fascination of the wether, and can harldy wait to play with the toys of the future (data sets, ontologies and their interfaces). I could suggest that some of the more classic mash-ups we are seeing today are on the whole just the very starting point of a process in which the ends are still pretty open for defintion. For the computing world (using industrial analogies) we are just starting to come through the steam engine revolution and hitting the internal combustion engine, very early on indeed.

I'm not to sure that those producing the data are in any way better off at presenting it. Some of the more elegant examples of the simpliest mash-ups with google mapping etc have been by people only interested in the interface or results, rather than the data itself. that does not mean there won't be failures, but in getting to where we may be heading to with ontologies and interrelated data sets is going to be fun.

the problem is how to get the seemingly unrelated data, in many different formats (as this should be the challenge instead of looking for conformity of data) talking to each other so that meaning is derived. Stuff around the edge such as privacy, pricing, copyright etc can be worked into this.

As an example, I would be really intersted in an interconnected world of ontologies that enabled a ubiquitious information environment. I'd love my cell phone to anticipate me driving somewhere and getting the latest forecasts to play on the car stereo, to talk to the holiday home to turn the hot water on, or to suggest a number of hotels with the cheapest rooms and which one has views and which one has a touring german ohm pa pa band staying, or to prebook the warrent at the local garage because I have neglected to do so before I left town, compare prices and work out which is the best mechanics not on holiday that weekend. These should not be things that make us lazy, but enable us to do the things we are doing today with better choices. One only needs to remember the fantastic claims of the Mighty Micro to realise that we don't work 3 day weeks and all work from home.

Whops off point a little. Back to data and NZ, why should we rely on the official? Using the weather again, our Dunners weather is out on the Taieri, bloody miles away and about as relavent some times as looking at Auckland forecasts for me living on the Otago Peninsula. So I look to the local weather web cams and amateur weather stations. They are on the whole accurate, and unless I have a lot economically riding on them, they provide very good context for my needs. Sure google maps is still lagging behind in what is available in NZ, but what is to stop us accessing other information or creating it ourselves, instead of waiting for google to catch up with this little blip on the Globe. As I have mentioned with the right systems and ontologial data sets communicating, context and knoweldge can be obtained from seemingly uncompatible sources. Who's to say that someones car taking a temperature every 5kms on the raod between Lumsden and Mossburn relaying that information via a cell phone to a database that can then be accessed by bus operators, with some form or reciprocial renumination taking place (what ever that may be). IS the data any less valid, and if it is we can work in redundancies and error factors to the equation.

Nice thread guys, even if I do go off on a rant. Love the earthquake map, for a boy from Hokitika living in Dunners I do miss earthquakes.

As an example, I would be really interested in an interconnected world of ontologies that enabled a ubiquitous information environment.

Heh. You might have trouble getting that past Cabinet. Most of them would probably think you were taking the piss ...

exactly what would be left for the Ministry of Information.

I think it's worth noting for the record that New Zealand is miles ahead of the game in this area. Even if it's just in the territory of draft discussion documents, rather than actual policy, it's certainly heartening.

Despite what the BBC might have you believe, Creative Commons is far from mainstream in the UK. I find myself having to explain it in the most astonishing of situations, and the copyright lobbyists have almost entirely successfully managed to equate it with 'anarchy' (if not 'terrorism') in the minds of the decision makers.

It is generally considered to be a synonym for public domain, only without the fluffy 'citizenship' connotations. In many circles, it's openly regarded as a threat. Which is, of course, nonsense.

One of the biggest arguments against Creative Commons put here --- particularly, but not exclusively by the record business -- is partly in terms of its failure to address monetisation (which is a little like accusing a parachute of failing to address engine reliability), but mostly in terms of its apparent irreversibility.

If you assign a Creative Commons licence to your works (as I have made a point of doing where possible), it seems at first glance that you may not later be able to change your mind. In other words, reclaiming control over permissions that you had previously granted in a blanket fashion might prove impossible.

Personally, I don't think it's a significant problem. It is actually doable, it's just not terribly enforceable, but I can't think of a single situation in which that would be a disaster -- or even much of an inconvenience.

These sorts of issues cloud the usefulness of Creative Commons licences for digital works, and I think it's commendable that the intention, the spirit and the phrase has made it into a policy document. Fingers crossed it stays there.

Copyright reform's up there with the nuclear free thing, in my book.

Andrew: Creative Commons licenses (like most open source licenses) can't be effectively revoked (see [http://blog.tph-lex.com/archives/weekly/week_2003_02_23.html#removing])

Because you've granted a perpetual license to whoever downloaded the work to copy *and redistribute* it, they're entitled to keep on redistributing your work - so any attempt to yank it back is futile.

The close off date for responses to the discussion paper is - 5pm, Wednesday 20 December 2006.

Is it also worth noting the feedback prefences or is this too prescriptive? (35 & 36) may help to provide some shape for the policy group who will be assessing various contributions.

Note this form is a guide to the format of your response. you are welcome to use additional pages.

1. Do you agree with the vision, purpose, outcomes and goals? Yes/No? Why/Why not?

2. A re the challenges and objectives the right ones? If so, which are the most important and why? If not, are there alternatives? Please describe.

3. A re the proposed actions the right ones? If so, which are the most important and why?
If not, are there alternatives? Please describe.

Input to the Next Stage
The strategy identifies three main elements to the strategy – building the digital foundations, unlocking
content, and leveraging opportunities. What things are important to you and your community or area
interest under each of these elements? Are there unique issues or opportunities that affect you or the sector
you work in? Are there ideas or information you need assistance with or want to share with others?

4. Building the Digital Foundations:
5. Unlocking Content:
6. Leveraging Opportunities:

Interesting.. As a musician (blatant plug: http://www.frey.co.nz) I'm more interested in the ideas surrounding Goal 2 (about New Zealanders being "at the forefront of creating and sharing digital content"). And this all seems in pretty good order.

I like the emphasis on the ideas of sharing, in particular the second Proposed Action, to "provide support and advice to communities on the standards and tools that enable creation and sharing of content".. I work for a company called Lumen Digital, we do (among other things) localised digital storytelling but with the aim being to get at it from both sides, as in both the retelling of existing stories and the creation and recording of new ones. Anyway it's the sharing part that has emerged as most interesting about digital tools (given as how digital makes duplication an effortless task) and it's good to see that reflected.

As far as Creative Commons goes.. As a musician, I see copyright as getting in the way of audiences finding out about me. If people like my music so much that they are willing to go the trouble of sharing it with their friends, it doesn't make sense for me to restrict them from doing that. Moreover, it seems that filesharing makes the idea of paying money for recorded music go back out the window from whence it flew in not one hundred years ago. If I'm to make money out of what I do in a way that makes sense I have to perform, just like every generation of musicians before this abberrant blip of the 20th century, since for an audience member being there is something that it is still impossible to digitise. In this context Creative Commons seems to me to be the only sensible way to offer recorded music.