Skip to content

Releasing open data on data.govt.nz

Have you checked the data for any Personal Information?

Guidance

What is personal information and the Privacy Act?

Data can contain values that identify a specific individual. This is called 'personal information' in New Zealand but is sometimes referred to as Personal Identifiable Information (PII). The Privacy Act 1993 controls how 'agencies' collect, use, disclose, store and give access to personal information. The Privacy Act applies to almost every person, business or organisation in New Zealand.

Learning outcomes

  • Recognise personal information in your data.
  • Understand how the Privacy Act applies to personal information.

Personal information

The Office of the Privacy Commissioner develops and promotes a culture in which personal information is protected and respected. They define what is considered to be personal information, and provide guidance surrounding the Privacy Act, its principles and other privacy codes.

This includes data such as:

  • Names
  • Phone numbers
  • Email addresses
  • Other observations where an individual is identified.

Note: where the above data is already public knowledge then this is ok to include in an open dataset. For example, the dataset of Marriage Celebrants on data.govt.nz includes contact details for these people, however, this data is public knowledge and already released elsewhere publicly (reading over the Privacy Act Principles helps to clarify any exemptions such as this). 

Privacy Act

The Privacy Act consists of 12 Principles which apply to data in New Zealand:

  • Principle 1: Purpose of collection of personal information
  • Principle 2: Source of personal information
  • Principle 3: Collection of information from subject
  • Principle 4: Manner of collection of personal information
  • Principle 5: Storage and security of personal information
  • Principle 6: Access to personal information
  • Principle 7: Correction of personal information
  • Principle 8: Accuracy, etc., of personal information to be checked before use
  • Principle 9: Agency not to keep personal information for longer than necessary
  • Principle 10: Limits on use of personal information
  • Principle 11: Limits on disclosure of personal information
  • Principle 12: Unique identifiers.

Detailed guidance


Source: https://data.govt.nz/manage-data/privacy-and-security/what-is-personal-identifiable-information-and-the-privacy-act/
Do you know whether you should carry out a full Privacy Impact assessment or not before publicly releasing the data?

Guidance

How to carry out a privacy impact assessment on your dataset

A privacy impact assessment (PIA) is a tool used by agencies to help them identify and assess the privacy risks arising from their collection, use or handling of personal information. A PIA will also propose ways to mitigate or minimise these risks.

Learning outcomes

  • Know whether or not you need to carry out a Privacy Impact Assessment
  • Understand how to do a Privacy Impact Assessment

The Office of the Privacy Commissioner provides agencies with a Privacy Impact Assessment Toolkit.

There are two parts to the toolkit.

Part one

First, there is guidance on how to assess whether or not you need to do a PIA and, if you do, how in-depth the assessment may need to be.

If the assessment will turn out to be complex, you may want to think about getting help from an external privacy expert. If you might not need to do a full PIA, you can also do a brief privacy analysis. This will be a helpful record of your decision and a reference to the basic details of the data you have gathered and why.

Part two

There is then a step-by-step guide on how to successfully complete a PIA, including:

  1. Gather all the information you need (the personal information involved, and why it has been collected)
  2. Check against the privacy principals (see: What is Personal Identifiable Information and the Privacy Act)
  3. Identify any real privacy risks and how to mitigate them (this is where it is helpful to have someone familiar with privacy helping with your PIA. The Office of the Privacy Commissioner can always help with advice)
  4. Produce a PIA report
  5. Take action
  6. Review the PIA and use it as a checkpoint once things are in operation (are problems starting to emerge and further changes needed?)

Detailed Guidance


Source: https://data.govt.nz/manage-data/privacy-and-security/how-to-carry-out-a-privacy-impact-assessment-on-your-dataset/
Have you decided what you would like others to be able to do with your data and selected an open licence?

Guidance

NZGOAL (New Zealand Government Open Access and Licensing) framework

The New Zealand Government Open Access and Licensing framework (NZGOAL) is all-of-government guidance for agencies to follow when releasing copyright works and non-copyright/public domain material for reuse by others. Use this guidance when you intend to release open data.

Learning outcomes

  • Correctly determine which data can be safely released using an open licence.
  • Select the appropriate Creative Commons licence for an agency's data release.
  • Applying the licence at point of release for others to legally reuse the dataset.

Do you work for a government agency and want to enable appropriate re-use of your agency’s material by licensing its copyright works or releasing non-copyright material for re-use? Or are you a New Zealander who would like to know more about NZGOAL and how it all works?

Learn more about NZGOAL here.

 

What is NZGOAL?

NZGOAL is guidance for agencies to follow when releasing copyright works and non-copyright material for re-use by others.

It aims to standardise the licensing of government copyright works for re-use using Creative Commons licences and recommends statements for non-copyright material.

 

How can I access the NZGOAL framework?

NZGOAL framework, version 2, Dec 2014

NZGOAL software extension, version 1, July 2016 [PDF 671 KB]

 

Can I use NZGOAL for software?

NZGOAL has a separate Software Extension (NZGOAL-SE) for licensing and releasing copyright software works under free and open source software licences.

NZGOAL-SE makes use of the General Public Licence (GPL) and the Massachusetts Institute of Technology (MIT) licence, providing software specific guidance for releasing publicly funded software as open source.

NZGOAL-SE guidance

NZGOAL software extension, version 1, July 2016 [PDF 671 KB]

 

How do I learn more?

NZGOAL guide for users, July 2015

Check out our online training video series which explains and addresses the questions you may have about NZGOAL including the application of appropriate Creative Commons licenses:

NZGOAL online training videos

 

Help! I still need more information

The series of guidance notes will give you practical assistance to apply NZGOAL. The notes cover website copyright statements, file formats, procuring copyright works, and databases and datasets.

Guidance notes

 

Contact us

For more assistance, contact opendata@stats.govt.nz

 

Documents

CAB Min (10) 24/25A - New Zealand Government Open Access and Licensing Framework [PDF 414 MB]

Cabinet paper - New Zealand Government Open Access and Licensing Framework [PDF 2.6 MB]

NZGOAL framework, version 2, December 2014 [PDF 939 KB]

NZGOAL framework, version 2, December 2014 [DOC 1.1 MB]

Letter from the State Services Commissioner to all Public service CEs informing them of NZGOAL [PDF 455 KB]

 

Last updated 12/10/2016


Source: https://data.govt.nz/manage-data/policies/nzgoal/
Have you written a good descriptive Title and Description for your data?
Do you know when your data was created, last updated and how frequently it will be updated?

Guidance

What metadata should I include with my dataset?

Metadata describes your dataset to others in a standardised way. Having good quality metadata helps people discover and use your dataset. This guidance provides a description and examples of good practice metadata when releasing on data.govt.nz.

Learning outcomes

  • Understand what attributes of metadata to provide when releasing an open dataset.
  • How to describe a dataset so that others are able to reuse it more easily.

When preparing to release Open Data on data.govt.nz you should include the following metadata properties alongside your downloadable data files.

You will also need these same values when you add your dataset to data.govt.nz. 

There are a number of other technical metadata properties you can submit alongside your dataset on data.govt.nz these are covered in full detail in the data.govt.nz metadata schema.

Title 

The title should be descriptive enough that a member of the public can have a reasonable expectation of its subject. It should be long enough to be informative but short enough to be readily understood in isolation by someone outside your agency – for instance:

‘Production of veneer, plywood, laminated veneer, lumber, particleboard and fibreboard, 1951 to most recent’

Use the dataset title at source (eg on your agency website) for the download link, and as the Title metadata element on data.govt.nz. 

Description 

This should fully describe the contents, purpose, source and structure of the dataset, and (where necessary) include any comments about its quality and reliability. 

For datasets that have a spatial extent (relating to a geospatial or geographic area) or temporal extent (relating to a period of time), note this in the description. 

Include any notes or cautions that may help reusers assess the data’s suitability for their purposes. It may also be appropriate to note the version. 

Use the description at source as the baseline information about the dataset, and as the Description metadata element when listing on data.govt.nz. 

For a description example, see https://catalogue.data.govt.nz/dataset/beacon-special-purpose-general-points-hydro-1-4k-1-22k

Tags/keywords 

Related keywords help others find your dataset, you can list these in your metadata along side your dataset and provide when you list it on data.govt.nz.  

File formats and resources

If you have your dataset available in multiple formats (CSV, KML, XLS, PDF etc), or want to include accompanying documentation, data dictionaries etc alongside your dataset, you can provide these as separate resources in your dataset in the data portal.

Each file can include:

  • the location you have uploaded the file publicly
  • a title and optional description
  • the file format (data.govt.nz will try to automatically determine the file type if you leave this blank)

  

Contacts 

Provide a named contact point, telephone number or email address (ideally both) for queries about the dataset, both at source. 

Dates 

The dates of creation of the original dataset (where known) and its last update should be provided. If not known, the date of last update may be available in the metadata of the file containing the data. 

Frequency of update 

If the dataset is refreshed at regular intervals, indicate this in your metadata. If the Frequency of Update options on data.govt.nz don’t apply, provide this information in the Description.  

Licensing  

Provide the name and URL of the Creative Commons license you are using to make the data available for legal reuse by others. For example, "Creative Commons Attribution 4.0 International" would link to https://creativecommons.org/licenses/by/4.0/ .

If you are not using a Creative Commons licence or have your own terms of use for your dataset then provide this separately in a "Rights" property or in your metadata description and leave out the "License" property. 

Rights 

If you have a custom license, certain conditions or caveats for use of your dataset that aren’t in a standard open license then you can list these in a separate "Rights" property or include in your dataset description. This helps users understand that there are some aspects of the dataset to be aware of when being reused that are not covered by a standard open license. 


Source: https://data.govt.nz/manage-data/releasing-data-on-data-govt-nz/what-metadata-should-i-include-with-my-dataset/
Have you got a point of contact or data maintainer email address for your dataset?

Guidance

What is a data custodian and what do they do?

To make sure data and information is maintained, there will often be a data custodian responsible for updating and preserving the data. Under the New Zealand Data and Information Management Principles, agency data custodians should implement recommended practices to support well managed data.

Learning outcomes

  • Understand what a data custodian is.
  • Understand the responsibilities of the data custodian.

The New Zealand Data and Information Management Principles were developed to ensure high quality management of the information the government holds on behalf of the public.

Under the Well Managed principle, agencies, as the stewards of government-held data and information, must provide and require good practices which manage the data and information over their life-cycle, including catering for technological obsolescence and long-term preservation and access.

A custodian would be responsible for making sure that their agency’s data was accessible and maintained, as guided by the Well Managed principle.

They also act at the contact person for queries or requests about the data. It is recommended that a named contact point, telephone number or email address (ideally both) is listed in the metadata of your dataset: What metadata should I include with my dataset?


Source: https://data.govt.nz/manage-data/data-stewardship/what-is-a-data-custodian-and-what-do-they-do/
Do you have documentation to help users understand what the values in your dataset mean?

Guidance

Creating a data dictionary

Data dictionaries are useful information to include alongside your datasets. They help describe the elements and values contained within your data to help users reuse it. A simple data dictionary can be created quickly and should include a few key piece of information.

Learning outcomes

  • Understand what a data dictionary is
  • Know the advantages of having a data dictionary
  • Learn how to create a basic data dictionary and what to include

What is a data dictionary?

On a basic level, a data dictionary is metadata that describes the values contained within the dataset, how they have been collected and any standards the data conforms to. This includes the data type, format, size, descriptions and how the data is used. This helps users understand how the data is structured and the relationship that data has with other data.

What are the benefits? 

  • Improved data quality - Creates more detailed and depth to the data, making it have more uses.
  • Reuse of data - By creating higher quality data, the reuse of that data is encouraged.
  • Consistency in data use - By sticking to high standards consistency is present in data.
  • Faster and easier data analysis - The data is structured in a simple manner, making it take less time to analyse the data.

Creating a simple data dictionary

Creating a simple data dictionary can be done in a csv or other spreadsheet file and included alongside your dataset. Below are a set of attributes you should look to include. You can add further attributes to suit your needs for exmaple how you collected the data, any caveats of use etc. We provide a template to get you started at the bottom of the page.

  • Data name - The name of the column as it appears in your data. For example if you needed to put a field name for a driver’s licence number you might use "licence_id.
  • Description  - A short description for the type of data that is readable for humans e.g. Driver licence number.
  • Data type - If it’s a number you might list as type "integer", if it's a name you may use "text" etc
  • Data format - If it’s a number for example put N and the amount of Ns for the amount of numbers, for date put DD/MM/YYYY etc. You might also list any standards you have followed, ISO8601 for date formats for example.
  • Field size - What is the max amount of characters for the data type e.g. 20 for a name.

Data Dictionary Template CSV

 

 

Additional Resources

More info about what a data dictionary is

More info on how to make a data dictionary


Source: https://data.govt.nz/manage-data/data-stewardship/creating-a-data-dictionary/
What file format is your data in?
Is your data available in both human and machine readable file format?

Guidance

Formats for open data, machine readable and human readable

Open data should be provided in a format that makes it easy for others to re-use. This often means it should be in a machine readable format. The New Zealand Government Open Access and Licensing framework (NZGOAL), the all-of-government guidance for agencies to follow when releasing material for re-use, provides the best format options for the different types of data you might release.

Learning outcomes

• Understand what a machine readable format is.
• Understand which file formats are best for re-use.
• Select the appropriate file format for your data.

Document formats such as PDF, Word and sometimes Excel are considered human readable. They often contain extra formating, colours and styles that help people read them. However this can make them unsuitable formats for providing data for re-use and may mean a lot of time is spend cleaning the data up to be repurposed for other uses.

JSON and CSV are two machine readable formats that are recommended by the NZGOAL framework, read on below for a full set of guidance on file formats suitable for open data.

Detailed guidance


Source: https://data.govt.nz/manage-data/releasing-data-on-data-govt-nz/formats-for-open-data-machine-readable-and-human-readable/
Have you made your data files publicly accessible?

Publicly accessible data files

So that others can access and download your data files you will need to make these available publicly on the internet.

Options include:

  • On your agency's website
  • In another data portal your agency maintains (as long as it's publicly accessable)
  • Hosted on data.govt.nz's catalogue (contact info@data.govt.nz for more information on this option)

Geo-spatial portals

Because you are looking to release a geo-spatial dataset you may also want to look at some of the available options for hosting and servicng these types of files:

  • Koordaintes
  • Arc Gis Open Data portal
  • Geonetwork

Most of the common geo-spatial options offer a level of automatic updates to data.govt.nz.

Have you (or someone in your agency) applied for a data.govt.nz Publisher Account?

Guidance

What is a publisher account and how do I apply?

Government agencies can publish and update their datasets on the data.govt.nz catalogue by applying for a publisher account.

Learning outcomes

  • Learn what a data.govt.nz publisher account is
  • Find out if your agency is eligible for a publisher account
  • Know where to request a publisher account and what information to provide

A publisher account allows you to add datasets to data.govt.nz, and edit existing datasets, and metadata, on behalf of your agency.

Eligible agencies

Data.govt.nz is the central catalogue of Government Open Data in New Zealand including data from:

  • Central Government
  • Local and regional Government
  • Tertiary education institutions
  • Crown entities

Other data releasing organisations can also apply for a publisher account - contact us to discuss your needs.

 

How to apply

To apply online, you will need to provide:

  • your name
  • the agency you will be publishing on behalf of
  • your work email address

You will also need to agree to the Publisher Terms and Conditions and the General Terms and Conditions of using data.govt.nz

The team will confirm with you what kind of data you’ll be publishing, so we can tailor our guidance, before working with you to add and manage datasets on data.govt.nz.

 

Where to apply

If you need to publish and maintain your agencies data on data.govt.nz, and do not yet have a publisher account:

 

Apply for publisher account

 

Once you have a publisher account, you’ll be able to login from the Publisher login link from the bottom of data.govt.nz.


Source: https://data.govt.nz/manage-data/releasing-data-on-data-govt-nz/what-is-a-publisher-account-and-how-do-i-apply/

Ready to publish

You should have everything you need to release an open dataset on data.govt.nz's catalogue.

A final checklist is available on the next step of this guide.

Top