This aide memoire updates you on feedback we’ve received from government agencies on the Algorithm Charter for Aotearoa New Zealand, to support your discussion at the Digital and Data Ministers meeting on 4 March 2020.
Aide memoire to the Minister of Statistics: Feedback from agencies on the Algorithm charter [PDF 1.4 MB]
The development of an algorithm charter was endorsed by Data and Digital Ministers in June 2019 and officials have since consulted on a draft version and made updates in response to public feedback.
Ahead of the upcoming Digital and Data Ministers meeting and as we discussed on 17 February, officials circulated the revised Algorithm Charter for Aotearoa New Zealand to gauge agency comfort with the proposed text.
While there is strong support for a Charter generally, some agencies believe that the expanded scope (extending from operational algorithms to all algorithms) will be unworkable and they would not be willing to sign the Charter on that basis. This includes agencies who use significant volumes of data and analytical techniques in their work.
As we have advised you previously, the views of many of the public submissions, particularly those from NGO’s that promote government transparency and public rights, are to remove the distinction between operational and all algorithms.
To support your discussion of the possible scope of the Charter with your digital and data colleagues, we’ve identified three options to discuss at the meeting, reflecting the different perspectives we’ve heard.
Published 27 February 2020.
Overall, agencies support the intent of the Algorithm Charter for Aotearoa New Zealand and believe that a Charter can provide assurance for the public in how government is using algorithms. Several agencies have indicated that they’re prepared to sign the current version of the Charter.
As we noted in our report of 13 February, during our previous discussions with some data agencies, there was a clear preference to limit the scope of the Charter to only apply to operational algorithms (those that have a significant impact on people or groups) due to the resource implications of retroactively reviewing current algorithm settings on a wider basis.
The feedback we’ve recently received from public consultation suggests that many submitters see no reason for the scope of the Charter to be limited to only operational algorithms, with submitters suggesting that government agencies should apply the kind of good practice set out in the Charter to all of their data-driven decision-making processes.
Based on the Algorithm Assessment report (2018), confining the scope to operational algorithms would limit the number of algorithms within scope to between one and six algorithms per agency. We understand that widening the scope to include all algorithms and to business rules (simple decision-making processes) would significantly increase the number per agency.
In our report of 13 February, we proposed removing the distinction between operational and non-operational algorithms to respond to public feedback, and limiting the initial scope of the charter to new algorithms, thereby providing a 5-year period before it would apply to all algorithms to enable agencies to plan and manage applying it more broadly.
In their recent feedback, some agencies have maintained their view that they can only support a Charter that is limited in initial scope to operational algorithms. They suggest that a widened scope would have high compliance costs, and ultimately lead to potential delays in delivering analytic work and reduced transparency as the public could find it difficult to determine which algorithms have the greatest impact on decisions affecting them.
Comments on scope were a major theme of the public consultation process, with more than half of submissions received providing views on whether the Charter should apply only to operational algorithms. We expect that responding to this feedback would improve the Charter’s impact on public trust and confidence.
Excluding business rules from the scope is one possible solution, that would respond to the feedback from public consultation to remove the distinction between operational and nonoperational algorithms, while also responding to the concerns of large data agencies about the quantum of items that might fall within an expanded scope.
An alternative approach could be to consider staged implementation of the Charter, starting by applying commitments to operational algorithms and expanding the commitments to all algorithms, excluding business rules, after an agreed period of time. This would provide a phased approach for implementation and could respond to public feedback on the Charter’s scope.
We provide more detail about each of these options below to support a discussion between digital and data Ministers about the scope of the Charter. We’ve also attached a version of the Charter reflecting each option for reference.
We propose to use the same set of definitions that were set out in the 2018 Algorithm Assessment Report. Under this definition an algorithm is an analytical process that interprets or evaluates information (often using large or complex data sets) to identify patterns and trends to inform decisions.
Under this option, the Charter would initially apply to all new algorithms, as identified by the signatory. Over 5 years the signing agency would be expected to apply commitments to existing algorithms.
We consider this to be the highest-impact option that would cover a large volume and wide range of algorithms (at this stage we do not have confirmation of the number of algorithms this would apply to across the system), and as a result would have the highest impact on public trust and confidence in government. This is also the highest-resource option for signing agencies.
Under this option, the Charter would initially apply to all new algorithms except Business Rules, and over 5 years the signing agency would be expected to apply commitments to existing algorithms that are not Business Rules.
Business Rules, as defined in the Algorithm Assessment Report, are: Simple algorithms created by people that use rules to constrain or define a business activity. They make determinations about individuals or groups, without a significant element of discretion.
We consider this to be a similar approach to that of the Law Foundation in their report on Artificial Intelligence (2019). Most Business Rules are likely to be of less interest or concern to the public, and this option would mean that agencies can focus efforts and resources on reporting on the rest of their new algorithms.
While some agencies have signalled support for removing Business Rules from the scope of the Charter, most have advised us that this scope change alone is still too broad without a staged implementation.
Under this option, the Charter would apply to operational algorithms only. This draws on the definition of algorithms used for the Algorithm Assessment Report: These algorithms result in, or materially inform, decisions that significantly impact on individuals or groups. They may use personal information about the individuals or groups, but do not need to do so exclusively.
This option does not address concerns raised by a majority of public submissions on this topic about the scope of the Charter. However, this option has the lowest-resource implications for signing agencies, making the prospect of a large number of agencies signing the Charter more likely, and shows government progress in this important and evolving area of data ethics.
Under this option, implementation of Charter commitments would apply to operational algorithms for the first 2 years after signing and then to all algorithms used by an agency, excluding business rules.
This option could address public submissions about the scope of the Charter and allow for a staged implementation for agencies. We would report on the progress of implementing the Charter during the 2-year period to ensure that the next stage of implementation is achievable and meets the intended objective of the Charter.
This option has low resource implications in the immediate term for signing agencies, which we believe would encourage agencies to sign and would also show government progress in this important and evolving area of data ethics. We understand this option has the broad support of agencies.
Some of the other feedback we’ve received from agencies has been about the implementation of the Charter, once the scope is agreed. As we have advised you previously, we have looked to the Ministry of Social Development Accessibility Charter as a model for the implementation of a voluntary charter by government agencies. We’ve been working with the officials involved in that Charter to learn from their experience, and we plan to develop further support and guidance for agencies in due course.
The Charter is one part of the work across the data system to improve the transparency and accountability of data use. Other examples of activity that is already established include the Data Protection and Use Policy developed by the Social Wellbeing Agency (formerly the Social Investment Agency), the Privacy Human Rights and Ethics tool developed by the Ministry of Social Development and our establishment of an independent Data Ethics Advisory Group.
As we noted in our report of 13 February, we have already made some updates to the Charter in response to the public feedback. These changes include clearer alignment with the Principles for the Safe and Effective Use of Data and Analytics, strengthening the commitment for agencies to actively consult with people, communities and groups interested in, or impacted by, algorithms and use of more accessible plain English text.
We recommend you discuss the options for amending the scope of the Charter at the meeting of Digital and Data Ministers on 4 March and agree on a preferred option.
Depending on the outcome of the discussion at the Digital and Data Ministers meeting on 4 March, signatories for the endorsed Charter will be discussed by Chief Executives at the Digital Government Leadership meeting on 9 March.
If you’d like more information, have a question, or want to provide feedback, email datalead@stats.govt.nz.
Content last reviewed 21 March 2021.