In New Zealand, the Official Information Act (OIA) is a pivotal piece of legislation that empowers citizens to request access to information held by government agencies and ministers. This transparency is fundamental to ensuring accountability, enhancing public trust, and fostering an open government. However, there are various practical challenges associated with ensuring compliance with OIA requests, particularly in the digital age where password protection of documents, often for legitimate reasons, can unintentionally hinder the public’s right to access information.
This issue was recently brought into focus when a government agency sought our expertise to assist in fulfilling a discovery request for all password-protected documents across their entire IT infrastructure. Password-protected files, while useful for security and confidentiality, can often create roadblocks in situations where transparency is legally mandated. The task required a careful balance between protecting sensitive data and being ready to comply with OIA obligations. In this article, we’ll explore how we approached this challenge, detailing the tools and strategies we employed, the limitations we encountered, and the broader implications for organisations navigating the intersection of security and public accountability.
The Legal and Operational Context: Understanding the OIA and Its Implications
The Official Information Act (OIA), enacted in New Zealand in 1982, provides the public with the right to request and obtain official information from government bodies, unless an exception applies. Underpinned by the principle of availability, the Act is designed to promote transparency and ensure that government actions remain open to public scrutiny. There are specific grounds under the OIA on which agencies can withhold information, including considerations of national security, legal privilege, personal safety, and privacy. In cases involving special categories of official information, agencies have limited grounds to withhold information, reinforcing the priority of transparency. However, non-compliance with the Act may result in complaints to The Ombudsman leading to investigations that could result in reputational damage for the agencies involved.
In today’s digital era, where vast amounts of data exist, often stored across a plethora of platforms and potentially secured with password protection, ensuring access to information has become a more intricate task. Government agencies routinely handle sensitive information, from confidential contracts to personal data. To safeguard this information, staff may apply passwords to files, often with good intentions, such as protecting personal information or intellectual property. However, these security measures can sometimes result in unintended barriers, where information that should be accessible under the OIA becomes effectively concealed behind layers of security.
This challenge is compounded by the Act’s timeframes, which require government agencies to respond promptly. Agencies must decide and communicate to requesters as soon as reasonably practicable, and no later than 20 working days after receiving a request. Moreover, once a decision is made to release information, it must be provided to the requester without ‘undue delay.’ These timeframes amplify the challenges for agencies, as they must work efficiently through password-protected files and other security measures to ensure compliance.
By navigating these issues thoughtfully, agencies can maintain both public trust and compliance with the OIA.
The Challenge: Discovering Password-Protected Documents
The government agency we worked with wanted to identify all password-protected documents within their IT ecosystem, which primarily relied on Microsoft 365. This presented the first of several challenges: How could we effectively search for password-protected files across a sprawling and complex cloud infrastructure without any in-built tools for this specific task?
We knew the solution would require a mix of creativity, technical expertise, eDiscovery and forensic talent. But before diving into the solution, it’s important to understand the two key cloud platforms we were dealing with: Exchange and SharePoint/OneDrive.
Microsoft Exchange: The Simplest Path to Discovery
When dealing with Exchange (which powers Microsoft Outlook for email services), the search for password-protected documents was straightforward using Microsoft Purview.
Through Purview’s eDiscovery, we exported all “unindexed items.” Unindexed items, according to Microsoft, are files or messages that cannot be indexed for one of several reasons, such as unsupported file types, file size limitations, or, crucially for our needs, password protection or encryption.
This feature was perfectly suited for our purposes. By identifying unindexed items in Exchange, we could isolate potential password-protected documents. Once exported, we used eDiscovery and forensic tools to examine these files further and specifically isolate those that were password-protected. Importantly, this process also included data from Microsoft Teams, which is integrated into the Microsoft 365 ecosystem and relies on the same eDiscovery platform.
SharePoint and OneDrive: A More Complex Challenge
After our success with Exchange, we moved on to tackling SharePoint and OneDrive. SharePoint and OneDrive do not generate “unindexed items” in the same way Exchange does, meaning that running the same approach would not have an effective outcome.
To address this, we had to develop a more customised solution. First, we created scripts to connect directly to the SharePoint Online platform and perform a comprehensive scan of all documents stored within the system. This included documents from both SharePoint sites and OneDrive for Business, which uses SharePoint as its backend for data storage.
The scripts were designed to inspect every file across the entire infrastructure, specifically focusing on common file formats that are often password-protected, such as Microsoft Office files (Word, Excel, PowerPoint), PDFs and compressed files (e.g., zip, 7z, etc…). The scripts analysed these files for password protection or encryption and generated a report detailing all password-protected files, including the files’ metadata.
Balancing Security and Transparency
This project highlighted an ongoing challenge that many government agencies face: balancing security risk management with maintaining accessibility. Protecting sensitive information is a top priority, and implementing security measures such as password protection is one option. However, when it comes to fulfilling OIA requests, these measures can sometimes hinder access to information that should remain available. Striking the right balance between security and accessibility is essential to uphold the principles of both transparency and information security. This tension between security and transparency is particularly relevant for public sector organisations, which are subject to the OIA while also needing to protect sensitive government, personal, or confidential information. In an ideal world, organisational policies and procedures must be carefully managed and reviewed to ensure they don’t potentially block compliance with legal obligations.
Recommendations for Government Agencies
Our experience in handling this discovery request has provided valuable insights into how government agencies can better manage password-protected documents while ensuring compliance with the OIA. Below are several recommendations based on our findings:
Regular Audits of Password-Protected Files: Government agencies should implement regular audits to identify and review password-protected files across their infrastructure. This proactive approach can prevent issues from arising when OIA requests are received. By knowing where password-protected files are stored, agencies can ensure they maintain access to information when needed. By undertaking this audit, agencies may uncover staff training opportunities.
Use of Centralised Document Management Systems: Agencies should adopt centralised document management systems that provide greater control over file security. Systems that allow for role-based access control (RBAC) and auditing of file access can help maintain the right balance between security and transparency.
Leverage eDiscovery Tools for Compliance: Microsoft’s Purview eDiscovery tool proved to be highly effective for identifying a subset of data that was likely to include password-protected files in Exchange and Teams. Agencies should consider defining a process ahead of OIA requests being received in how to search and identify relevant data within non-searchable / non-indexable content.
Clear Policies on Password Protection: Agencies should implement clear policies regarding the use of password protection on documents, particularly those that might be subject to OIA requests. These policies should outline when and how password protection can be applied, and who has access to the passwords and where the passwords should be stored.
Custom Solutions for SharePoint and OneDrive: Since Microsoft’s native tools do not fully support discovery of password-protected files in SharePoint and OneDrive, agencies may need to develop or adopt custom scripts for this purpose. Alternatively, agencies may need to partner with specialists who have experience in this domain.
Conclusion: The Path Forward
Handling OIA requests in a world of increasing digital security measures is a challenge that many government agencies face. While protecting sensitive information is a necessity, agencies must also ensure that they can be fully compliant with the OIA and other transparency obligations.
By leveraging the right tools, developing custom solutions where necessary, and implementing clear policies, government agencies can ensure they strike the right balance between security and transparency.
Our experience with this government agency underscores the importance of being prepared. While they did not have an OIA to respond to, they knew some documents that could be part of an OIA request were password-protected, which led to a wide-scale search to ensure any other documents were identified. In our view, the government agency involved was being proactive and agile in responding to this potential risk, fundamentally looking to ensure that they could maintain public trust and show best efforts to address the risk.