Agile Digital

Background

The National Archives has two primary activities. The first being the archiving of significant governmental documents (both digital and paper) as a national asset. The second is providing access to the enormous corpus of archived information for the benefit of citizens and other 3rd parties.

Agile Digital’s design of a contemporary API-based solution permitted:

• A single box search ability for citizens without inhibiting the more advanced searching archivists expect
• Simple incorporation by citizens and other third-parties with full self-service developer support and onboarding
• Automatic and light-weight Fair-use and other API usage protections

Mission

The challenge comes in finding a way to provide and promote rich yet simple access to the archive’s data for people untrained in the art of archiving, whilst also enabling low-friction integration with modern applications and maintaining a reasonable level of control. The system achieves this through the elimination of entry barriers, avoiding the nuances of archival categorisation that casual users don’t need.

Highlights

• Designed intelligent search API to make archival records easily discoverable
• Applied machine learning and natural language processing to make searching easy
• Integrated Contemporary API Management & Analytics to make 3rd Party onboarding simple and low-risk
• Designed User Generated Content APIs to enforce strong attribution policies, permitting future rapid incorporation and reuse.

Approach

The API was designed to be exposed to the internet via an enterprise-class API management platform such as 3 scale or Apigee. These provide the low friction on-boarding of new third-party developers and their applications whilst maintaining Terms of Service and other legal frameworks for data protection.

Providing the simple search was facilitated through the use of machine learning to enhance data stores with relevant searchable information through the industry standard full-text searching capabilities of Lucene – Solr.

Result

The design of the National Archives of Australia’s services for controlled, valuable, and contemporary internet access to their vast public archive data was a full success. The proof of concept provided a solid basis for the organisation’s future product delivery.