The Project

The National Archives has two primary activities. The first being the archiving of significant governmental documents (both digital and paper) as a national asset. The second is providing access to the enormous corpus of archived information for the benefit of citizens and other 3rd parties.

Agile Digital’s design of a contemporary API-based solution permitted:

  • A single box search ability for Citizens without inhibiting the more advanced searching archivists expect
  • Simple incorporation by Citizens and other 3rd Parties with full self-service developer support and on-boarding.
  • Automatic and light-weight Fair-use and other API usage protections

The Mission

The challenge comes in finding a way to provide and promote rich yet simple access to the archives data for people untrained in the art of archiving, whilst also enabling low-friction integration by contemporary applications and maintaining a reasonable level of control. The system achieves this through the elimination of barriers to entry, avoiding the nuances of archival categorisation unnecessary for casual public users.

Highlights

  • Designed intelligent search API to make archival records easily discoverable
  • Applied machine learning and natural language processing to make searching easy
  • Integrated Contemporary API Management & Analytics to make 3rd Party on-boarding simple and low-risk
  • Designed User Generated Content APIs to enforce strong attribution policies, permitting future rapid incorporation and reuse.

Technology

The API was designed to be exposed to the internet via a enterprise class API management platform such as 3 scale or Apigee. These provide the low friction on-boarding of new 3rd party developers and their applications, whilst maintaining Terms of Service and other legal frameworks for data protection.

Providing the simple search was facilitated through the use of Machine Learning to enhance data stores with relevant searchable information through the industry standard full-text searching capabilities of Lucene – Solr.

Result

The Design of the National Archives of Australia’s services for controlled, valuable, and contemporary internet access to their vast public archive data was a full success. The proof of concept provided a solid basis for the organisations future product delivery.