Highlights

  • Designed intelligent search API to make archival records easily discoverable
  • Applied machine learning and natural language processing to make searching easy
  • Integrated Contemporary API Management & Analytics to make 3rd Party on-boarding simple and low-risk
  • Designed User Generated Content APIs to enforce strong attribution policies, permitting future rapid incorporation and reuse.

The Challenge

The National Archives has two primary activities. The first being the archiving of significant documents (both digital and paper) for the good of the country. The second, is that of providing access to the enormous archived information for Citizens and other 3rd parties.

The challenge comes in how do you provide and promote rich, simply access to the archives data for people untrained in the art of archiving; since requiring the general public to understand the nuances of archival categorisation is a barrier for usage. On top of this, how do you enable low-friction integration by contemporary applications whilst maintaining a reasonable level of control.

Our Goals

Solution design of a contemporary API based system that permitted:

  • a single box search ability for Citizens without inhibiting the more advanced searching archivists expect
  • simple incorporation by Citizens and other 3rd Parties with full self-service developer support and on-boarding.
  • Automatic and light-weight Fair-use and other API usage protections

Technology

The API was designed to be exposed to the internet via a enterprise class API management platform such as 3 scale or Apigee. These provide the low friction on-boarding of new 3rd party developers and their applications, whilst maintaining Terms of Service and other legal frameworks for data protection.

Providing the simple search was facilitated through the use of Machine Learning to enhance data stores with relevant searchable information through the industry standard full-text searching capabilities of Lucene – Solr.

Result

The Design of the National Archives of Australia’s services for controlled, valuable, and contemporary internet access to their vast public archive data was a full success. The proof of concept provided a solid basis for the organisations future product delivery.