Measuring Lexical Quality of a Historical Finnish Newspaper Collection – Analysis of Garbled OCR Data with Basic Language Technology Tools and Means

Kimmo Tapio Kettunen, Tuula Anneli Pääkkönen

    Research output: Chapter in Book/Report/Conference proceedingConference contributionScientificpeer-review

    Abstract

    The National Library of Finland has digitized a large proportion of the historical newspapers published in Finland between 1771 and 1910 (Bremer-Laamanen 2001). This collection contains approximately 1.95 million pages in Finnish and Swedish. Finnish part of the collection consists of about 2.39 billion words. The National Library’s Digital Collections are offered via the digi.kansalliskirjasto.fi web service, also known as Digi. Part of this material is also available freely downloadable in The Language Bank of Finland provided by the Fin-CLARIN consortium. The collection can also be accessed through the Korp environment that has been developed by Språkbanken at the University of Gothenburg and extended by FIN-CLARIN team at the University of Helsinki to provide concordances of text resources. A Cranfield-style information retrieval test collection has been produced out of a small part of the Digi newspaper material at the University of Tampere (Järvelin et al., 2015). The quality of the OCRed collections is an important topic in digital humanities, as it affects general usability and searchability of collections. There is no single available method to assess the quality of large collections, but different methods can be used to approximate the quality. This paper discusses different corpus analysis style ways to approximate the overall lexical quality of the Finnish part of the Digi collection.
    Original languageEnglish
    Title of host publicationProceedings of the Tenth International Conference on Language Resources and Evaluation (LREC 2016)
    Number of pages6
    PublisherEuropean Language Resources Association (ELRA)
    Publication dateMay 2016
    Pages956-961
    ISBN (Print)978-2-9517408-9-1
    Publication statusPublished - May 2016
    MoE publication typeA4 Article in conference proceedings
    EventWorkshop on language engineering for online reputation management - Portoroz, Slovenia
    Duration: 25 May 201627 May 2016
    Conference number: 10 (LREC)

    Fields of Science

    • 6121 Languages
    • 113 Computer and information sciences

    Cite this