The Power of Annotation

We most of us believe that we know what annotation is. However, the basic concept has been applied to several different fields, for quite different purposes, and in many different ways. A review of the landscape for textual annotation was very useful to me, and I hope that others may find this useful too.

A term that goes hand-in-hand with annotation is mark-up (or “markup” in the US), to the extent that they have become virtually synonymous in certain areas. One of the first things to consider is the origin of the two terms, and how their meanings may have shifted over time.

I wanted to call this article “marking up the wrong tree”, but obscure titles are not always the best policy, no matter how side-splittingly hilarious they may seem to you. [Pull yourself together Tony]

According to the dictionary, to annotate is “to add notes to (a text or diagram) giving explanation or comment”, and an etymology is given of “Late 16th century: from Latin annotat- 'marked', from the verb annotare, from ad- 'to' + nota 'a mark'”.[1] This is probably the first usage that most of us would think of.

Figure 1 – Annotated page of text.[2]

As an aside, the analysis of an annotated document is interesting because it often involves a mixture of primary and secondary information whose layers must be considered individually, although not separately.

The term mark-up originates from the annotation of manuscript (and manual typescript) documents with symbols providing printer’s instructions, including corrections, layout, and typesetting. Similar systems of symbolic annotation are used in the field of textual scholarship, which is a collective term for textual studies that encompass analysis, description, transcription, editing, or annotation of texts. The branch of textual scholarship known as diplomatics (not to be confused with diplomacy) involves the scholarly analysis of documents and texts. In particular, a diplomatic transcription reproduces an historic manuscript as accurately as possible (a diplomatic edition) in typography, and including significant features such as original spelling and punctuation; contractions, suspensions, and other abbreviations; insertions, deletions, and other alterations; obsolete characters such as thorn and eth; superscript and subscript characters, and brevigraphs (e.g. the ampersand); these usually employ a system of mark-up in order to capture their essence in a modern typeface. A semi-diplomatic transcription relaxes the requirement for accuracy, usually for readability or practicality. For instance, some original forms are difficult to reproduce in simple typescript, particularly if the original was already marked up by hand, but more on that later. Mark-up may also be used during peer review of a document, or by an author themselves. One more field that I have to mention is corpus linguistics, or the analysis of language using selections of natural text compiled from transcribed writings or recordings (corpora). This uses annotation for such things as tagging parts of speech (POS tagging), e.g. “corpus_NN1 annotation_NN1 is_VBZ hard_AJO” where the suffixes categorise the words (e.g. noun, adjective).

We’ve seen that annotation may actually be symbolic or textual, and that mark-up often includes text as well as symbols or editorial marks. So what is the difference? In his work on corpus linguistics, Martin Weisser comes to the following conclusion:[3]

While the term markup is sometimes used to indicate the physical act of marking specific parts of a text using specific symbols, and, in contrast, annotation may often refer to the interpretive information added, the two may also be used synonymously.

It would seem that the modern usage of these terms employs annotation for the addition of meta-data (related textual or other information) to the text, and mark-up for the scheme by which such annotation is represented or encoded.

This is born out by the concept of mark-up languages, which are systems for annotating a document that are syntactically distinguishable from the text, and hence more structured than mere symbols or marginal notes. A very important distinction has to be made, therefore, between the following types of mark-up:

1. Handwritten mark-up, as applied to a manuscript or typescript document.
2. Typed mark-up, as applied to a typescript or digital document.
3. Mark-up language, as typically applied to a digital document.

The first two of these are designed to be humanly-readable whereas the third type is designed to be computer-readable and so must involve grammatical rules that allow it to be parsed by software. To illustrate the difference, consider the following corrected sentence:

My favourite colour is blue red.

A representation of this using a simple typed mark-up (type 2, above) might be:

My favourite colour is <blue> ^red^.

whereas a mark-up language (type 3) might encode it as follows:

My favourite colour is <del>blue</del> <ins>red</ins>.

These may appear equivalent from a visual perspective, but consider the consequences if the altered text contained either angle brackets or carets, or if the replacement word required some clarification — a mark-up language would be able to represent these cases unambiguously so that software could process it. Also, since a mark-up language is designed purely to communicate the information to software, it means that the representation of that same information to the end-user is not fixed, and the choices would be dependent upon the capabilities of the display medium and the sophistication of the display software.

This leads us nicely to perhaps the two best-known mark-up languages: HTML (HyperText Markup Language) and XML (Extensible Markup Language). HTML was created with predefined semantics (for creating Web pages) but XML was created as a general-purpose syntax with no predefined semantics. Interestingly, the semantics associated with HTML have been refined since its initial development; the last example, above, shows a modern Semantic HTML form, but an older form might have been:

My favourite colour is <s>blue</s> red.

This is literally encoding the visual representation, as in the first example sentence, above. The modern shift in emphasis is from presentation to content structure, such that the mark-up would now show what was deleted and what was inserted rather than simply that a line was drawn through a word.

So why might someone use XML rather than HTML when transcribing an historical document? Well, despite that shift in emphasis, HTML is not a good tool for transcribing text. Consider a document that has original emphasis, such as underlines added by the author, or which has already been marked up by an editor; this information has to be preserved and yet be distinguishable from anything employed during the transcription process, and these are not just presentational matters — there are semantics associated with the original formatting. With a mark-up language such as XML then you have the flexibility to represent all the different types and levels of information without any conflict or ambiguity. Both TEI (Text Encoding Initiative) and STEMMA employ mark-up languages with support for transcription, and both have XML representations.

Using the terminology from Markup_language, there are several forms of mark-up that are required for micro-history narrative:

• Descriptive: Marking the text in order to capture its structure and content, rather than specific visualisations of it. Ultimate control over explicit physical rendition such as colour, bold, italic, underline, font name, and font size are best left to the tool presenting the text (e.g. HTML+CSS).
• Presentational: This mark-up would be essential for a faithful transcription of something. Although modern systems (such as HTML5) frown on explicit presentational information, it may provide important information necessary for the analysis and correct interpretation of transcribed material. STEMMA’s approach to transcription separates structure and content from presentational or stylistic matters: see Descriptive Mark-up.
• Semantic: Although the aforementioned wikipedia link suggests that this is an alternative name for Descriptive mark-up, the usage here is more distinct. This mark-up provides information about the meaning or interpretation of textual references. It is therefore different from the structure and layout in a purely textual context, and is precisely what is needed to identify entities such as Persons and Places.

Semantic mark-up is especially important for narrative essays and narrative reports stored in a genealogical context. Although both TEI and STEMMA have their own schemes, there is a divergence that will become more important once the genealogical industry acknowledges a narrative requirement: the semantics are not independent of the data model. This may be hard to explain, but simply flagging a name as that of a person or place — irrespective of whether it makes a conclusional identification — is an isolated semantic that is addressed in a roughly similar fashion by the two schemes. However, linking such a reference into a chain of conclusion-evidence-information-source would not make any sense outside of a genealogical data model. In effect, TEI is a very comprehensive text-encoding scheme but it cannot deal with semantics associated with an all-embracing data model.

A familiar form of mark-up that we might encounter in wikis or blogs is a lightweight markup language. These have a simple syntax that can be entered directly by the editing user, as opposed to being generated in response to some graphical operation or option selection. Although still designed to be computer-readable, they are easier for a human to read — and, hence, to write. For instance:

**bold text** __underline text__   //italic text//

When looking at the mechanics of adding mark-up to an electronic document then there are two very different approaches. The most common is inline, or embedded, mark-up, where the mark-up language is interwoven with the text in a manner such that it can still be distinguished. For example:

Here is a link: <a href="http://parallaxview.co/stemma/">STEMMA</a>

The alternative is known as stand-off, or remote, mark-up and involves holding the mark-up in a separate file (or other location) to the underlying text, usually linking them by character coordinates. The concept of stand-off mark-up is attributed to Henry Thompson and David McKelvie in 1997,[4] and the advantages include:

• The ability to mark-up read-only (protected) or very large files.
• The ability to support mark-up from independent editors, held as separate layers, and without them having to form a single code hierarchy.
• The ability to combine disjoint segments into a single annotation.

Others are stated but I’m less convinced of their value. In contrast, the advantages of inline mark-up include:

• Simplicity. One file to maintain or distribute.
• The text and mark-up are edited together, with less chance of them getting out-of-step.

Which is best really depends on the application requirements.

A common example of stand-off mark-up, which isn’t always viewed as such, is CSS (Cascading Style Sheets). It was mentioned above that Semantic HTML favours content structure in place of presentation. This works because modern HTML now goes hand-in-hand with CSS, which can describe the presentational aspects in a separate file. Rather than being linked by character coordinates, they are linked by such things as element type and class, collectively described as selectors, which may explain why CSS is rarely described as stand-off mark-up. In effect, HTML then becomes an inline mark-up describing content that relies on a stand-off mark-up for presentation. The advantages of being able to change the overall presentation style of a Web page in a consistent way, or share the style between multiple pages, should be clear.

I want to round off this review of annotation with a quick mention of the humble word-processor. So familiar and useful is this tool that we give little consideration to how it works, or what goes on inside — oh, how I wish genealogy would catch up there. It allows the end-user to add presentational mark-up (e.g. bold, or a specific font-face) and semantic mark-up (e.g. a hyperlink, or a review comment), but you don’t see the associated mark-up. The associated mark-up language is complicated and so made deliberately invisible to the end-user. The net effect of that is to reinforce the user-interface model and give the impression that the end-user is somehow annotating the visible text directly. This is an important distinction — that a hidden nuts-and-bolts mark-up supports the notion, and the physicality, of annotation in the user interface — and it should be an important consideration for future genealogy tools. There is no excuse for expecting the end-user to edit the raw mark-up rather than using a WYSIWYG (“What You See Is What You Get”) interface.

---

[1] Oxford Dictionaries Online (http://www.oxforddictionaries.com/us/definition/english/annotate : accessed 1 Mar 2016), s.v. “annotate”.

[2] John Keats, “Ode to a Nightingale” (1819); image credit: Ryan Johnson (https://www.flickr.com/photos/kmonojo/4288773728 : accessed 1 Mar 2016); Attribution-ShareAlike 2.0 Generic (CC BY-SA 2.0).

[3] Martin Weisser, Practical Corpus Linguistics: An Introduction to Corpus-Based Language Analysis (John Wiley & Sons, 16 Feb 2016), ch.11.

[4] Henry S. Thompson and David McKelvie, “Hyperlink semantics for standoff markup of read-only documents”, May 1997, technical report, Language Technology Group, HCRC, University of Edinburgh (http://www.ltg.ed.ac.uk/~ht/sgmleu97.html : accessed 2 Mar 2016).

Evolution and Genealogy

In this post, I want to pose some thoughts concerning the evolution of genealogy, as opposed to the genealogy of evolution, and consider how my recent visit to RootsTech 2016 may have tempered my prior views.

When asked how genealogy has progressed in recent decades, many people would cite the increased availability of information through the digitisation of records and other sources. But what about the process of genealogy: the methodology, attitudes, and tools?

Figure 1 - Tree of Life, Ernst Haeckel (1834–1919).[1]

A shining light in genealogy is the Genealogical Proof Standard (GPS). Although controversially using the term “proof” in its name, this is essentially a guide to ensuring thorough research, and of reaching sound conclusions. It is not a method (as sometimes thought), and not even a step-wise recipe — hence, its five parts being described as “elements” rather than “steps”. The earliest mention of the GPS by name was by Helen F. M. Leary in "Evidence Revisited — DNA, POE and GPS," OnBoard 4 (January 1998): pp.1–2, 5, and the text of that article may be found online at http://www.bcgcertification.org/skillbuilders/learyevidence.html. Although the actual wording of this standard has undergone a number of refinements before reaching its current form, the underlying principles can be traced back to at least the 1930s, and probably the late 19^th Century. It is hardly surprising, therefore, that those same elements (with minor wording changes) would apply to any field of research, including those outside of the fields of genealogy and history, and including those that do not use the term “proof” in this manner. The expression of these principles was designed to help all genealogists, not just professionals, or even budding professionals, but also the many newcomers to this increasingly popular pursuit.

Another major contribution that has been embraced by the wider community is Evidence Explained: Citing History Sources from Artifacts to Cyberspace (EE) by Elizabeth Shown Mills. Now in its third edition, this mammoth book is sometimes viewed as a recipe book for source citations, and is unfairly criticised when the one you really want is unlisted. However, the book is really about evidence analysis, and the first couple of chapters guide the reader through the things they need to know — the principles, conventions, and requirements — to be empowered to do their own analysis and cite their own sources. There is a lot to learn, here, but it’s well worth the effort. Although not the first such book, it may be argued that this particular one set the scene for the more recent works designed to help genealogists make a better job of what they love doing.

What about software genealogy? You would think that this technologically-equipped field would be contributing greatly to our advances, but I have not seen this yet. I have previously been very critical of the limited, and rather skewed, picture that software paints of genealogy. With a few notable exceptions, tools are still focused on family trees, conclusions, and form-fill data entry. Part of the problem is that such tools are produced by companies governed by commercial forces, including commercial constraints, and if not commercial then at least proprietary. In other words, genealogy has no software team of its own, and is often subjected to software written by non-genealogists. This contrasts sharply with written material such as books and online publications.

RootsTech

This year, I was lucky enough to attend RootsTech 2016 — my first, but hopefully not my last — and found it to be an incredible experience. Meeting all those people that I only knew by name or profile-picture would have been enough on its own, but the overwhelming message of this year’s RootsTech was of particular importance to me: stories and memories. I have long bemoaned the industry’s focus on lineage at the expense of stories and other forms of narrative, and this limitation was a major force in me developing my own software. As Steve Rockwood, the new CEO of FamilySearch, said in his Wednesday Innovator Summit keynote, it’s less about “facts of the chart” and more about “stories of the heart”. His emphasis was primarily on preserving stories for future generations but this new focus still gets my vote.

A particularly moving presentation that demonstrated the power of stories was the Friday keynote by David Isay, founder of Storycorps. The premise is for two close people to record a 40-minute question-answer session between them as though it was their last chance to say something important. David presented a number of these recordings in the keynote and they were so moving that I found myself discretely reaching for my pack of tissues, only to realise that from the folks around me there was an inordinate amount of gentle coughing, nose-blowing, and dust-in-the-eye maintenance. The amazing thing about these stories is that they told themselves; they didn’t need any hype or advertising, and that’s just how stories should be. Whether we’re telling personal stories or recounting historical ones that we may have researched then that potential to tug on the heart strings will always be there. David was obviously aware of this as his introduction was a mere 3.5 minutes before the first story. Perfect!

I strongly believe that this same emotional connection can be found in descriptions of long-passed events , even if we only know of them through research rather than personal experience. One such case took me unawares while writing a narrative report entitled Like Father, Like Son. This contained a description of a house fire in 1937 Nottingham, England, that destroyed a family and their youngest children. The newspaper reports contained eye-witness accounts of a young girl at the window of the top-most of the three levels, but they had to watch as she was consumed because of the ferocity of the flames. The firemen later recounted their grim discoveries of her body, and of her mother and another daughter on the middle level. Housing, then, was largely back-to-back terraces, and many neighbours were related or worked together — much more so than in modern cities — and everyone knew their neighbours, whether good or bad. I was writing about the out-pouring of grief, and of the thousands of people who lined the streets for the funeral procession, when I reached the part describing the coffin of the youngest girl being carried by her school friends. I was so choked that I really had to take a break. Since publishing that account, I have been contacted by direct descendants of the family who thanked me for solving a mystery because no one in the family wanted to talk about those events when they were young, and eventually there was no one left who had experienced them.

I recently summarised the status of my STEMMA research project in a three-part article on my blog, and the second part presented a view of narrative genealogy that embraced story telling, narrative reports, proof arguments, and transcription (of both old and new material). I believe that this seamless inclusion is necessary for useful genealogy, and for micro-history in general. So how did this view tally with the views and products I encountered at RootsTech?

I talked to one high-profile software developer who was very keen on the concept of semantic mark-up (as discussed in my article), but his working constraints currently made it difficult to justify any associated development. Looking around the booths in the Expo Hall was informative; many of the new vendors openly admitted that they were not genealogists, and the designs for sharing memories appeared to demonstrate more than a little influence from existing social media. Selling the idea of sharing memories by alluding to products such as Instagram is smart, but this is different from designing a product using a similar paradigm. For those memories to become essential parts of our family history, and to make them shareable, searchable, and navigable on the Web, then someone needs to look at the nuts-and-bolts of how the data is be stored, and how it's going to be found by search engines such as Google.

As I write this, findmypast have announced a partnership deal with Twile, a Web site that organises your stories and memories according to a timeline, and it will be interesting to see how they view potential integration. Twile was placed third in the RootsTech Innovator Showdown, but was also the people’s choice. So what are my arguments against simply publishing videos, recordings, and images on some “social history” Web site? Well, not all stories are going to be private; sometimes you want them to be found by the families of other people who were there, whether they’re related to you or not, and irrespective of how they were referenced in your story. Not all stories will relate to one particular event that can be neatly placed on a timeline, or hung off someone’s tree. Essentially, the data model by which the stories should be organised is not trivial, and it requires the bigger picture to be considered.

When asked about their data model, those representatives that were aware of the term talked about how the resources (videos, images, etc) were organised on a Web server. None had organised the information according to real-world relationships, as opposed to operational ones, and none had any integration with other genealogical or family history information, such as lineage, places, events, sources, and so on. To me, given both my background and my personal requirements, this approach is several keys short of a piano, but to recognise the requirement itself is a huge step in the right direction. I just hope that the commercial momentum, and the desire to get there first with a killer product, isn’t at the expense of a more-considered design that gives the end-user greater scope and potential.

Most of the representatives expressed some interest in the use of mark-up, and in any guidance, best practices, or data standards that might emerge. Of particular note were Legacyscribes, Pass it down, and kindex, all of whom wanted to know more. In fact, Legacyscribes already had a basic semantic mark-up scheme in place, and I was thrilled to see it demonstrated.

On the subject of transcription, things were a little quieter. Most listened as I explained how finding information in a recorded story needed more than some simple tags applied externally to the associated digital file, but it was still viewed very much as a service rather than an essential cohesive feature. There was clearly a lack of knowledge about subtleties of transcription, and how it would apply to historical documents as well as to modern ones, and to digital recordings. Only two were entirely dismissive, both of the opinion that it just required someone with a word-processor. Although I didn’t ask at the time, I would wager that none had heard of TEI: Text Encoding Initiative.

Collaboration

To a lesser extent, I asked about the process of collaboration in genealogy. I have also criticised the current concept of a unified family tree at What to Share, and How, and presented ideas for crowd-sourced types of collaboration at Collaboration Without Tears, and for tree-based approaches that embrace alternative conclusions, authorial control, and automatic attribution at What to Share, and How – Part II.

Just before I left for RootsTech, a new development project emerged called Trepo, and this has since grabbed the attention of several software developers. It claims to support alternative views in a unified tree, and although it’s unclear how complex this might appear to an average end-user, it’s great that someone is thinking about this.

One of the booths where I questioned the practicality of a unified tree was the one for Geni. In 2012 Geni was acquired by MyHeritage.com and their World Family Tree contrasts with the user-owned trees of MyHeritage. The representative vehemently denied that there were any conflicts or differences of opinion that could not be resolved by a moderator. To ignore the very real “wars” that exist on such trees, and the losses incurred when someone else takes down more than they should, is rather unprofessional, but to believe that evidence analysis is a mechanical process, and that there will always be one unambiguous winner that can be decided by some third-party moderator is just naïve; the same evidence may support more than one view of the truth. I could see that this conversation was doomed. I wanted to point out that when recounting our first-hand personal experiences, or presenting information given directly to us by a family member, then no one else has a right to change it, and so his suggestion of “giving up control” was ill-founded. The stories and memories would be our recollections, expressed in our words, and this was a fundamental message of the conference. However, I was shouted down so I went and got a coffee instead.

In summary, it is high time that genealogy came down from the trees, and walked upright alongside history, but it will not be today.

---

[1] "Tree of Life", by German biologist and naturalist Ernst Heinrich Philipp August Haeckel (1834–1919). Image in the public domain. Copy obtained from Wikimedia Commons (https://commons.wikimedia.org/wiki/File:Tree_of_life_by_Haeckel.jpg : accessed 11 Feb 2016).