The Power of Annotation

We most of us believe that we know what annotation is. However, the basic concept has been applied to several different fields, for quite different purposes, and in many different ways. A review of the landscape for textual annotation was very useful to me, and I hope that others may find this useful too.

A term that goes hand-in-hand with annotation is mark-up (or “markup” in the US), to the extent that they have become virtually synonymous in certain areas. One of the first things to consider is the origin of the two terms, and how their meanings may have shifted over time.

I wanted to call this article “marking up the wrong tree”, but obscure titles are not always the best policy, no matter how side-splittingly hilarious they may seem to you. [Pull yourself together Tony]

According to the dictionary, to annotate is “to add notes to (a text or diagram) giving explanation or comment”, and an etymology is given of “Late 16th century: from Latin annotat- 'marked', from the verb annotare, from ad- 'to' + nota 'a mark'”.[1] This is probably the first usage that most of us would think of.

Figure 1 – Annotated page of text.[2]

As an aside, the analysis of an annotated document is interesting because it often involves a mixture of primary and secondary information whose layers must be considered individually, although not separately.

The term mark-up originates from the annotation of manuscript (and manual typescript) documents with symbols providing printer’s instructions, including corrections, layout, and typesetting. Similar systems of symbolic annotation are used in the field of textual scholarship, which is a collective term for textual studies that encompass analysis, description, transcription, editing, or annotation of texts. The branch of textual scholarship known as diplomatics (not to be confused with diplomacy) involves the scholarly analysis of documents and texts. In particular, a diplomatic transcription reproduces an historic manuscript as accurately as possible (a diplomatic edition) in typography, and including significant features such as original spelling and punctuation; contractions, suspensions, and other abbreviations; insertions, deletions, and other alterations; obsolete characters such as thorn and eth; superscript and subscript characters, and brevigraphs (e.g. the ampersand); these usually employ a system of mark-up in order to capture their essence in a modern typeface. A semi-diplomatic transcription relaxes the requirement for accuracy, usually for readability or practicality. For instance, some original forms are difficult to reproduce in simple typescript, particularly if the original was already marked up by hand, but more on that later. Mark-up may also be used during peer review of a document, or by an author themselves. One more field that I have to mention is corpus linguistics, or the analysis of language using selections of natural text compiled from transcribed writings or recordings (corpora). This uses annotation for such things as tagging parts of speech (POS tagging), e.g. “corpus_NN1 annotation_NN1 is_VBZ hard_AJO” where the suffixes categorise the words (e.g. noun, adjective).

We’ve seen that annotation may actually be symbolic or textual, and that mark-up often includes text as well as symbols or editorial marks. So what is the difference? In his work on corpus linguistics, Martin Weisser comes to the following conclusion:[3]

While the term markup is sometimes used to indicate the physical act of marking specific parts of a text using specific symbols, and, in contrast, annotation may often refer to the interpretive information added, the two may also be used synonymously.

It would seem that the modern usage of these terms employs annotation for the addition of meta-data (related textual or other information) to the text, and mark-up for the scheme by which such annotation is represented or encoded.

This is born out by the concept of mark-up languages, which are systems for annotating a document that are syntactically distinguishable from the text, and hence more structured than mere symbols or marginal notes. A very important distinction has to be made, therefore, between the following types of mark-up:

1. Handwritten mark-up, as applied to a manuscript or typescript document.
2. Typed mark-up, as applied to a typescript or digital document.
3. Mark-up language, as typically applied to a digital document.

The first two of these are designed to be humanly-readable whereas the third type is designed to be computer-readable and so must involve grammatical rules that allow it to be parsed by software. To illustrate the difference, consider the following corrected sentence:

My favourite colour is blue red.

A representation of this using a simple typed mark-up (type 2, above) might be:

My favourite colour is <blue> ^red^.

whereas a mark-up language (type 3) might encode it as follows:

My favourite colour is <del>blue</del> <ins>red</ins>.

These may appear equivalent from a visual perspective, but consider the consequences if the altered text contained either angle brackets or carets, or if the replacement word required some clarification — a mark-up language would be able to represent these cases unambiguously so that software could process it. Also, since a mark-up language is designed purely to communicate the information to software, it means that the representation of that same information to the end-user is not fixed, and the choices would be dependent upon the capabilities of the display medium and the sophistication of the display software.

This leads us nicely to perhaps the two best-known mark-up languages: HTML (HyperText Markup Language) and XML (Extensible Markup Language). HTML was created with predefined semantics (for creating Web pages) but XML was created as a general-purpose syntax with no predefined semantics. Interestingly, the semantics associated with HTML have been refined since its initial development; the last example, above, shows a modern Semantic HTML form, but an older form might have been:

My favourite colour is <s>blue</s> red.

This is literally encoding the visual representation, as in the first example sentence, above. The modern shift in emphasis is from presentation to content structure, such that the mark-up would now show what was deleted and what was inserted rather than simply that a line was drawn through a word.

So why might someone use XML rather than HTML when transcribing an historical document? Well, despite that shift in emphasis, HTML is not a good tool for transcribing text. Consider a document that has original emphasis, such as underlines added by the author, or which has already been marked up by an editor; this information has to be preserved and yet be distinguishable from anything employed during the transcription process, and these are not just presentational matters — there are semantics associated with the original formatting. With a mark-up language such as XML then you have the flexibility to represent all the different types and levels of information without any conflict or ambiguity. Both TEI (Text Encoding Initiative) and STEMMA employ mark-up languages with support for transcription, and both have XML representations.

Using the terminology from Markup_language, there are several forms of mark-up that are required for micro-history narrative:

• Descriptive: Marking the text in order to capture its structure and content, rather than specific visualisations of it. Ultimate control over explicit physical rendition such as colour, bold, italic, underline, font name, and font size are best left to the tool presenting the text (e.g. HTML+CSS).
• Presentational: This mark-up would be essential for a faithful transcription of something. Although modern systems (such as HTML5) frown on explicit presentational information, it may provide important information necessary for the analysis and correct interpretation of transcribed material. STEMMA’s approach to transcription separates structure and content from presentational or stylistic matters: see Descriptive Mark-up.
• Semantic: Although the aforementioned wikipedia link suggests that this is an alternative name for Descriptive mark-up, the usage here is more distinct. This mark-up provides information about the meaning or interpretation of textual references. It is therefore different from the structure and layout in a purely textual context, and is precisely what is needed to identify entities such as Persons and Places.

Semantic mark-up is especially important for narrative essays and narrative reports stored in a genealogical context. Although both TEI and STEMMA have their own schemes, there is a divergence that will become more important once the genealogical industry acknowledges a narrative requirement: the semantics are not independent of the data model. This may be hard to explain, but simply flagging a name as that of a person or place — irrespective of whether it makes a conclusional identification — is an isolated semantic that is addressed in a roughly similar fashion by the two schemes. However, linking such a reference into a chain of conclusion-evidence-information-source would not make any sense outside of a genealogical data model. In effect, TEI is a very comprehensive text-encoding scheme but it cannot deal with semantics associated with an all-embracing data model.

A familiar form of mark-up that we might encounter in wikis or blogs is a lightweight markup language. These have a simple syntax that can be entered directly by the editing user, as opposed to being generated in response to some graphical operation or option selection. Although still designed to be computer-readable, they are easier for a human to read — and, hence, to write. For instance:

**bold text** __underline text__   //italic text//

When looking at the mechanics of adding mark-up to an electronic document then there are two very different approaches. The most common is inline, or embedded, mark-up, where the mark-up language is interwoven with the text in a manner such that it can still be distinguished. For example:

Here is a link: <a href="http://parallaxview.co/stemma/">STEMMA</a>

The alternative is known as stand-off, or remote, mark-up and involves holding the mark-up in a separate file (or other location) to the underlying text, usually linking them by character coordinates. The concept of stand-off mark-up is attributed to Henry Thompson and David McKelvie in 1997,[4] and the advantages include:

• The ability to mark-up read-only (protected) or very large files.
• The ability to support mark-up from independent editors, held as separate layers, and without them having to form a single code hierarchy.
• The ability to combine disjoint segments into a single annotation.

Others are stated but I’m less convinced of their value. In contrast, the advantages of inline mark-up include:

• Simplicity. One file to maintain or distribute.
• The text and mark-up are edited together, with less chance of them getting out-of-step.

Which is best really depends on the application requirements.

A common example of stand-off mark-up, which isn’t always viewed as such, is CSS (Cascading Style Sheets). It was mentioned above that Semantic HTML favours content structure in place of presentation. This works because modern HTML now goes hand-in-hand with CSS, which can describe the presentational aspects in a separate file. Rather than being linked by character coordinates, they are linked by such things as element type and class, collectively described as selectors, which may explain why CSS is rarely described as stand-off mark-up. In effect, HTML then becomes an inline mark-up describing content that relies on a stand-off mark-up for presentation. The advantages of being able to change the overall presentation style of a Web page in a consistent way, or share the style between multiple pages, should be clear.

I want to round off this review of annotation with a quick mention of the humble word-processor. So familiar and useful is this tool that we give little consideration to how it works, or what goes on inside — oh, how I wish genealogy would catch up there. It allows the end-user to add presentational mark-up (e.g. bold, or a specific font-face) and semantic mark-up (e.g. a hyperlink, or a review comment), but you don’t see the associated mark-up. The associated mark-up language is complicated and so made deliberately invisible to the end-user. The net effect of that is to reinforce the user-interface model and give the impression that the end-user is somehow annotating the visible text directly. This is an important distinction — that a hidden nuts-and-bolts mark-up supports the notion, and the physicality, of annotation in the user interface — and it should be an important consideration for future genealogy tools. There is no excuse for expecting the end-user to edit the raw mark-up rather than using a WYSIWYG (“What You See Is What You Get”) interface.

---

[1] Oxford Dictionaries Online (http://www.oxforddictionaries.com/us/definition/english/annotate : accessed 1 Mar 2016), s.v. “annotate”.

[2] John Keats, “Ode to a Nightingale” (1819); image credit: Ryan Johnson (https://www.flickr.com/photos/kmonojo/4288773728 : accessed 1 Mar 2016); Attribution-ShareAlike 2.0 Generic (CC BY-SA 2.0).

[3] Martin Weisser, Practical Corpus Linguistics: An Introduction to Corpus-Based Language Analysis (John Wiley & Sons, 16 Feb 2016), ch.11.

[4] Henry S. Thompson and David McKelvie, “Hyperlink semantics for standoff markup of read-only documents”, May 1997, technical report, Language Technology Group, HCRC, University of Edinburgh (http://www.ltg.ed.ac.uk/~ht/sgmleu97.html : accessed 2 Mar 2016).

Evolution and Genealogy

In this post, I want to pose some thoughts concerning the evolution of genealogy, as opposed to the genealogy of evolution, and consider how my recent visit to RootsTech 2016 may have tempered my prior views.

When asked how genealogy has progressed in recent decades, many people would cite the increased availability of information through the digitisation of records and other sources. But what about the process of genealogy: the methodology, attitudes, and tools?

Figure 1 - Tree of Life, Ernst Haeckel (1834–1919).[1]

A shining light in genealogy is the Genealogical Proof Standard (GPS). Although controversially using the term “proof” in its name, this is essentially a guide to ensuring thorough research, and of reaching sound conclusions. It is not a method (as sometimes thought), and not even a step-wise recipe — hence, its five parts being described as “elements” rather than “steps”. The earliest mention of the GPS by name was by Helen F. M. Leary in "Evidence Revisited — DNA, POE and GPS," OnBoard 4 (January 1998): pp.1–2, 5, and the text of that article may be found online at http://www.bcgcertification.org/skillbuilders/learyevidence.html. Although the actual wording of this standard has undergone a number of refinements before reaching its current form, the underlying principles can be traced back to at least the 1930s, and probably the late 19^th Century. It is hardly surprising, therefore, that those same elements (with minor wording changes) would apply to any field of research, including those outside of the fields of genealogy and history, and including those that do not use the term “proof” in this manner. The expression of these principles was designed to help all genealogists, not just professionals, or even budding professionals, but also the many newcomers to this increasingly popular pursuit.

Another major contribution that has been embraced by the wider community is Evidence Explained: Citing History Sources from Artifacts to Cyberspace (EE) by Elizabeth Shown Mills. Now in its third edition, this mammoth book is sometimes viewed as a recipe book for source citations, and is unfairly criticised when the one you really want is unlisted. However, the book is really about evidence analysis, and the first couple of chapters guide the reader through the things they need to know — the principles, conventions, and requirements — to be empowered to do their own analysis and cite their own sources. There is a lot to learn, here, but it’s well worth the effort. Although not the first such book, it may be argued that this particular one set the scene for the more recent works designed to help genealogists make a better job of what they love doing.

What about software genealogy? You would think that this technologically-equipped field would be contributing greatly to our advances, but I have not seen this yet. I have previously been very critical of the limited, and rather skewed, picture that software paints of genealogy. With a few notable exceptions, tools are still focused on family trees, conclusions, and form-fill data entry. Part of the problem is that such tools are produced by companies governed by commercial forces, including commercial constraints, and if not commercial then at least proprietary. In other words, genealogy has no software team of its own, and is often subjected to software written by non-genealogists. This contrasts sharply with written material such as books and online publications.

RootsTech

This year, I was lucky enough to attend RootsTech 2016 — my first, but hopefully not my last — and found it to be an incredible experience. Meeting all those people that I only knew by name or profile-picture would have been enough on its own, but the overwhelming message of this year’s RootsTech was of particular importance to me: stories and memories. I have long bemoaned the industry’s focus on lineage at the expense of stories and other forms of narrative, and this limitation was a major force in me developing my own software. As Steve Rockwood, the new CEO of FamilySearch, said in his Wednesday Innovator Summit keynote, it’s less about “facts of the chart” and more about “stories of the heart”. His emphasis was primarily on preserving stories for future generations but this new focus still gets my vote.

A particularly moving presentation that demonstrated the power of stories was the Friday keynote by David Isay, founder of Storycorps. The premise is for two close people to record a 40-minute question-answer session between them as though it was their last chance to say something important. David presented a number of these recordings in the keynote and they were so moving that I found myself discretely reaching for my pack of tissues, only to realise that from the folks around me there was an inordinate amount of gentle coughing, nose-blowing, and dust-in-the-eye maintenance. The amazing thing about these stories is that they told themselves; they didn’t need any hype or advertising, and that’s just how stories should be. Whether we’re telling personal stories or recounting historical ones that we may have researched then that potential to tug on the heart strings will always be there. David was obviously aware of this as his introduction was a mere 3.5 minutes before the first story. Perfect!

I strongly believe that this same emotional connection can be found in descriptions of long-passed events , even if we only know of them through research rather than personal experience. One such case took me unawares while writing a narrative report entitled Like Father, Like Son. This contained a description of a house fire in 1937 Nottingham, England, that destroyed a family and their youngest children. The newspaper reports contained eye-witness accounts of a young girl at the window of the top-most of the three levels, but they had to watch as she was consumed because of the ferocity of the flames. The firemen later recounted their grim discoveries of her body, and of her mother and another daughter on the middle level. Housing, then, was largely back-to-back terraces, and many neighbours were related or worked together — much more so than in modern cities — and everyone knew their neighbours, whether good or bad. I was writing about the out-pouring of grief, and of the thousands of people who lined the streets for the funeral procession, when I reached the part describing the coffin of the youngest girl being carried by her school friends. I was so choked that I really had to take a break. Since publishing that account, I have been contacted by direct descendants of the family who thanked me for solving a mystery because no one in the family wanted to talk about those events when they were young, and eventually there was no one left who had experienced them.

I recently summarised the status of my STEMMA research project in a three-part article on my blog, and the second part presented a view of narrative genealogy that embraced story telling, narrative reports, proof arguments, and transcription (of both old and new material). I believe that this seamless inclusion is necessary for useful genealogy, and for micro-history in general. So how did this view tally with the views and products I encountered at RootsTech?

I talked to one high-profile software developer who was very keen on the concept of semantic mark-up (as discussed in my article), but his working constraints currently made it difficult to justify any associated development. Looking around the booths in the Expo Hall was informative; many of the new vendors openly admitted that they were not genealogists, and the designs for sharing memories appeared to demonstrate more than a little influence from existing social media. Selling the idea of sharing memories by alluding to products such as Instagram is smart, but this is different from designing a product using a similar paradigm. For those memories to become essential parts of our family history, and to make them shareable, searchable, and navigable on the Web, then someone needs to look at the nuts-and-bolts of how the data is be stored, and how it's going to be found by search engines such as Google.

As I write this, findmypast have announced a partnership deal with Twile, a Web site that organises your stories and memories according to a timeline, and it will be interesting to see how they view potential integration. Twile was placed third in the RootsTech Innovator Showdown, but was also the people’s choice. So what are my arguments against simply publishing videos, recordings, and images on some “social history” Web site? Well, not all stories are going to be private; sometimes you want them to be found by the families of other people who were there, whether they’re related to you or not, and irrespective of how they were referenced in your story. Not all stories will relate to one particular event that can be neatly placed on a timeline, or hung off someone’s tree. Essentially, the data model by which the stories should be organised is not trivial, and it requires the bigger picture to be considered.

When asked about their data model, those representatives that were aware of the term talked about how the resources (videos, images, etc) were organised on a Web server. None had organised the information according to real-world relationships, as opposed to operational ones, and none had any integration with other genealogical or family history information, such as lineage, places, events, sources, and so on. To me, given both my background and my personal requirements, this approach is several keys short of a piano, but to recognise the requirement itself is a huge step in the right direction. I just hope that the commercial momentum, and the desire to get there first with a killer product, isn’t at the expense of a more-considered design that gives the end-user greater scope and potential.

Most of the representatives expressed some interest in the use of mark-up, and in any guidance, best practices, or data standards that might emerge. Of particular note were Legacyscribes, Pass it down, and kindex, all of whom wanted to know more. In fact, Legacyscribes already had a basic semantic mark-up scheme in place, and I was thrilled to see it demonstrated.

On the subject of transcription, things were a little quieter. Most listened as I explained how finding information in a recorded story needed more than some simple tags applied externally to the associated digital file, but it was still viewed very much as a service rather than an essential cohesive feature. There was clearly a lack of knowledge about subtleties of transcription, and how it would apply to historical documents as well as to modern ones, and to digital recordings. Only two were entirely dismissive, both of the opinion that it just required someone with a word-processor. Although I didn’t ask at the time, I would wager that none had heard of TEI: Text Encoding Initiative.

Collaboration

To a lesser extent, I asked about the process of collaboration in genealogy. I have also criticised the current concept of a unified family tree at What to Share, and How, and presented ideas for crowd-sourced types of collaboration at Collaboration Without Tears, and for tree-based approaches that embrace alternative conclusions, authorial control, and automatic attribution at What to Share, and How – Part II.

Just before I left for RootsTech, a new development project emerged called Trepo, and this has since grabbed the attention of several software developers. It claims to support alternative views in a unified tree, and although it’s unclear how complex this might appear to an average end-user, it’s great that someone is thinking about this.

One of the booths where I questioned the practicality of a unified tree was the one for Geni. In 2012 Geni was acquired by MyHeritage.com and their World Family Tree contrasts with the user-owned trees of MyHeritage. The representative vehemently denied that there were any conflicts or differences of opinion that could not be resolved by a moderator. To ignore the very real “wars” that exist on such trees, and the losses incurred when someone else takes down more than they should, is rather unprofessional, but to believe that evidence analysis is a mechanical process, and that there will always be one unambiguous winner that can be decided by some third-party moderator is just naïve; the same evidence may support more than one view of the truth. I could see that this conversation was doomed. I wanted to point out that when recounting our first-hand personal experiences, or presenting information given directly to us by a family member, then no one else has a right to change it, and so his suggestion of “giving up control” was ill-founded. The stories and memories would be our recollections, expressed in our words, and this was a fundamental message of the conference. However, I was shouted down so I went and got a coffee instead.

In summary, it is high time that genealogy came down from the trees, and walked upright alongside history, but it will not be today.

---

[1] "Tree of Life", by German biologist and naturalist Ernst Heinrich Philipp August Haeckel (1834–1919). Image in the public domain. Copy obtained from Wikimedia Commons (https://commons.wikimedia.org/wiki/File:Tree_of_life_by_Haeckel.jpg : accessed 11 Feb 2016).

Anatomy of a Source

What is a source? When are sources independent and when are they not? How do citations describe related sources? These questions may seem to have obvious answers, but a detailed analysis of the relationships is essential when working electronically with sources.

Traditional genealogical work may take these relationships for granted, but software considerations are too often relegated to a mere reference-note citation added at the end of some narrative, or an electronic bookmark added at some point in a tree. By ‘working electronically with sources’, I mean source-based genealogy using a computer, and so the organisation of entities related to both sources and citations is then a fundamental consideration. The use of computer software as a tool during research, rather than simply for the maintenance of some database afterwards, is still not the norm, but it will be — one day — and not before time.

Figure 1 - Anatomy of a Source.

Background Knowledge and Observations

Let’s begin with some basic knowledge to set the scene. I will frequently refer to terms and concepts from the works of Elizabeth Shown Mills, and also Thomas W. Jones, which I hope will be obvious enough that I don’t have to cite every single occurrence.

Expressed as succinctly as I can, information is semantic data (data with meaning), and a source is the someone, something, or somewhere from which the information was obtained. Citations are statements that identify such sources, and these are most commonly recognised as the sentences used in footnotes/endnotes. In truth, a footnote/endnote may contain multiple citation sentences, each referencing a distinct source, but we’ll ignore that here for simplicity. For the curious, an example may be found at Cite Seeing.

A citation has a number of purposes: intellectual honesty (not claiming prior work as your own), allowing your sources to be independently assessed by the reader, and allowing the strength of your information sources to be assessed. To this end, there are a number of core principles to good citations: identification, description, and evaluation. The identification of the source is what existing software tends to focus on most, and it amounts to naming it and citing its location. If we include the location of the specific information within the source then these details are conveniently summarised as the five Ws: who, what, when, where-is, and where-in.

One of the most important mechanisms in a citation sentence is layers: the segments separated by semicolons that are used to describe the provenance of the source information, or of the source itself, and to provide analytical notes. When only one copy of a source exists, or when it is very rare with only a few copies existing, then citing the repository is the correct thing to do — this contrasts with widely published materials such as books and newspapers. When a source identifies the source of its own information then it is usually termed the source-of-the-source, and the corresponding layer would indicate this with a preceding “citing” or “the author cites”. However, provenance may have been determined by independent means, or there may be a mixture as in this example:

“Literary Miscellanea: Sketch of a Railway ‘Navvy’ ”, book extract, Bath Chronicle and Weekly Gazette (23 June 1859): p.6; citing: [Samuel Smiles,] The Story of the Life of George Stephenson [, Railway Engineer (London: John Murray, 1859)]; abridged by the author from the original and larger work: The Life of George Stephenson (1857).

A point to note is that a layer isn’t just another source; it could also be a repository, it could be provenance details, or it could be analytical notes. This means that layers cannot be modelled by simply linking software entities that describe sources.

In one of the papers submitted to FHISO[1], a set of desirable properties were presented for citations. One of these (section 2.10) is described as Canonical, and it suggested that citations should be one-to-one with sources. Expressed symbolically for two sources: S₁ and S₂, and their respective citations: C(S₁) and C(S₂), this would mean that S₁=S₂ and C(S₁)=C(S₂) would be equivalent statements. However, real-life citations can never be that precise.

A case where this immediately breaks down is when the same citation is expressed in different languages for users from different locales. In other words, even for two absolutely identical sources, with identical provenance and evaluation, there could be several different citations required for the same information. Going further, there may be a need to support different citation styles (e.g. CMOS, MLA), or different variants of a reference note for the first and subsequent usage. These facts mean that any organisation of software entities must support a one-to-many relationship between sources and citations.

Real sources are not all equivalent, even when the associated information has a common origin. Sources may be categorised as original, derivative, or authored works. The latter is effectively a hybrid of original opinions and conclusions, but derivative sources of information.  An original is when the material exists in its first oral or recorded form, and a derivative source is one produced by the copying an original or where the content has been manipulated, such as transcripts, abstracts, extracts, translations, and databases. Image copies are a special sub-category of derivatives that include digital scans and photographic facsimiles. Since they should capture the information exactly as it was then they are often treated the same as originals. However, they are still technically derivatives since the contrast may be lacking, or a film may be scratched, or the resolution too low; even the loss of colour in a monochrome image may have removed essential information. This doesn’t necessarily mean that a copy is always poorer than an original since you could have a damaged original and a copy that was made before the damage occurred — each case has to be evaluated on its own terms.

One of the really difficult areas to deal with — and one I’ve been slowly building up to with this summary of basic knowledge — is where derivatives were formed by manipulation of the content. This is because there are so many ways that they can be formed, and the associated chain of provenance can be difficult to determine. This subject was recently covered by Sue Adams in a series of blog-posts culminating in The Original in Context. For my own example, I’m choosing the baptism of an Amelia Kirk at Nottingham St Mary in 1809. The Nottinghamshire Archive has the original parish registers, and also the bishops’ transcripts: those hand-copied derivatives that were sent to the diocesan centre each month. The archive also has image copies of the parish registers on microfiche. The Nottinghamshire Family History Society (NottsFHS) has a searchable database of transcribed details that may be purchased on CD. Findmypast has incorporated a copy of the NottsFHS transcriptions into their own online databases. Ideally, the information in these derivative forms should be identical to that in the original, but that would be a rare event indeed.

Given that the information should (in principle) be the same, and ideally will not differ too much, how best would copies of alternative derivatives be organised so that they can be worked on (electronically) together? The derivation path may not be a straight line — there could be branches — and so you may have different copies that you want to compare and contrast. Treating them as wholly independent would be both wasteful and inaccurate. In other words, the required organisation must acknowledge the common origin of the information while still allowing the sources to have their own citations, their own evaluations, and their own resources (digital images, paper-based images, textual transcriptions, etc.).

There are two main ways that source references can be related to similar source references: by containment and by derivation. We’ve just looked at the derivation case where the information has a common origin but has been copied or manipulated along the way; containment is the case where you’re looking at a part of a larger source unit. The most common example of this may be when referring to specific pages or chapters in a book, but it can apply to many sources, including separate households or schedules on a given census page. One requirement, here, is that it must be possible for the software to know that two source references are to parts of the same unit (or item, in archival terms), or that one of them is, itself, a reference to that larger unit. For example, that two source references are to pages in the same book, or that one is to the book as a whole.

When working with information from different parts of a source, each distinct where-in reference will have an associated context which must, at least, specify the where and when. For instance, if citing pages from a biography then they may mention the subject when he was in a particular city, and during a particular time frame. In a census page, information for two different households would have been taken at the same time but they would have distinct addresses. This ability to dissect a source, and to characterise references according to their context, is essential if assimilating information for later analysis.

Implementations

I’ll first look at STEMMA’s source-citation relationships. As of V4.0, the Source entity connects to multiple Citation entities and/or multiple Resource entities (e.g. media files). This structure has gradually evolved through trying to model my existing source-based research.

Figure 2 - STEMMA Source entity.

Looking at each of the functional issues mentioned above:

Support for derivatives. The Source entity brings together source information that has a common origin so that it can be compared and contrasted, and generally assimilated in one place. In other words, it is not representing one unique source, but sources with a common information origin. The <Frame> context section embraces citations for all of those corresponding sources.

Support for containment. The SourceLet sections allow the dissection of a source into parts with a related context, and the associated citations would refer to specific where-in parts of the sources already identified outside of the SourceLets. Each SourceLet can also provide where and when contextual details for the associated information. The option to have two tiers of SourceLets — one for related derivatives and one for those specific where-in parts — was not taken for simplicity.

Support for citation language, modes, and styles. The Citation entity supports sets of preformatted citation text strings in alternative languages and citation modes (e.g. first reference note and subsequent ones). There is currently no support for alternative citation styles such as CMOS or MLA. These preformatted strings are all optional but the same Citation entity also supports a mandatory set of citation-elements, such as author, title, and publisher in the case of a book. These may be tagged with semantic types from alternative taxonomies, such as Dublin Core, and inclusively so if desired.

Support for analysis. Terms such as quality, reliability, and credibility may be used to describe a source or some information obtained from it. Analysis of the source information as a whole, including the derivatives with the same origin, is all done within a single Source entity. Although comments and observations on those individual derivatives can still be made, they are not divorced from the context of that shared provenance.

Support for layers. The Citation entity models the layers in a citation using its parent hierarchy (see below). This is possible because its extreme flexibility allows it to model any of the following: a simple citation, a repository, provenance details, and even attribution. The fact that the layers are supported by the Citation entity, and not the by Source entity, is dictated by the observation that layers are not just other sources.

As of V4.0, the citation layers may be characterised according to the terms in the following table. Scheduled for V4.1 is an additional Reworked category.

Layer-type	Comments
AbstractOf	A brief summary or a précis of --
Citing	Information cited by the source. Source-of-the-Source.
Comment	Analytical comments.
ConsultedAs	Consulted through derivative, usually online or in database
ExtractOf	Extracted portion from --
ImagedAt	Consulted through general image copy
MediaCopy	Media conversion from --
Provenance	Other provenance information, differing from ‘Citing’.
Repository	Location of original source.
ReworkOf	Revised, abridged, or otherwise modified from --
TranscriptionOf	Transcribed details from --
TranslationOf	Translated details from --

In GEDCOM, there are Repository, Source Repository Citation, Source Record, and Source Citation records. These are designed to implement a relatively straightforward, but limited, citation→source→repository model with a scope for supporting bibliographic citations — not for source analysis or source-based genealogy.

GEDCOM-X has a wider focus than GEDCOM, although the current draft specification is known to be incomplete in this area at the time of writing.

Figure 3 - GEDCOM-X SourceDescription entity.

Although the SourceDescription relates to a unique source, there are sets of linkages to other sources related by derivation and by containment. It’s probably too early to see how this would work in practice, but the fact that ‘analysis documents’ are tied to individual SourceDescriptions would suggest that some collective analysis of derivatives would be difficult to organise. Also, the treatment of parts of a source (i.e. containment) by distinct SourceDescriptions would seem to compound the issues of collective analysis.

There is no specific support for layers; they cannot be handled by those derivation links since, as I’ve already mentioned, layers are not just other sources. Each SourceDescription may have multiple citations supporting different languages but there is no explicit consideration of alternative citation modes (e.g. first/subsequent), or styles (e.g. MLA). The SourceCitation entity is not hierarchical, and there are currently no citation-elements in either SourceCitation or SourceDescription.

The SourceReference appears to be provided solely to allow the attribution information of a SourceDescription to be overridden.

Relevance to the Reader

While software developers may understand what I’ve written here, I know there will be a significant number of other people thinking ‘I don’t get it. My software already handles sources’. There’s probably a good reason for this and I want to mention it in rounding-off this article.

A scenario that those same people might be able to associate more-closely with would be when adding a marriage date to their tree. They might have found a record with the date and place recorded so they add it to their tree and then include a citation for the record — or an electronic bookmark if found in the online databases of the tree’s host. Where does my “organisation of entities related to both sources and citations” fit into that? Well, in that scenario, it has little relevance. As my second paragraph suggested, most software only thinks about sources and citations as they appear in this limited type of scenario. But if we’re doing source-based genealogy then that organisation is a fundamental requirement.

Tradition genealogists — and good researchers everywhere — would look at more than the one date and place mentioned in such a source; they would look at other information, and the associated context of that information. If anything caught their eye as potentially significant, or requiring further research, then they would note it. That may happen in their heads (yikes!), or be written with pencil & paper, or captured on their computer using a text editor (e.g. Notepad) or some rich-media editor (e.g. Evernote). That note-taking is essentially what I mean by “assimilation” of the source information, and my approach to source-based genealogy is just including the note-taking and the initial analysis into my main genealogy program. There’s no onus on me to draw conclusions, or to attach any of it to a tree; it is a working area where sources can be dissected, and information partially digested so that I can find it and use it later.

I personally find this very natural since my career has hitherto involved developing and using cutting-edge software , but I also appreciate that the majority would not feel as comfortable relying on software to this extent — especially when most of it appears to be form-fill data entry of conclusions, and any basic methodology or representation of real-life scenarios are denied.

** Post updated on 19 Apr 2017 to align with the changes in STEMMA V4.1 **

---

[1] Luther Tychonievich, "Desirable Citation Properties", FHISO Call For Papers, CFPS 112 (http://fhiso.org/files/cfp/cfps112.pdf : accessed 15 Dec 2015); this paper was not listed on the main 'papers received' page (http://tech.fhiso.org/cfps/papers) but was referenced from other papers.