Archive for July, 2012

GSoC Baseline: Recap of Activity Journal – Current Status (0.8.0 Release), Part 2

Sunday, July 22nd, 2012

In what follows, I’ll talk a bit more about how the Activity Journal displays extended periods of your past computer life in terms of your activities, and thereby gives you access to the entities you directed these activities at.

Subsequently, activity browsing by time is contrasted to other means for relating specifically to information you’ve used in the past. These include file browsing by folders and by recently visited folders, access via recent documents lists, as well as file search with date or date range constraints (over ‘creation’, ‘last accessed’, and ‘last modified’ timestamps, as portrayed in 1, 2, 3, 4, 5), and, finally, searching with the Synapse launcher (1, 2, 3, 4, 5) or using the also Zeitgeist-powered Ubuntu Unity file browser (1, 2, 3) and Unity’s scopes and lenses. Please note, that the discussion of most-used access notions and of browsing and search in application-specific histories will be postponed.

As a final introductory remark, it seems overall way less common to have results of users’ re-finding attempts ordered by time, than what could be expected from research findings, presented by one of the thought leaders in the field of information retrieval: In experiments with the ‘Stuff I’ve Seen’ system, it was found that date was by far the most preferred sort order of users, searching and browsing the stuff they’d seen before (slide 7).

With this outline and comments in mind, let’s now resume the recap of the Activity Journal.

Display & Access … of your day-by-day computer work

Just as the prototypes of the new Journal (depicted here and there), their ancestor Activity Journal shows your day-by-day computer work, thus, what you worked on, when you did so and, to some extent, also together with whom.

The Activity Journal can not only be used to display and browse a chronological record of the information you’ve considered in your computer work, but also to regain this very information, that is, first of all, your files. Surpassing display and access of file activity only, other relevant activities such as commits to code hosting platforms, chat conversations in channels and in in private, as well as web activity are also exposed and can be returned to.

To this end, the time-anchored data from your Zeitgeist activity log are presented to you via three separate views, which can be switched among. These views are the default three-days-in-columns view (called ‘multi view’) and the two single-day thumbs and timeline views. For illustration, see this video (multi view: 0:00 – 0:29, thumbs view: 0:30 – 0:52, timeline view: 0:53 – 1:01). Each of the views emphasizes different aspects of your prior activity.

Multi View and Thumbs View

The multi view shows three days in a row (e.g., ‘today’ and the two previous days), and so gives you simultaneous access to structured lists of what you then did (e.g., chatted with a colleague) and touched (e.g., documents and diagrams).

While the multi view also provides file preview on mouseover, the thumbs view brings you per-day overviews, particularly useful for file contents which are easy to recognize graphically.

In order to have both thumbs view and multi view better scale with the number of files and other information touched per day, these two views group your activities into categories by type, with numbers in brackets indicating the size of these groups.

Timeline View

A visualization of your activities, which focuses on their detailed distribution over time, can be achieved with the timeline view.

Recognition of Workflow. With the timeline view, it is particularly convenient to recognize your workflow, i.e. the progression and course of your activities over a day, overall as well as in detail: Overall processes, such as from ‘planning out a day’ to ‘diving into the planned work’, and onto later ‘sharing or reporting about it’ (as was discussed in our talk at the Gran Canaria Desktop Summit in 2009, slides 15f.) can be just as well recognized as a detailed sequence, like in ‘directing your attention away from a certain document’ to ‘finding and inserting an image’, and ‘back to the document’.

Simultaneous Activities. In addition to visualizing, which activities you performed one after another, in the timeline view, it is well visible, what you’ve done in parallel. This, for instance, shows you, which documents you used in preparing a presentation and makes it immediately apparent, which code files you edited, while chatting with a colleague. Likewise, it becomes straightforward to regain the music, you listened to during a chat with your loved one.

Time Tracking. Finally, the timeline view also holds information on how long you worked with a document or with a set of information over a day. Consequently, beyond tracking your activities with files and the like, the Activity Journal also provides basic support for time tracking.
However, please note, that information about duration of your activities can be imprecise at times. This is due to principal difficulties with the tracking of thereto relevant events. Although, the focus time extension for tracking window focus (geared to produce more reliable duration information) has received some attention in the past (1, 2, 3, 4, 5), there is still some hard road ahead.

In both timeline and thumbs views, zooming can be utilized for smooth transitions between broad visual scanning and more detailed inspection.

Activity Browsing … by time

In contrast to file browsing by folders, activity browsing by time helps you remember and re-find what you’ve done during days in the past, rather than it supports the recollection of what you’ve explicitly stored and linked together, e.g., into folders, in your keeping, organization, and maintenance.

Referring to a broad classification of human memory, a time-wise representation of activities focuses on episodic relationships such as co-occurrence, ‘did this before’ and ‘that after’ (as emerging from your courses of action directed at information and people here and there). On the other hand, hierarchical and interlinked groups, and also tags, emphasize what belongs together topically or in terms of projects (i.e. designating more static, but still developing, semantic relationships).

In other words, activity browsing by time is less based on structures as expressed in manually kept information collections, but instead enabled by temporal structures (such as ‘on that day’, ‘in the morning’, or ‘at 5 p.m.’), which can be understood as labels, your memory often implicitly assigns to you’re doings (i.e. to your activities). These labels become reflected in your activity log and are surfaced (i.e. made available) by the Activity Journal.

Recognition over Recall … of what you’ve done

Re-membering with your prior activities and the involved information is supported via cued recall , which, in many situations, beats free recall in convenience and efficiency: often, it is simply easier to recognize something than it is to remember that thing off the top of your head, or to think up, how to describe or specify that thing, for example, via a query term.

In accordance with this observation, each of the three views of the Activity Journal enables you to recognize previous document and image use, chat conversations led, and the like, as well as other information and media use, which occurred around these example activities, and vice versa.

(Recent Documents) ++

The Activity Journal’s approach to overviewing, browsing, regaining, and resuming activities can be understood as bearing some basic resemblance to employing recent document lists, in that it frees you of having to remember files, their names and their locations. However, it goes beyond the limited time range and the confined number of entry types typically covered by these lists.

Plus, the Activity Journal pictures your return and reuse (i.e. repeated activity directed at a certain piece of information). The timeline view, for instance, shows multiple return and reuse occurrences per day, in one row per file. In multi and thumbs views, on the other hand, reuse within days is not displayed, but each day anew at the time when you last accessed a file.

Finally, the Activity Journal also facilitates text searching through your past activities. This is in a way similar to what Synapse and some of the other means, mentioned in the second and third paragraphs of this post afford, but differs in so far as with the Activity Journal you can see your reuse and activity distribution over time, where results are highlighted within the context of your other displayed activities, which is again known to foster the understanding of search results.

Regaining Previous Information Experience as Anchored in Personal Narratives

Depiction of reuse over time in the Activity Journal specifically points out that the temporal maps of your personal information space displayed by the Activity Journal put you vis-à-vis with a re-narration of your activities rather than just providing menus or lists for accessing before used documents and media. Instead, what is shown, can be thought of as pictures of how your lines of action touched upon and (r)evolved around web pages, images, and other objects, which contain the information you work with.

What’s Next?

In the following posts, first, the conceptual placement of Zeitgeist log data (and thereby also of the thereon-based Activity Journal) between lists of recently used information and detailed version histories will be discussed. Second, I’ll review in more detail issues related to re-finding and will explain re-finding strategies, which are specifically enabled by the Activity Journal. Finally, third, the means for enabling user control (beyond view manipulations as to an otherwise sometimes considered ‘not manipulable’ log), implemented in the past two releases of the Activity Journal will receive some attention. See you soon!

GSoC Baseline: Recap of Activity Journal – Current Status (0.8.0 Release), Part 1

Wednesday, July 18th, 2012


This post is part one of a recap of the Activity Journal in its current status. A quick impression of the 0.8.0 release of the application can be gained from three of Stefano’s screencasts (1, 2, 3).

Looking Backward … and some more

In that incarnation, the Activity Journal is about looking backward in time. It’s a viewport into your past, which provides means for navigation, search, and view filtering and, in addition, gives you some preventive control over what goes into your Zeitgeist activity log , allows you to selectively clean it, as well as lets you annotate and pin activities.

Automatic Experiential Maps of Personal Information Space

The Activity Journal focuses on mapping out over time your past activity/experience with information (as, e.g., captured in web pages and your files) and with people (as, e.g., via instant messaging). It does so by presenting automatically generated, time-based views onto your personal information space, which are updated in near real-time.

This, on the one hand, reflects the idea that your personal information space is implicitly and gradually assembled by what you’ve experienced and interacted with (in analogy to your brain, keeping memories about what you’ve seen and done).

Time as Interaction Metaphor

On the other hand, the approach of automatically drawing personal time maps echoes the observation that time, though amongst the most natural reference figures and orientation aids to humans, is a currently underleveraged resource for improving user experience with personal technology. This seems contradictory to the fact that, likewise space, time is part of an unavoidable systematic framework for organizing our experiences.

Therefore, the Activity Journal employs time as a powerful unifying frame of reference for mapping out footprints/traces of your ‘digital’ activities.

This follows the line of argument that while computer systems have contributed to information fragmentation, they can also be used to put the pieces together again: Fueled by the per-device activity log of your net-/notebook or desktop as maintained by Zeitgeist, the Activity Journal displays a rather comprehensive account of your activities (across places/folders/collections and web, and across applications/types and other domain notions), by merging all of the corresponding footprints into temporally structured pictures of your doings.

As a side note, the feasibility of cross-device, mobile, and cloud coverage has been successfully prototyped, but that’s beyond the focus of this recap.

Overview of Targeted User Tasks

The Activity Journal enables the utilization of temporal navigation strategies, such as traveling backwards in time by cursoring over the line-up of your activity footprints as well as the navigation by temporal landmarks.

The deployed temporal navigation strategies not only allow you to re-find stuff in your personal information space by employing your remembrances of when (i.e. at what day or within some other time frame) you used, saw, or listened to that stuff, but the presented activity footprints also tell you about what else you then did, at the same time, before and after.

Consequently, the other way round, you can appropriate any of the displayed activity footprints as temporal landmarks, which allow you to narrow in on a looked-for activity: particularly, you might remember having performed a target activity simultaneously, before, or after a just encountered landmark activity.

So, with the Activity Journal, you can look for both your stuff and your previous activities. The latter effectively helps you remember (i.e. reminds you) about what you’ve done and experienced in the past, and thereby leads to a better overall awareness of your activities as well as of their distribution over time. This again contributes to an increased feeling of control as to reporting tasks and offers the opportunity to review and reflect upon your overall workflow. Finally, some basic time tracking support is given.

Referring to situations in your past (first and foremost defined by their placement and extent in time), still have another application domain, namely task switching. Navigating back to an anchored time frame, which shows you (embedded in an overall, structured-by-the-day activity picture), what you did before an interruption occurred, is particularly helpful with regard to reinstatement of complex information tasks: the context and associated resources, … all you need to resume the task is just there.

Towards Interlinked Complementary Maps of Personal Information Space

Hence, the time-based perspective of the Activity Journal is beneficially complementary to hierarchical and other information and access structures in your personal information space, as exemplified by folders, links, bookmarks and by metadata such as tags.

Beyond complementary co-existence, the Activity Journal also affords some basic interactions with the aforementioned structures, in the sense of ‘upward navigation’ (for gaining the contexts and corresponding navigation/orientation means they make available), and in terms of drag and drop for the creation of information collections from activity footprints.

Between Recent Documents and Versioning Histories

With respect to level of detail and temporal coverage, the activity log data shown by the Activity Journal ranges in between the somewhat imprecise and temporally very gappy activity metadata held by standard file systems (such as ‘created’ and ‘last modified’ timestamps), on the one hand, and the usually very precise and abundant metadata for retracing file development, which is captured (to varying, but often fairly fine-grained temporal granularities) by versioning file systems.

What’s Next?

In the following posts, I’ll first talk a bit more about the views provided by the Activity Journal and about how overviewing and browsing activities by time differs from using some other, currently more widely deployed means for relating back to information you’ve used in the past. Second, the conceptual placement of Zeitgeist log data (and thereby also of the thereon-based Activity Journal) between lists of recently used information and detailed version histories will be discussed. Then, I’ll review in more detail issues related to re-finding and will explain re-finding strategies, which are specifically enabled by the Activity Journal. Finally, fourth, the means for enabling user control (beyond view manipulations as to an otherwise sometimes considered ‘not manipulable’ log), implemented in the past two releases of the Activity Journal will receive some attention. See you soon!

GSoC 2012 – Journal! Revamp!

Sunday, July 8th, 2012

As announced (here) by great hacker and Open Source enthusiast Stefano Candori, I am currently mentoring him in a revamp of the Activity Journal (Stefano is its current maintainer) towards an application simply called Journal. In fact, it’s one of this year’s GSoC projects for the GNOME free software desktop project, and here’s the proposal.

Point of Departure and Motivation

The Activity Journal project was started back in October 2008 (then still under a different name) and since April 2009 has been widely covered and discussed beyond the project’s contributor base in tech news, in interviews, and on various portals and blogs across open source communities. Some examples of the rather extensive echo with a mostly positive twist are 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21 (with corrections in 22), 23, 24, 25, 26, 27, 28, 29, 30 (list sorted by time).

More than 1700 users have downloaded the 0.8.0 release of the Activity Journal from Launchpad (only one way of getting it) and many of these users bless us with a whole bunch of very useful feedback. By the way, some other ways of getting the current release of the Activity Journal are via Ubuntu apps dir, Debian packages, Fedora packages, and RPM).


  • Overall Flexibility of Display. As the number of external application plugins and Zeitgeist dataproviders grows (and thereby also the number and variation of activities to be shown in the Activirty Journal), there is increasing need of flexibility in presentation form. For example, right now, all of the following must be fitted in the same fixed-sized boxes in the default ‘multiview’: document editing, picture use, chat activity, and music listening. And, well, ‘one size fits all’ is at least questionable here.
    As another example, consider calendar items: people use them to structure and orient within their days. So why not using them for re-orienting back to what users did during and around the events they stand for? However, again, calendar items and other such anchor elements for later return don’t really find their place in the Activity Journal’s current visualizations.
  • Presentation of Matches. The density of result presentation as to searching and filtering in the Activity Journal’s entries is limited, because of the predefined columns in multiview.
  • Personal Cues based on Analysis of User Action. Two of the three views currently provided don’t really allow to reflect, how much and in which patterns a user interacted with any of the information shown.
  • View Switching. Then again, switching among the views can be troublesome.

Consequently, in this project, beyond adding new features and types of activity footprints, we’re introducing a new way of showing the Activity Journal to its users.

Current Prototype

Some design considerations regarding the new Journal and a first prototype have recently been presented by Stefano. Below, a screenshot reproduced from there. The code is here.

The new Journal’s main view (early prototype): A two-column line-up of variably-sized Activity Bubbles on a timeline with continuous scrolling and a ‘logarithmic’ Timebar for navigation

The new Journal’s main view (early prototype): A two-column line-up of variably-sized Activity Bubbles on a timeline with continuous scrolling and a ‘logarithmic’ Timebar for navigation

We think that the above presentation form will allow us to clear the just discussed deficiencies of the former Activity Journal visualizations. They are taken up in the same order as they were brought up above:

  • Overall Flexibility of Display. With the two-column line-up of variably-sized activity bubbles, different information formats and user activity footprints can be differently presented.
  • Presentation of Matches. The above illustrated structure of mapping out user activities in time has the potential to be more densely populated with search and filter results than what’s currently possible, while we’re not losing the option to also present matches ‘in context’, so together with what happened around them.
    Still, it seems evident that we’ll need to also revamp something like the visual guidance as provided by the highlighting of days in the histogram slider of the current Activity Journal. This will help users to better orient to search results, which occurred further in the past.
  • Personal Cues based on Analysis of User Action. Intensity, quality, distribution, and patterns of information use, re-use and other user activities can be reflected in more differentiated ways.
  • View Switching. Finally, by heading for a combination of the advantages of the Activity Journal’s thumbs and multiple-days/columns views, the amount of views to be switched among will be reduced.
    However, for now, it remains undecided, how we’ll integrate the Activity Journal’s timeline view and its feature most often positively commented by users: bringing about a basis for time tracking support.

What’s Next?

While the implementation progress of Stefano’s GSoC project will be detailed on his blog, overview of relevant related material will be given on a GNOME wiki page about the revamp project. Here on this blog, I’ll accompany and flank Stefano’s work with related recaps, reflections, and thoughts about where the Journal’s journey might be taken. See you soon!