Writings On Organizing Information

Call For Contributions
  Have you read what is written here and would like to contribute? Create a Drummer account, create an OPML file. Use the top level nodes for titles, sub-nodes (child nodes) for the paragraphs, and then make the outline public. Let me know the public URL for our outline.
Default Expand/Collapse
  A problem I see here is that a page can quickly become very long as new essays are added, consequently I am included to collapse the top level nodes by default, but what does that do to discovery?
  I've decided to collapse the contributors sections by default. Click the title to expand, you will see what I mean by how long the page can become. I wish pagePark provided a simple way toggle Expand All and Collapse All.
  A consequence of enabling the collapse attribute for nodes and then publishing the OPML using pagePark is that links to child nodes don't work as expected because they are basically buried within the collapsed node. Therefore, when I added a new item to my contributions section below I set collapse to false on my parent node so that the link to the node I just added works as expected. Problem is that this function won't work as I prefer.
  In an ideal world a link to a node on a page would cause the rendered outline to expand to that node automatically, but that requires more sophistication in hosting application than pagePark has. Perhaps a compromise is to not collapse the top level, contributor nodes, collapse child nodes except the one to the most recent contribution.
Frank McPherson's Contributions
  An Index Of Web Sites Worth Remembering
    Due to writing and thinking about indexes of information, I decided to create my own in Drummer. In guess in a way this is a public version of browser bookmarks. The trick here, I think, is to simplify adding sites. Currently this means keeping Drummer open in a browser tab in order to edit the index file. I did create a friendly URL: http://index.frankm.info that redirects to the rendered page.
  Web Sites Worth Remembering, An Index
  Live Documents
    What if there were a RSS feed just for an article/essay like the one I wrote here? Imagine that each paragraph in the second then was a RSS item. I think then you would want the RSS content to persist in a reader but be updated whenever a change was detected in any of the items. I think the result would be a "living" document updated on one's computer whenever the author made a change. Hold on.... it just occurred to me we already have this with OPML, Drummer, and opmlBrowser! Time to pull together a demo.
    Ok, here it is... a separate public OPML file that can be opened via URL in Drummer and viewed publicly here. I've asked Andy to create a public OPML file of his first blog post that I tend to include in this OPML file. I think the rendered result will be the "live" document I described above. So live, that if you this page opened in your browser while I am typing you would be seeing the changes!
    I think there are a couple of downsides to using Drummer and Dave's pagePark installation for hosting and rendering the OPML file. One is that in order for Andy's outline to be included it needs to be expanded in Drummer, as soon as I collapse it from Drummer the content included from his OPML file disappears. Consequently, I suspect that any updates Andy makes might not appear unless i have the file open in Drummer. Not sure if it has to be the active tab before the upgrades take affect.
    I therefore think Andy's implementation of the opmlPackage might be better as it doesn't need Drummer to facilitate the updates. To test that theory I probably need to provision another server. In the mean time, I hope that you see how people could collaborate on a document that represents a body of knowledge on a particular topic.
    I do wish that pagePark provided a simple way to Expand All/Collapse All, perhaps by double clicking the document title to toggle between the two.
  How Could This Work?
    The maintainer makes a call for contributing essays/writings. The contributors "publish" their writings in a public OPML file that the maintainer includes into the "main" essays document. Contributors edit/add as they then see fit, the essays update automatically updates.
    The contributing OPML could include text or it could simply be an outline of link nodes. If you expand my contributions section below you will notice the first item is a link to an index that I have created.
  Ideas Being Generated
    Now that I have put this document together and interacting with it in the browser, I think discovering changes is a challenge. What I think is needed is some indication of whether something has changed. Perhaps an icon next to the updated head? When I first thought of this topic my mind went to RSS because of how it handles updates as there is a way for pings to be sent to aggregators to let them know about a change. I don't think the same type of notification is built in to OPML, though I do know that if you had this outline open in a Drummer tab a lightning bolt would appear once I made this change, which at least notifies you that an outline/tab has changed.
    Drummer does not appear to be the best tool for this idea because updates only happen when Drummer is running and the "master" outline is open and the included outlines are expanded. If a contributor makes changes to their outline they do not appear in the "published" version until the maintainer of the "master" outline opens that outline. This is an instance of making a tool something that it is not intended, in this case the constraint is that Drummer is a "client" app that works well for creating new outlines but what this idea calls for a server app that monitors updates and publishes them as they are saved by the contributor.
    Note that when the "master" outline is closed that the state of the published version is maintained until it is opened again.
    I can imagine, however, cases in which the maintainer of the "master" outline wouldn't want to have changes automatically published, and that they would want to "moderate" changes. In this instance the model that Drummer provides might be better. An "open" publishing model requires an amount of trust.
  The Value Of Indexes
    Andy Sylvester recently wrote a post on organizing information for use based on his experience developing a web app that presents a "list" of outlines in OPML, the OPML files could be stored on other servers. I am going to riff a bit on this topic here. If one were to look back at past tools used to collect and store information and produce knowledge like CommonPlace books and Zettlekasten I see a common denominator that both had an index that provides a structure to the information contained so that it could be found. Recall also that before Google there was Yahoo, which aspired to be the Index for the Web.
    Of course, the Web grew so fast that Yahoo's original manually curated Index did not scale, thus Google search won and Yahoo is now really a distant memory. However, while one does not see it, there is an index to the web that Google automatically produces, it's just that how we use the index is by searching for words or phrases rather than browsing the index.
    Search is fast, but what is lost? I think what is lost is the ability to visually see the relationship between entries in the index. As I understand it the Zettlekasten methodology is basically using the index one builds to discover related items that are not obvious. Recent applications like Roam, Logseq, and Obsidian enable Zettlekasten-like ways to "discover" relationships by making it easier for one to link information with each other.
    I personally use Roam, but if I am honest, I have not "discovered" anything new with it, probably because I don't really "work" within it but instead use it for storage and retrieval. For me the problem with Roam is that while it does outlining it is not an outliner, it is a text editor that does outlining and wiki links. For me, an outline is more akin to an index rather than a document. For example, look at this outline of the technology that I use, it's a list of top level items. If you want to "discover" what tech I use every day, click Every Day Tech to expand the list of tech items I most use. When you click the iPad Mini 6 you see some details and links to more information about the iPad Mini 6 and you know that the iPad Mini 6 is one of the tech items I use every day.
    The problem with this page is that the top level list makes most sense to me, but it might not make sense to you. What if you could create a top level index that made most sense to you? Perhaps you want a list of Apple Products so you make that the top level node of our index, but what if you could then simply include my iPad Mini 6 node into your index? Imagine right clicking the node and selecting "Copy outline node", and then imagine going back to your outline and selecting "Paste outline node" that automatically created an include of that node of the OPML file I manage.
    The use case I describe above requires something deeper than OPML includes, it requires OPML node includes, which I don't think currently exist. The benefit though is that whenever I updated that piece of information in my file you would see that update with no real effort on your part. What if you could atomize other snippets of information (a paragraph or two) that transformed into an outline node and updated live...could this be done by "copying" the RSS behind that snippet, and "pasting" that RSS snippet into an OPML node? (Of course, that requires each paragraph on a page to be a separate RSS item.)
    I would love it if something like this could be made, I think in fact that tools like Hypothesis come close, but coming back to my main point, I think there is value in indexes that one actually sees and interacts with. A few years ago I built a wiki and I found a plugin that automatically creates a page that is an index of all the pages in my wiki in alphabetical order. My wiki site index is like the index you see in the back of some books, like text books. While I can search for pages in my wiki, I often just go to the site index page and quickly scroll down the page to find the title of topic I want to open. The site index page would not exist if it weren't automatically produced because it would take too my effort to manually maintain it. I think Andy's app shows a way to publish an index in outline form that one not only sees as an index, and it's relationship, but can also interact with as an outline. The "magic" would be in the automation and simplification of using something like it for one to make what ever index they prefer.
  The First Internet Index Was DNS
    DNS is so fundamental to how we use the Internet that we might forget what it really is, which is a distributed index of web sites. DNS is effectively the first index of the Internet. Today when one builds a web site the register a URL for it so that others may find the site using DNS. DNS is what takes the URL you enter in the address bar of your browser and translate it to the specific IP address of the server hosting the site. While most think of DNS has how others find your site, you should also think of it as a way to find your own site or pages that you publish on the Internet.
    I registered a domain, frankm.info that I use as a friendly name to forward one to the site that is my "Now" page. Think of a Now page as a status profile page. The actual URL is to an OPML file I edit using Little Outliner. Over time I have added subdomains to point to other pages so that I can easily get to them on the web. To see my tech outline I enter tech.frankm.info, to see a list of the books that I read I enter books.frankm.info, search.frankm.info is go a list of frequent Daytona searches, and index.frankm.info is the "new" index page I created this morning.
    All are simple URLs to remember, the another benefit is that if I use them in a blog post but move the actual location of the page to another URL all I need to do is update the associated DNS forwarder records with my DNS provider and the links will continue working.
    Obviously, I can share these with other people, and I do use them when referring to these items in my blog posts, but mostly, I they are for my own use to quickly get to this information when I need to.
  Navigate, Don't Search
    Unbeknownst to me, Linus Lee wrote a couple of essays earlier in the year that are similar to, better thought out, and make a a better point than my points about indexes written here previously. Linus points out that humans are bad at coming up with search terms but good at incrementally narrowing down options toward finding information. Search might be good for the Internet, but in seeking, and thus organizing personal information like notes, he writes:
    "But when designing an interface for finding a note in a small pile of personal notes, or building an app to organize a small team’s working documents, most of the “finding stuff” interface ideas are in play. It’s in these situations where I want to make the argument: prefer interfaces that let the user incrementally move towards the right answer over direct search."
    In another essay titled A GPS For The Mind, Lee writes:
    "When I’m thinking deeply, I’m trying to understanding something new by internalizing an explanation, or I’m trying to find a solution to a problem in front of me. I’m not blindly jumping from idea to idea, hoping for an insight – there are far too many ideas, and many more connections between them, for random walks through idea webs to ever yield anything interesting. So clearly, there’s some directed-ness to the way our minds are navigating our internal webs of ideas – good thinking is effective navigation through the idea maze."
    Lee goes on to provide some good examples to make his point, but I think his point of navigating through to find information and UIs that make that navigation easy are why I find hierarchy like outlines (such as this document) and the story board/breadcrumb like navigation of Federated Wiki intuitive and powerful.
    Taking this concept a step further, in Hyperlink Maximalism Lee introduces us to a experimental notetaking application he developed called Notation. The neat thing about Notation is that it automatically creates hyperlinks to words and phrases within a document in what Lee calls heatmaps that look like highlighted words in the document. As he says in the essay introducing Notation, it is optimized for keyboard navigation but I have found that you can double-click a highlight and see the associated mentions in other documents/notes that Lee has in his personal database.
    Notation demonstrates that which I wish Roam had, which is the computer doing the work of relating notes for me rather than my having to do so via a Zettelkasten-like indexing method. I value indexes and I know that they must be made by computers because as we learned from Yahoo, manual creation of indexes does not scale. Think of what Notation is doing as augmenting hyperlinking native to the web, writers can still make links to direct readers to information, but the computer augments that with layering over it a component of the user's personal database of information.
    "Notation’s interface is one small attempt to approach and improve thinking as a navigation problem. Highlights and heatmaps drawn by Notation from its understanding of language help us find interesting connections between ideas where we may have remembered none by ourselves, and it turns ideas written down in text form into a kind of a literal “map”. When combined with powerful new techniques in NLP, I think interfaces like this can turn computers into powerful creative collaborators in our attempts to understand the world more deeply." source
  Knowledge Tools, Finite and Infinite
    In his essay titled Knowledge Tools, Finite and Infinite, Linus Lee states that note taking tools contain a finite amount of information whereas Google appears to relate to an infinite amount of information. Lee writes:
    "Your average note-taking app is quite finite. If you have a thousand notes, you have a thousand notes. The tool will give you the means to search through and organize them effectively, but there is an end to the pool of knowledge the tool contains. Way on the other end of the spectrum is Google, which like the world’s largest and fastest-growing library feels infinite and bottomless. There’s no sense in which Google “contains” all the information and knowledge it can let you access; it’s merely an interface through which you reach into a seemingly infinite expanse of knowledge, growing faster than you can comprehend."
    Lee then goes on to provide some examples for ways in which note taking tools could become more infinite:
    "The easiest way to open up our notes to infinites would be to let our notes search online to grow autonomously. By this, I mean: what if our notes searched the web for ideas that agreed with us or critiqued us, while we weren’t tending to them? This would transform note-taking into a truly valuable investment – put in a few ideas, discover many more!"
    I see this as another example of how a note taking app that might use could help my expand my thinking by automatically surfacing for me related items. Thinking about this idea draws me toward Readwise, which is where most of my highlights go. What if Readwise scanned my highlights and then overlayed links to related items I haven't read. Evernote could do this with notebooks that I've told it to do it with.
  Render Conference, August 16, 2022
    On Friday I stumbled upon Jess Martin's summary of the Render Conference, a confab of 75 technologists working on tools for thought or adjacent experiments. The summary includes several embedded videos from presentations given at the conference, though unfortunately the audio quality is not the best. Jerry Michalski is one of the co-hosts who has been using The Brain for 25 years and makes a version available at JerrysBrain.com. The Brain is a mind mapping tool that pre-exists all of the modern "tools for thought" applications, and at one time I owned a copy that I used but I did not keep up paying for the version upgrades. For some reason I prefer the upon/down hierarchy of outlines rather than the radient, multi-direction 0f mind maps. One thing that recall liking about The Brain is that you can use it to link to files on your computer, so it works well to link to other documents on your computer.
    Readwise, which I use to store highlights from Pocket and Kindle books in Roam and Evernote, was in attendance and gave demonstration of the reading application they are developing. I am keeping an eye on this because it has the potential of being more useful to me than Pocket if it enables me to high and annotate web articles.
    Martin notes in his summary of the conference that interoperability was a big theme. Included in the conference was a explanation of a protocol called Noosphere that appears intended to allow different tools for thought products to share information with each other. A point made during the presentation of Noosphere is that the Internet itself is a tool for thought. I watch some of the video on Noosphere but it was over my head, but if the gist is providing an API for exchanging information then I think the developer is on the right track. Earlier in the year Dave Winer was pushing OPML as the "protocol" that could be that glue between tools of thought, but OPML is file format and the current iterations of these tools aren't really working with files, but rather non-structured databases. Moving files around is tedious in comparison to exchanging data via APIs.
    I think there is more to mine out of the videos from the conference that I need more time to digest. Linus Lee was in attendance and he participated in a session about AI and ML in building tools for thought, which I think gets us closer to what the ultimate goal of making this software easier to use for personal augmentation.
Andy Sylvester's Contributions
  More thoughts on organizing information for use
    I recently created a web app to use the opmlPackage NPM module from Dave Winer to display the contents of four OPML files at one time. The app uses the includes feature, which can read other OPML files from a single OPML file and save the content locally for processing. I created four OPML files myself for this test app, but the four files could have been created by anyone.
    So the main benefit of the OPML Includes site is that it can display multiple outlines (which can be edited by multiple people, not just one person) and have the content refreshed whenever someone goes to the site. If the content is changing on a frequent basis, this could be an easy way to see the updates. If the content is fairly static, it may not be significant. In that case, single page apps could be used to view the outlines separately, so perhaps there is not much benefit in using the includes feature in opmlPackage.
    Ken Smith has had some thoughts about potential uses:
      "And I'm still musing about uses. About the slide down into the archive problem of blogging and social media, and maybe using the app as a partial remedy. Keeping the good stuff in view, and adding to the good stuff over time and linking the good stuff to a wider circle of relevant content. Taking the web part of the web seriously rather than letting the slide down into the archive turn it all into ghostly memories."
    Again, an OPML outline, rendered using existing tools, could provide a way to collect that information in a single document. In 2010, Jay Rosen’s Studio 20 journalism program at New York University partnered with ProPublica to research the area of “explainers”, or explanatory journalism at a site called explainer.net. The site is still available on the Internet Archive. I think that this is a method of addressing the “slide down into the archive” problem, but it does take effort. Someone has to create such an explainer, and then monitor the Web for items related to that topic, read and curate them, and add them to the explainer as needed. The main problem is someone has to have the desire and interest to collect information on a topic and share that information. If there is no one with that interest, no tool for collecting and displaying that information is going to be of much use.
  Resources for organizing information
    I have compiled some links on how to better organize information:
    Lifehack.org - Use the LATCH principle (Location, Alphabet, Time, Category, or Hierarchy), Mind Mapping, Create Lists, Create Collections, Place Priority on Key Information
    The Visual Communication Guy - Gives examples of using the LATCH principle listed above
    UXPin - How to Organize Information Effectively: What You Can Learn From Information Architecture
      Create systems for classification, labeling, navigation and search
      Information architecture is affected by content, context, and users
      Article continues by providing more specific examples
    ASBMB.org - How to gather and organize information
      Describes a method to assist someone performing research to write an academic paper
  Resources for writing
    In organizing information for use, writing skills can be particularly helpful, especially in trying to summarize topics. Here are some resources I recently came across:
    How To Make Notes and Write by Dan Allosso and S. F. Allosso - Learn to make effective notes on sources and your interpretations, then turn those into clear and compelling output. This link is to a website version of the book, an ePub version can also be downloaded
    The Documentation System from Divio - A theory of documentation composed of four types (tutorials, how-to guides, technical reference and explanation) with excellent explanation of the differences and examples on how to create these four types of documentation effectively.
  Adding value to information
    Ken Smith read my post commenting on his "Why write?" post, and focused on a link talking about adding value to information. Ken tweeted about it, and I would like to explore this more.
    I follow multiple RSS feeds using River5 and specialized lists by category (my main list has 155 feeds). I view these feeds through a standalone river-of-news app that displays the feeds and has its own URL. The newest items are at the top of the page. River5 truncates the items to be less than 500 characters (I think). When I see an article I want to read fully, I right-click on the link to open a new browser tab. I usually scroll through the news app page until I get to the last place I read to, then start reading the tabs I have opened.
    My process has mostly been to be a "collector" (see this excellent article on the "Collector's Fallacy", I have this disease), with multiple link dump documents (mostly not organized, one or two documents organized on a single topic.). I spent a little time exploring how to organize these links, but have made little progress. From the link I mentioned at the start of this post, I am past "reading" and I am at the "collecting quotes" stage (if I change "quotes" to "links", I think that is a better description what I am doing).
    The next step up the value chain is "creating excerpts", or a summary of a text (think "blog post" or "story"). Sometimes I add a sentence summarizing what a link is about, most of the time not. This would be a new area of refinement for me, and I am considering it. The zettle author Christian has a great post on how to process the reading from RSS feeds, and how to create a "knowledge cycle" (think "workflow") to manage the moving of the information read from RSS feeds into a zettlekasten or other knowledge system. I am going to spend some time unpacking these posts this week and see if I can improve my information processing workflow.
    One last note on the notes in a zettlekasten system. There has been much discussion about "atomic notes", which represents the key ideas from a person's research on some topic or source (sources one and two). These are not the kind of thing I am interested in creating/collecting, or at least not what I have been doing. A far more typical thing for me is something I did at work today. I was trying to figure out how to convert the output of a program into another format. I did some searching, installed a tool, found a script, played with the script in the tool, figured out how to use it, then wrote down a summary of my steps and added links to what I found in my search. Since I am not doing research for a book or for writing academic papers, the idea of an atomic note does not fit into my information world. However, capturing the steps of a discovery or how I worked out a problem. is very real and concrete to me. I used to know a fellow engineer who wrote "technical notes" to capture work he was doing (like a journal entry). Maybe that is how I should consider this type of knowledge creation.
    I am adding this to Frank McPherson's collection on organizing information.