Obsidian-like behaviour

(Over on Mac Power Users newsgroup there’s a discussion about Obsidian.)

Now we have cross-draft syncing I’d want to take inspiration from Obsidian.

I’m wondering what we can learn from Obsidian.

The one thing I know we don’t have is their graphing. I’m wondering how we could simulate that - or if @agiletortoise thinks that a worthy challenge. :slight_smile:

(I don’t consider graphing essential.)

What is the benefit of the graph? It looks nice and you can see clusters, but an ordered count of back links would give you that. I’m honestly curious as I have tried Obsidian and other apps with such graphs and for me they are always a “that’s nice” feature rather than something I find myself needing.

If you did need a graph, you could structure the connection data between drafts and then use some code within an HTML prompt to do the mapping. I’d hazard a guess that mapping nodes onto a web canvas element probably has a few open source options to make that task a lot easier.

2 Likes

I think you’re right. The graph has visual appeal. If you could click on a node/note/draft to see that item it would be nice. But it’s far from essential - as you’ve shown.

I’m told D3 can do something like this. I’d be inclined to use a GraphViz renderer. (In fact there is one on iOS, at very least.)

So this item near the top of their forum disappointed me. If they can’t get layout right I don’t want to know about them. Yes, layout is hard but it is a key feature. I would’ve hoped they were building on a layout engine that worked, perhaps an open source one.

It would be a bit of a programming challenge, but seems like would be totally possible to build a tree of cross-links in an appropriate format that it could be pushed to MindMode or other mind-mapping software and get a nicer visual mapping.

I guess that would just be hierarchical, not handle multiple cross-references, but still might be interesting.

IThoughts has a nice tree-based CSV file format - which I’m very familiar with. In filterCSV I actually manipulate it. In the README for the project I extend iThoughts’ own documentation for that CSV format.

I haven’t yet experimented with how iThoughts does cross-tree links but I know it probably does - as iThoughts itself supports such a thing. If this looks like a goer I’ll experiment. Right now I’m doing some other things with filterCSV.

IThoughts says it supports Markdown in a tree node. While I’ve used Markdown links and bulleted lists I haven’t tried to exhaust its support yet.

I talk to the developer of iThoughts on a quite frequent basis, in case we need his help or a tweak to iThoughts.

If tapping on a node in iThoughts were to open the corresponding note in Drafts that would accomplish the task.

1 Like

I’d been thinking about this as well! Haven’t brought it up because I appreciate that Drafts != Roam/Obsidian/Logseq, and as @sylumer queried, I do wonder what benefit graphing actually offers.

Exporting hierarchical “trees” of notes to iThoughts is relatively easy to do— I’ve already hacked together a couple of actions based on this notion. You can write markdown links in iThoughts nodes, so it’s again easy to construct tidy links that point back to individual drafts.

I don’t think it’s as easy to graph a visualisation of networked notes rather than branch/tree nodes, however. Cross-links across branches work differently than regular child-parent connections. Even if there’s a way of communicating the necessary information from Drafts (how a note links to other notes, beyond a hierarchy of indents), iThoughts currently doesn’t render those kinds of links particularly well— for example, cross-branch links have to be manually arranged so as not to cross through nodes.

With that, and the fact that the map you create when you move from Drafts to iThoughts isn’t “live”— I’ve ended up not making much use of the actions I’ve already experimented with.

Over the weekend, I went back to playing around with a few Javascript visualisation libraries. Mermaid.js— @galtenberg has already done some work on an action to make it possible to render Mermaid diagrams from simple syntax in Drafts. Nomnoml syntax looks like it would be easy to work with. D3.js is great for visualisation, I just can’t see my way to coding for something for that with my current grasp of javascript.

After thinking further, I arrived at the conclusion that visualising the already hardwired links between documents probably wasn’t all that meaningful for me— what I might actually be thinking about is some way of doing this: https://noduslabs.com/cases/evernote-iphone-notes-text-network-graph/— and that the graphed visualisation is nice to look at, but that what it actually facilitates is foregrounding unexpected connections (rather than the connections that already exist). Still dreaming in this direction…

Another way to think about this: for a while I was using an iThoughts map to serve as a top-level overview for themes/topics/subjects in my notes. Terminal nodes in that map either link to specific drafts or to keywords for a Drafts search action. I think this might be akin to Zettelkasten’s “structure notes“?

1 Like

If everything in iThoughts were Level 0 - which is doable - and we could implement cross links in the CSV format - which might not be doable - we could get all the way there. (And I take your point about Markdown links in the nodes - which I’ve used a lot but never back to Drafts.)

Meanwhile I’m going to see if I can get the GraphViz .dot route to work.

1 Like

I could use a small favour: Given a draft I’d like the javascript to extract the [[ ... ]] links from the draft. This I can then match against the drafts in a user-specifiable workspace.

(I don’t need the brackets; Just an array of link text strings.)

That would get me going.

draft.content.match(/\[\[(.*)\]\]/g).join("\n").replaceAll("[","").replaceAll("]","").split("\n");

The match function produces an array of the link matches. The join, replace alls and split is just a quick and dirty way of removing the square brackets.

Except that was greedy. I added a ? after the .* and that seems to have done the trick. But thank you for getting me much further along with it.

1 Like

So here’s what the result of my prototype looks like:

GV1

The .dot file looks like this:

digraph {
    N0[label="Draft D"]
        N0 -> N2
        N0 -> N3
        N0 -> N1
    N1[label="Draft C"]
        N1 -> N0
    N2[label="Draft A"]
        N2 -> N3
        N2 -> N1
    N3[label="Draft B"]
        N3 -> N2
}

I’ve not solved (and might never solve) the problem of how to make the nodes clickable. But it is at least a nice visualisation.

(My next step will be to try to handle the case where a link is dead. i.e. doesn’t lead to another valid draft in the same workspace.)

A pleasing place to stop for the night. :slight_smile:

1 Like

So I solved the problem of missing drafts:

GV1

In this case Draft Zero doesn’t exist. (I could adjust the colours, of course.)

I’ve also discovered that if you render the GraphViz .dot file to SVG (at least) you can add clickable URLs on nodes. So the next step would be to figure out the URL to any given draft.

When I’ve done that I’ll post the code here.

1 Like

So here is the code, with each draft being accessed by clicking on the node:

// Get the workspace
theWorkspace = Workspace.find("Test")

// Get an array of drafts in the workspace
drafts = theWorkspace.query("inbox")

// Process each draft to get its title and the titles of the drafts it links to
draftTitles = []
draftLinks = []
linkRegex = /\[\[(.*?)\]\]/g
for(i = 0 ; i < drafts.length ; i++){
    theDraft = drafts[i]
    draftTitles[i] = theDraft.title
    draftLinks[i] = theDraft.content.match(linkRegex).join("\n").replaceAll("[","").replaceAll("]","").split("\n")
}

// Compose the .dot output
GraphViz = 'digraph {\n'
missingDrafts = []
for(i = 0 ; i < drafts.length ; i++){
    theDraft = drafts[i]
    draftURL = 'drafts5://open?uuid=' + theDraft.uuid
    // Write the node out
    GraphViz += '    N' + i.toString() + '[label="' + theDraft.title + '" ; URL = "' + draftURL + '"]\n'

    // Write each link out - if found
    for(j = 0; j < draftLinks[i].length; j++){
        otherDraftName = draftLinks[i][j]
        otherDraft = draftTitles.indexOf(otherDraftName)
        if(otherDraft > -1){
            // Have another draft to point at
            GraphViz += '        N' + i.toString() + ' -> N' + otherDraft.toString() +'\n'
        }else{
            // This is a missing draft
            missingDraftIndex = missingDrafts.indexOf(otherDraftName)
            if (missingDraftIndex == -1) {
                // Missing draft not already in the list
                missingDrafts.push(otherDraftName)
                missingDraftIndex = missingDrafts.length - 1
                // Write the node out
                GraphViz += '    M' + missingDraftIndex.toString() + '[fillcolor = red ; style = filled ; label="' + otherDraftName + '"]\n'
            }
        
            GraphViz += '        N' + i.toString() + ' -> M' + missingDraftIndex.toString() +' [color = red]\n'
        }
    }
}
GraphViz += '}\n'
console.log(GraphViz)

There are a couple of issues with it:

  1. I don’t know how to find the current workspace - so the first line needs to be changed to the workspace you’re interested in.
  2. I don’t know how to copy a string to the clipboard - so the resulting string is written to the console.

Help with the above would be appreciated.

For now the instructions for use are:

  1. Change the workspace in the first line.
  2. Run the script.
  3. Scrape the entry in the console log, omitting the last line.
  4. Paste into a file.
  5. Import the file into GraphViz or run dot against it.
  6. Export to SVG.
  7. Open in your browser.

It’s been an interesting prototype to build. If I could solve the UI problems I’d be happier. One extension I might build would be to add nodes for external links - probably in a different colour.

2 Likes

Try:

app.currentWorkspace.name

I use this to determine where in an ordered list of workspaces the user is for TA_loadWorkspaceNext() and TA_loadWorkspacePrevious()` functions.

I’m not sure I follow what you mean by “resulting string”, do you mean something like this?

let strText = "lorem ipsum";
app.setClipboard(strText);
console.log(strText);
1 Like

Ah! Dunno how I missed the app object in the reference. :slight_smile:

Will add those two in and also external links - later today.

Coding practice suggestions also welcomed. I think with @sylumer’s tweaks this is worth pursuing further.

One other note: If you get the digraph into GraphViz on Mac you can click on the links. The .dot view I found in the iOS App Store ignores the links and also doesn’t emit SVG.

So here’s where we are:

GV1

I’ve softened the colouring and added a new one: Green for external links.

You might guess that “Beeb News” and “BBC News” point to the same URL. The question is whether to detect such potentially misformed duplications and colour them yellow. It could be legitimate to have two sets of words for the same external link or it could be unintended inconsistency.

I’ve also - thanks to @sylumer - changed the code so that it processes all the drafts in the current workspace. It also now writes to the clipboard.

Anyhow here’s the code as it currently stands:

// Takes the drafts from the current workspace and makes a
// GraphViz .dot file on the clipboard.

// Get an array of the drafts in the current workspace
drafts = app.currentWorkspace.query("inbox")

// Process each draft to get its title and the titles of the drafts it links to
draftTitles = []
draftInternalLinks = []
draftExternalLinks = []
foundExternalLinks = []

//internalLinkRegex = /\[\[(.*?)\]\]/g
internalLinkRegex = /\[\[.*?\]\]/g

//externalLinkRegex = /\[(.*?)\]\((.*?)\)/g
externalLinkRegex = /\[.*?\]\(.*?\)/g

// Iterate over the drafts in this workspace
for(i = 0 ; i < drafts.length ; i++){
    theDraft = drafts[i]
    draftTitles[i] = theDraft.title

    // Get internal links - if any
    matches = theDraft.content.match(internalLinkRegex)
    if (matches != null){
        draftInternalLinks[i] = matches.join("\n").replaceAll("[","").replaceAll("]","").split("\n")
    }else{
        draftInternalLinks[i] = []
    }

    // Get external links - if any
    matches = theDraft.content.match(externalLinkRegex)
    if (matches != null){
        draftExternalLinks[i] = matches.join("\n").replaceAll("[","").replaceAll("]","").replaceAll("(","!!!XYZZY!!!").replaceAll(")","").split("\n")
    }else{
        draftExternalLinks[i] = []
    }
}

// Compose the .dot output
GraphViz = 'digraph {\n'
missingDrafts = []
for(i = 0 ; i < drafts.length ; i++){
    theDraft = drafts[i]
    draftURL = 'drafts5://open?uuid=' + theDraft.uuid
    // Write the node out
    GraphViz += '    N' + i.toString() + '[label="' + theDraft.title + '" ; URL = "' + draftURL + '"]\n'

    // Write each internal link out - if found
    for(j = 0; j < draftInternalLinks[i].length; j++){
        otherDraftName = draftInternalLinks[i][j]
        otherDraft = draftTitles.indexOf(otherDraftName)
        if(otherDraft > -1){
            // Have another draft to point at
            GraphViz += '        N' + i.toString() + ' -> N' + otherDraft.toString() +'\n'
        }else{
            // This is a missing draft
            missingDraftIndex = missingDrafts.indexOf(otherDraftName)
            if (missingDraftIndex == -1) {
                // Missing draft not already in the list
                missingDrafts.push(otherDraftName)
                missingDraftIndex = missingDrafts.length - 1
                // Write the node out
                GraphViz += '    M' + missingDraftIndex.toString() + '[fillcolor = lightcoral ; style = filled ; label="' + otherDraftName + '"]\n'
            }
        
            GraphViz += '        N' + i.toString() + ' -> M' + missingDraftIndex.toString() +' [color = lightcoral]\n'
        }
    }

    // Write each external link out - if found
    for(j = 0; j < draftExternalLinks[i].length; j++){
        if(foundExternalLinks.indexOf(draftExternalLinks[i][j]) == -1){
            // Register fresh external link name/URL combo
            foundExternalLinks.push(draftExternalLinks[i][j])
            linkNumber = foundExternalLinks.length - 1
        
            // Create node for external name/URL combo
            splitLink = draftExternalLinks[i][j].split("!!!XYZZY!!!")
            otherPageName = splitLink[0]
            otherPageURL = splitLink[1]
            GraphViz += '    E' + linkNumber.toString() + '[label = "' + otherPageName + '"; fillcolor = darkseagreen2; style = filled; URL="' + otherPageURL + '" ]' + '\n'
        }
            GraphViz += '        N' + i.toString() + ' -> E' + linkNumber.toString() +' [color = darkseagreen2]\n'
    }
}
GraphViz += '}\n'
app.setClipboard(GraphViz)

And here’s the GraphViz .dot output:

digraph {
    N0[label="Draft D" ; URL = "drafts5://open?uuid=CAC17EE9-1DBF-42D6-82F7-4D1A827B7D1C"]
        N0 -> N3
        N0 -> N2
        N0 -> N4
    E0[label = "Beeb News"; fillcolor = darkseagreen2; style = filled; URL="https://www.bbc.co.uk/news" ]
        N0 -> E0 [color = darkseagreen2]
    N1[label="Draft E" ; URL = "drafts5://open?uuid=ED8459D8-C0C6-4ECF-9BD0-F0CFABCE7DD0"]
    N2[label="Draft B" ; URL = "drafts5://open?uuid=114B9649-42BC-40DE-AC3B-823180A1867B"]
        N2 -> N3
    E1[label = "IBM"; fillcolor = darkseagreen2; style = filled; URL="https://www.ibm.com" ]
        N2 -> E1 [color = darkseagreen2]
    E2[label = "BBC News"; fillcolor = darkseagreen2; style = filled; URL="https://www.bbc.co.uk/news" ]
        N2 -> E2 [color = darkseagreen2]
    N3[label="Draft A" ; URL = "drafts5://open?uuid=07CE3469-17F0-4114-BB2F-C88329F9C4EA"]
        N3 -> N2
        N3 -> N4
    M0[fillcolor = lightcoral ; style = filled ; label="Draft Zero"]
        N3 -> M0 [color = lightcoral]
    N4[label="Draft C" ; URL = "drafts5://open?uuid=BB986330-D24B-453C-9B75-7ECDC2FB925A"]
        N4 -> N0
}
3 Likes

I would add, in compliment to the graphing, the additional two bits Drafts needs to behave similar to Roam and Obsidian are:

  1. Auto generating backlinks. Yes, [[test]] creates a link to the page… but, would be good to have it link back to the source automatically, ala Roam.
  2. Auto update links. Ie; [[test]] becomes [[test2]] > it needs to update everywhere in Drafts.

I suspect neither will happen. BUT if they did, I would leave Roam in an instant.

1 Like

Your Item 1 could be fairly well approximated if you were to accept an action to do it. The action would insert the reference to the other draft and then insert in the other draft the reference back to this one.

The question would become “where should the back reference in the other draft be placed? Just after the title? At the end? (I don’t know if it’s possible to have a cursor in each draft - as I’ve not tried it.)

Your Item 2 would be tricky but I could see another action that does the rename and then corrects the references - but that one would have to scan the whole Drafts database, unless you limit it to eg the current workspace.

Love your approach.

If you want to get rid of these ellipses use

node [shape="box"];

after opening braces

Putting details in a node I found records very helpfull
They are done with :

digraph structs {
  node [shape=record];
  struct1 [shape=record,label=" left| middle| right"];
  struct2 [shape=record,label=" one| two"];
  struct3 [shape=record,label="hello\nworld |{ b |{c| d|e}| f}| g | h"];
  struct1:f1 -> struct2:f0;
  struct1:f2 -> struct3:here;
}

Imgur

1 Like