Bulk import of notes, preserving date / times

gcaprio · August 23, 2020, 1:01pm

Hey everyone,

I’m migrating my notes into Drafts and wanted to know if there’s a way to bulk import .md files while also preserving the date/time of the file? Every time I import notes, they get lumped to the top of my list and set to today’s date/time.

Andreas_Haberle · August 23, 2020, 5:39pm

Hi,

as I understand Drafts creationDate is a fixed parameter. As far as I know there is no obvious way to change it.

https://scripting.getdrafts.com/classes/draft#createdat

sylumer · August 23, 2020, 5:54pm

As Andreas notes, the creation property is read only for scripting, and as you’ve noted, the standard non-scripting import processes create “now” and so the timestamps reflect that.

There is a way to do it, but it needs a bit of work to do:

Now, it would be possible on the Mac to build an automation to convert a file into a Drafts json file ready for import. Pretty much any scripting language is going to let you read file meta data (creation date, etc.) and write it out to a text file with the content and some standard text to create the JSON. The ease of doing so just depends on what tools or scripting languages you are comfortable working with, and how well suited they are to this sort of thing.

Hope that helps.

Andreas_Haberle · August 23, 2020, 9:16pm

great note.
should be easily done by an experienced programmer.
I might wrap my head around it during a session the upcoming week.

gcaprio · August 26, 2020, 12:45pm

awesome thanks everyone. I’ve got got a lot of programming experience, so should be easy. one question: where can I export as JSON from Drafts so I can see what the format looks like? I can only see export as plain text as an option in the latest app.

agiletortoise · August 26, 2020, 12:55pm

Select drafts in the draft list…

On Mac, right-click on list > Export > JSON.
On iOS, use “Operations” > Export at bottom of list.

gcaprio · September 6, 2020, 4:07am

thanks! I’ve actually got a small python app that will create my JSON from a directory of .md files. However, when I try and import a test note, I get the following message:

Here’s the JSON I’m trying to import:

[
{
“accessed_at”: “2020-09-05T22:09:15”,
“content”: “# 2018 Tech Summit Goals Presentation”,
“created_at”: “2020-09-05T22:10:03”,
“created_latitude”: “46.055530370721598”,
“created_longitude”: “-88.70568067585917”,
“flagged”: false,
“folder”: 0,
“languageGrammar”: “Markdown”,
“modified_at”: “2020-09-05T22:09:15”,
“modified_latitude”: “46.055530370721598”,
“modified_longitude”: “-88.70568067585917”,
“uuid”: “1b8abd97-2329-4f11-8283-035626e1288b”
},
]

Any ideas on what I’m missing?

agiletortoise · September 6, 2020, 12:46pm

Looks pretty close. You need a tags: [ ] key, even if not assigning tags and the dates should be valid ISO 8601 dates, which need a time zone specifier…which means just add a Z at the end if your dates are already UTC dates.

[Update: Also, the long/lat should be numbers, not strings. Do not wrap them in quotes]

Andreas_Haberle · September 6, 2020, 6:03pm

If this is the last entry the extra comma between the closing braces might be the problem

gcaprio · September 12, 2020, 4:19pm

Thanks for the help everyone. I’ve gone ahead and uploaded my solution (Python3) here for anyone else that needs it:

gist.github.com

https://gist.github.com/gcaprio/023e2e36c5ad99772a440cb2a8a957d7

convert_markdown_files_to_drafts.py

# Usage:
# python3 convert_markdown_files_to_drafts.py <directory> > DraftsExport.draftsExport
import os, time
import json
import uuid
import sys

from datetime import datetime, timezone

output_datetime_format = '%Y-%m-%dT%H:%M:%S%Z'

This file has been truncated. show original

Thanks!

Andreas_Haberle · September 12, 2020, 7:33pm

Great stuff less than 100 lines thats your genius in python.

Just love this

Andreas_Haberle · September 13, 2020, 3:16pm

Shall I add some improvements. Maybe adding a logging (loguru is a great choice) and a command line interface (like click or typer)

I would see this as a giving back effort. But only if you like

gcaprio · September 13, 2020, 9:57pm

Sure thing feel free to do whatever you want to meet your needs.

Andreas_Haberle · September 14, 2020, 8:13pm

Started to modify the script

If you want to take a look it is here https://github.com/42sol-eu/drafts_pro/tree/setup/_scripts

Main changes:

using typer
add a config.json file

Plans:

looking into a front-matter / back-matter function (see YAML <> JSON Converter/Support)
looking into options
adding local testing of the functions
error handling (if something is missing)
other config file formats (like json5, toml and yaml)

Andreas_Haberle · September 14, 2020, 9:09pm

Hi @gcaprio, I am missing something in the import step in drafts. Neither the ‘Import …’ menu nor the Drafts/Inbox folder seems to work.
it only imports my json and does not create to drafts.

how did you do it?

Andreas_Haberle · September 14, 2020, 9:47pm

@gcaprio found it …

The Date is not right yet, but that should not be so hard to fix

gcaprio · September 16, 2020, 12:53am

oh awesome. did you figure out the date issue?

Andreas_Haberle · September 16, 2020, 4:13am

did not look into it yet.
I guess a string formatting or time zone thing

agiletortoise · September 16, 2020, 12:44pm

The dates should be iSO 8601 format to get parsed properly.

Andreas_Haberle · September 16, 2020, 6:17pm

Yep. ISO 8601 has two possible ways to define the time zone. Python picks the other than drafts. A simple replace fixes this. Now it works. Thanks Greg.