I just imported a bunch of text files using the “watch this folder” feature and for some reason, it created lots of duplicates
I can’t see an action in the directory for “remove duplicates” (which surprises me)
Is there a way to do this easily?
Removing duplicates would need to have a lot of flexibility built into it. As well as the content, you could need to consider things like:
- Which instance to keep: most recently modified, first one created, flagged, etc.
- Alternatively, would deduplication remove all older versions and create a new one.
- What meta data to compare: created/modified dates within a particular time frame, geolocation, tags, etc.
- Whether to include archived and trashed drafts.
- Whether it is an exact match, or a very close match: e.g. should leading and trailing whitespace be trimmed.
- What would need to be done in regards to draft versions?
It could absolutely be done, and done in a way to make it easy to deduplicate, just run an action. But it isn’t trivial to build such an action as there are many options to consider, and many of them could be compound considerations. The logic would need to be perfect to avoid data loss and so it would require a lot of testing.
That being said, let’s go back to your underlying issue.
Your first option on deduplicating would be to remove all the drafts you just added via the watched folder. Presumably this is the iCloud “Inbox” folder for Drafts. If not, please provide more details.
Assuming it is the Inbox, that can have a tag set against it which may help you identify your imported files. If not, then you would need to rely on a time frame to identify those auto-imported drafts … and hopefully you can pick out anything you have done manually since then too.
I would suggest starting by moving those imported drafts into Drafts’ trash. This should give you a clean slate, and should you find there is anything accidentally missing, you can grab it from the trash.
Then we can focus on the original issue of how the duplicates were generated. Let’s start with some basic checks.
Checks:
- The files being imported each contain unique content.
- If you import files with duplicate content, you are going to get drafts with duplicate content.
- On your devices, only one has been enabled to import content from the folder.
- If you have multiple devices set to import content, each device could be creating a new draft and the overall result would be a duplication.
Assuming the checks are all okay, we will need to get some more details as to the pattern of what is occurring:
Some initial things to watch out for and try are:
- Do you get duplication dropping one file at a time, or is it when you drop a large set of files? If so how many?
- When duplicates occur, how far apart, in time, do they occur? A couple of seconds, 30 seconds, a few minutes, etc.
I have some additional thoughts, but they are very much based on what you find when you start digging in on all of the above.
Thanks for taking the time to give such a considered response
I give every imported item a tag (import) and have set up a workspace to see them in isolation. Practically, I have gone through and manually deleted the dupes for now.
It feels like some kind of bug because Drafts keeps crashing during the process (I put maybe 80 .txt files in the Inbox folder (yes, iCloud)
Each time there is a dupe, it created about 7 of them
Because you are getting app crashes, I would strongly recommend that you raise a support request with Greg:
Contact Support: Use the “Send Feedback” option in Drafts settings (or the Help menu on Mac), or email support@getdrafts.com.
Anything where you can reproduce a crash is something that would benefit from investigation at the internal code level.