😄 KanjiEater's Smart Japanese Kindle Highlights

This add-on makes Anki cards from words you've highlighted. By highlighting only the word you read on your kindle, this add-on can generate a card containing:

1. The deconjugated word you read
2. The sentence the highlighted word was used in
3. The meta data of when and where this word was read

Here's a video covering the features & setup:

How does it work?

It works by looking up the highlighted text in the Kindle's built in Vocab Builder feature which automatically records sentences of words you looked up.

It then matches the words to sentence usages, and pulls the deconjugated word, sentence, and meta data, and creates a card in a deck named after the book.

Every time you run it, it records when you last generated cards, and continues from where it left off - making adding new cards from you Kindle practically effortless.

Current Limitations

• This only supports kindles in the Japanese language and Japanese books; Each kindle's highlight format is different depending on the language
• Windows is the only officially supported platform, though Mac & Linux should work; PR's welcome
• Only supports Kindle's with My Clippings.txt (no html/Android based clippings)

Kindle Limitations

• Books must use a Kindle vocab compatible format: .azw3 is compatible; .mobi should be too; I used Calibre to easily convert my books that were mobi to azw3 without any issues (covers still work if you delete any Amazon ids)
• Only matches what your Kindle dictionary finds. The Kindle automatically makes /system/vocabulary/vocab.db and updates it with words that were found in the dictionary. The entries in this are automatically created by the Kindle itself. If your dictionary couldn't find it, you'll have to make a card manually for the word (the add-on will prompt you in this case).

Instructions 0. Install the official Anki Japanese Support add-on: https://ankiweb.net/shared/info/3918629684

1. Make sure Vocab Builder is on, on your Kindle device (inside Settings -> Reading Options)
2. Make sure you Kindle is in Japanese
3. Make sure your Kindle highlights are in KindlePath/documents/My Clippings.txt, where KindlePath is something like F: or whatever your Kindle's drive is: F:/documents/My Clippings.txt
4. I recommend erasing the contents of My Clippings.txt for the first time you use this, just to make sure you don't have any words or sentences that could mess up the matching.
5. Set up the add-on config variables:

{
    "model_name": "Japanese", 
    "word_field": "Vocab",
    "sentence_field": "Expression",
    "source_field": "Meta",
    "deck_name": "!優先::1 自分::kindle",
    "last_added": null,
    "path": "F:/"
}

6. model_name is the model to use for the cards. This model must have fields in it named whatever the word_field, sentence_field, and source_field are.

7. word_field is where the deconjugated word will be put in the card

8. sentence_field is where the sentence where the word was used will be put in the card

9. source_field is where the meta data of what page and what time you highlighted the word. It looks something like this: ページ78 2021年2月3日水曜日 8:50:57 食べて

10. deck_name is the "root" deck where sub decks will be created. Every highlight imported will be placed in a subdeck named after the book it was read from. So in my case, I have a deck named !優先, which is where my Japanese cards are, with a subdeck of 1 自分 where I put cards I make myself, with a sub-deck of kindle where are the kindle cards are generated to. Inside kindle I have decks like 狼は眠らない, which is the name of a book I read, and generated cards to automatically.

11. last_added is when the last Kindle Highlight was added. Don't change this unless you want it to rescan & readd cards from that time. If you set it to null it will start at the beginning of all of your highlights.

12. path the root path of your Kindle device. It expects F:/documents/clippings and a hidden folder & file F:/system/vocabulary/vocab.db to exist.

Tips on Usage

• After hitting 10k sentence cards, I switched all of my cards to Vocab cards. As generated vocab cards, these autogenerated cards work great. As sentence cards, you would want to manually make the sentences as short as possible to provide context. Shorter sentence cards are better. Hot Take: Vocab cards are even better if you're actually spending time reading daily. Use Anki for vocab entries, use immersion to get things to actually sink in. Sentence cards take 30% longer to review I've found.

Thank You

• HelenFoster: Your add-ons are some of the best written code, and I frequently use snippets in my own code
• gmcmanus: Your English kindle highlighting add-on helped inspire this add-on & I was able to re-use a portion of that code
• stegatxins0#7346: https://rentry.co/mining Dope mining guide that contains an alternative set of instructions (that are probably better than my own)

Update History 2/13/2021 - Initial Release! 2/14/2021 - Bug fix on importing the add-on. Added a dialog so feedback of the highlight scanning starting is immediate (otherwise if you have hundreds of highlights it might look like it's not working). 2/20/2021 - Better error handling for bad configuration. The add-on will now tell you (instead of throwing an error) when something goes wrong with file paths, Note Types, and missing fields 2/20/2021 - Updated again to fix issues with parsing for some users 3/11/2021 - Add-on now ignores buggy highlights that are duplicates (same source and vocab) or if the highlight is blank. The add-on now creates cards in chronological order. 4/15/2021 - Older versions add cards as duplicates. Update to 2.1.35 to avoid chaos 4/18/2021 - Bookmarks no longer show in bad highlight clippings popup. Also fixed an issue where the matching between kindle's built in vocab.db and highlights could get off & skip highlights in between. More highlights should add automatically now. 6/5/2021 - Massive Update! More highlights should add automatically. The accuracy relies on the highlight and the dictionary lookup happening within two minutes of each other. The deinflection logic has been improved to now include words from 大辞泉 & 大辞林. Mecab fallback parsing has also been improved. Other minor bug fixes has been included as well. 8/1/2021 - Another large update. The code that matches the vocab lookup to the clipping has been completely rewritten, and is now 99.9% less dumb. A match should always happen now if the Kindle Vocab Builder picked it up (which doesn't always happen as the Kindle is buggy). There was also a bug fix for a breaking error if you managed to get an image highlighted in a clipping (a very odd thing one user reported on Discord). 11/27/2021 - Explicit error if your kindle isn't in Japanese (add-on requires your Kindle language to be in Japanese). Also fixed the locale error others were seeing (hopefully). Please make any additional reports on the discord server.

Support

If you have an issue, join the discord & post in #tech-support: https://discord.com/invite/agbwB4p

If you find my tools useful please consider supporting via Patreon. I have spent countless hours to make these useful for not only myself but other's as well and am now offering them completely 100% free.

Motivate me through PayPal: Please follow the a link below to buy me a coffee, sandwich, meal, or anything else you'd like if you find my work interesting.

If you can't contribute monetarily please consider following on: