[Technology] Data Debt: part 3

While in school, and for a period after, I neglected my systems of organization.  I continued to accumulate information, but without spending effort to manage it.  This has resulted in a large data debt. 

I have

• years of e-mail, chat and text logs in various formats that I want to de-duplicate and consolidate.
• thousands of screenshots and saved images of memes and wallpapers and artwork from some of my favourite video games and media sitting in massive folders.
• years of photos grouped just by date taken (surprisingly useful) with no indication of people present (actually not a huge problem given Google Photos)
• thousands of To Do items spread across various formats and systems, that have no priority, they just end up in an endless queue that only ever grows.
• old websites, blogs, accounts that are redundant or obsolete.
• projects that need starting, video games that need completing.

Part of my problem is hope.  I have hope.  Hope that this can all be managed.  Hope because I have the skills necessary.

Part of my problem is that I am late.  There are good opportunities, especially at the time of transitioning from one process to another, or at the time I acquire a record, to act then and keep-up.  They were missed.  And now I have debt

I just need the time.  So here I am, stealing an hour here, an hour there, out of my busy days, not dealing with problems of today, but dealing with a self-created problem of years past, for a goal that, like my dad's, may be pointless.

How many photos do I ever review?

How many e-mails will I ever re-read?

How many ToDo items do I ever expect to complete?

My alternative is recency.  Focus on what is recent and applicable to me today.  But I know I can do it, and I know that, like my father sifting through his decades of paper data, there are gems to be found.  Memories to recall.  People that I do not want to lose as the unfrequented connections in my brain fade to make room for more recent experience.

I have created a large data debt, one that might be manageable with the programming skills I have, and maybe more manageable as AI continues to improve.  But does it matter?  Probably not.  Just my vanity.

[Technology] Data Debt: part 2

As just noted, my father managed to accumulate quite a bit of information in physical form that he now spends time sorting through and discarding what has no value after all, which is most of it.

I think about that in relation to the data I keep, mostly digital.  Files upon files upon files.  Photos upon photos.  A vain effort to preserve every text message and e-mail forever.  A vain war against data loss, against entropy, against chaos.  Vain, my vanity.

I am not a fan of the paradigm of search first.  I love being able to meaningfully search my information.  I do not believe, at present, or perhaps ever, that search will be capable of being able to completely and comprehensively report back to me the information I seek.  I can only search for what I know exists.  Artificial intelligence has made search much more useful than ever before; e.g. I can search for "cake" in Google Photos and I get hundreds of photos, mostly actually of cake!  But I also know many more photos that I have taken of cake are missing, and there are photos not of cake present.  Precision, accuracy and recall are not at 100%, yet, and given the indefinite nature of some definitions, can they ever be.

Organize.  A dead motto.  Another vanity project.  Just as I do not expect search to be able to accurately identify all of the most relevant results for me, I do not believe that I can adequately and exhaustively categorize all of my information in a complete way.

But between the two, I still prefer organizing over searching.  I still prefer a reasonable hierarchy of folders and some futile "tags".  It scales a bit better.  Perhaps I cannot granularly categorize every record I have, but I can at least definitely categorize them into broad groups that I can add specificity to as needed.

What is the major obstacle that I have?  Volume of information.  I collect too much information.  And I am generally unwilling to let any detail go.  Storage is cheap.  I have a 1TB SSD that I purchased for 80CAD, and it stores tens of thousands of photos and millions of lines of text from the past decade, in a small block that can fit into the palm of my hand.

And my flexible approach to organizing it all works fine... if I can keep up.  But my data debt grows faster than I can spend my daily available time to manage it.

[Technology] Data Debt: part 1

Technical debt.  Time debt. 

My father is a bit older now, and he has been going through his Stuff.  Not just big physical Stuff like objects and thingy things.  But also informational Stuff, like receipts, letters, newspaper clippings, magazines, mileage notebooks.  Stuff he read or took notes on over the past several decades that have accumulated into quite a many boxes (formerly of shoes)or Piles or Stacks.  Sometimes, bags.

He's looking at all of this informational Stuff, and wondering, what was the point?  He definitely found value in some of it.  One of his joys in the past few years has been surfacing a memory of some event, of something he read or did, and then searching through his many records to find the Evidence of it.  I can understand the thrill.  Finding that newspaper clipping of something he did in our small town, once, or a letter from a friend from 2003, and hey, maybe Richard can find the phone number of the sender now, and hey, I did, and now my dad has renewed contact after a decade and a 1/2.

But there's a lot that he will never need, and never really did need in the end.  He has been meticulous in recording how many kilometres he has driven between gas refills, how much gas he added, and how much it cost.  It has given him a good sense of his spending in that regard, and of his cars performance.  It is all jotted in tiny notebooks.  Nothing digital.  Nothing that he can load easily into a spreadsheet now, and review larger trends.  And, honestly, numbers that he has not needed for three decades.

I love this history that he has collected, these peeks into his life and mind and priorities over the past several decades.  I do not get to enjoy them so directly, though.  I am reluctant to dig through his Stuff, it feels like an invasion of privacy.  He is shredding or recycling stuff now that he does not expect to need after all, which is most of it.  Saving information for rainy days that mostly never came.

I find it interesting that his approach is to review it all.  It definitely has had merit.  He finds random points of interest and it creates conversation topics for us.  It used to be something he could show his friends at the coffee shop, back when coffee shops were places where you could sit with friends.  But there is so much, and so much he finds is of no value to him.  I am pretty sure this year he will give up in the face of the sheer volume of information he has collected in the past several decades and just say good bye to it all.  It was never that useful anyway.

late

Late at night, cool air after a muggy day.  Some residual moisture in the air.  Darkness and stars.  Lilacs on a breeze.  That's me.

[General] She-Ra and the Princesses of Power

I believe that She-Ra may officially be my favourite television series to date.  It's hard to say that, as some media fluctuate a lot over time, or have different favourites that meet different needs.  But sometimes it's clear.  Scott Pilgrim v.s. the World is clearly my favourite film since I was first taken to see it in theatres.  Breath of Fire II has been my favourite video game since elementary school. 

Some past favourite televisions shows or candidates have included Teenage Mutant Ninja Turtles, Sailor Moon, the Vision of Escaflowne, (maybe even Last Exile?), Buffy the Vampire Slayer, Gilmore Girls,  Star Trek: The Next Generation, Orphan Black, and (very recently) I am not okay with this.  Some of them may still be favourites within certain specific categories.  I'm probably missing a dozen series right now that have been formative but whatever reason escape my memory at this instant.

It's also hard to say, as there's a strong recency effect.  Often I'll see a movie and be in awe of how perfect or amazing it was, but a week later, the feeling has subsided a bit.  I felt that way a bit about Star Wars: Episode I: The Phantom Menace, and it subsided a bit.  I felt that way about Star Wars: Episode VII: The Force Awakens, but it remains one of my favourite Star Wars films.  My favourite films are much easier to list, though.  My top five include: Fucking Åmål, Garden State, Eternal Sunshine of the Spotless Mind, Amélie, Excalibur, Zootopia, and the aforementioned top of the list, Scott Pilgrim v.s. the World.  (Don't try counting those.) What happened to The Life Aquatic with Steve Zissou?   I'm not sure, but it was on the list once.

Music also jumps around a bit.  Aqua, Evanescence, the Human League, Josh Ritter, The Frames, Grimes, and now Kitzl.

She-Ra has had five seasons, which is a lot.  But the first episode aired in November 2018, which is recent.  5 seasons in 18 months.  A strong recency bias.  But, it's so re-watchable in a way that still hits hard each time.  I don't think I've cried this much due to a story in long time (or perhaps ever so much, with Sailor Moon perhaps being the second most moving?).  Its characters have a depth and richness that I often feel lacking and they have priorities I care deeply about.

I haven't cried much in the past 4 years for a variety of reasons.  I think about that a bit here and there.  It's been very important for me to able to empathise with characters again so strongly.  I'd talk about specifics of the story that contribute to this, but as I'm adamantly against spoilers and I adore this show so, I won't.  Let me know if you ever watch it, though.  I have a lot of thoughts and feelings to share.

[Technology] JavaScript's querySelector () is one of my favourite APIs

I don't mind traversing a DOM or using XPath, but CSS selectors work so nicely.

[Gaming] Fire Emblem: Three Houses

(potential spoilers, but I'm being vague)

Too big for my own good: extroversion

I've gotten myself into a small predicament.  I overextended myself in the game, and now I just want to finish it with increasing desperation, so I can move on to other things.  It's not taking up that much time out of my life, as I'm only playing it for a bit a day week, but I feel like so little progress is made.

You see, in the game, you get to lead one of three different houses at an academy.  However, you can also poach students from other houses by recruiting them.  I did.  I recruited every possible student I could.  It was very tense near the end of the first part, when I was trying to recruit that last person, having to frantically upgrade my own skills to persuade them to join, but I did it. 

Complexity and spreadsheets

It's great in many ways.  I get to see a lot more story through the dialogue of my own house's students and the recruited ones.  It changes the feel of the story, too.  Like the people I was friends with early on remain my allies against a much more isolated antagonist.  However, it's greatly increased the complexity of managing inter-character relationships, as well as my other goal of getting everyone to a Master class.  I ended up building two specific tracking spreadsheets that I hope to post here eventually. One has tables for the various Master classes, which skills at which levels are required for them, and the applicable skill levels of the characters I manage.  The other spreadsheet tracks which meals each character really likes at the dining hall. 

Meals for classes, it's like Raphael is always saying, gotta eat to grow

The meals are actually pretty important.  To be able to teach my students and colleagues and improve their skills, they have to be motivated.  Feeding someone a meal they like increases their motivation, allowing me to instruct them.  Each meal is shared between myself and two others.  There's a limit to how many meals a week I can eat.  So, to be efficient, I want to pair up as many unmotivated students as possible with meals they both enjoy to motivate both of them in one meal.  It was fairly annoying having to page through the list of meals and each meal's list of characters that enjoy them, so the spreadsheet really helps by allowing me to filter on two characters and see just the meals that they like.  There's another component, that meals need ingredients, and you need to grow ingredients in the greenhouse with seeds, with some element of random chance as to what actually grows, but thankfully, I've not had to directly worry about that; just regularly growing random things has given me most of the ingredients I've ever needed.

One more consideration for meal sharing is whether the relationship between the two characters I want to dine with can be built up more.  So, first I look at my ever-decreasing list of characters that still need to attain a Master class, then I check with whom each can still build support relationships, favouring the ones I find most interesting, then match their meal preferences.  Phew.

I've also taken to gambling some; even if someone is unlikely to pass a test for their Master class, e.g. their odds are 33%, I've still been having them take them.  They'll be a bit underskilled, but at least then they can grow within the class, rather than driving me nuts trying to rank up dozens of skills across a couple dozen characters. :D

Wish list

Some things I wish the game had are:

• when shopping, it would tell you the quantity of an item that you already have
• the ability to sort more lists (inventory in particular is painful)
  
• see visible restoration over time to damaged buildings
• more books to read in the library

[Technology] Messaging and the Flow of Time

Something fairly depressing in the realm of technology is the lack of monotonic progress messaging has made in the last 30 years.

Recently, Instagram finally started supporting messaging from the web (theverge.com).   It still doesn't allow me to organize messages by archiving them, though.

Facebook Messenger lets me archive them and message from the web or my phone (yay), but it's horrible trying to actually read their history; trying to search or browse back in time is incredibly painful.  That was something that used to be so easy a decade ago.

Every few years, a new messaging platform rises up, and with it a classic set of limitations.  How many times have I been compelled to use a new platform to find that, once again, group messaging or video chat aren't available?  Or that there isn't end-to-end encryption?

I briefly had hope a decade ago when it looked like XMPP was going to help standardize messaging protocols and that I'd finally be able to interoperably chat with friends across networks.  Now they're more siloed than ever before.  I almost hope for Facebook to integrate Messenger, Instagram Direct and WhatsApp so that people will stop pestering me to install yet-another-privacy-violating application.

On a positive note, I really enjoy per-message reactions that a few services like Facebook Messenger and Signal now support.

[General] It's a mood

I love the modern usage of "mood".  Big mood.

[General] Unintuitivity

Apparently most people in an FB group I am in (mostly younger people) have backpain and sneeze in a regular day. O_O

[General] I am not okay with this.

"I am not okay with this" is one of the best TV series I've watched so far.

[Technology] Java 13, switch and text blocks, yahoo!

From
https://blog.jetbrains.com/idea/2019/07/intellij-idea-2019-2-java-13-preview-features-profiling-tools-services-tool-window-and-more/

Java 13 is coming out in September, and, as usual, IntelliJ IDEA is ready for it. The IDE provides support for the Switch Expressions preview feature (JEP 354) and its new syntax, and for the Text Blocks preview feature (JEP 355).

 Yes, Java will soon support multi-line String literals and more compact and versatile switch statements/expressions.  Wahoo!

[General] Blogging and barriers

I think a big reason I blog less these years is because I don't spend much idle time at a full computer anymore.  There's also a much more significant reason in that I feel like I've figuratively lost my voice (what is worth saying anymore?), but using the Blogger app on my phone has been unsatisfying for a variety of reasons.

It would be easier if I could set up some shortcuts on my phone's home screen to pop open the composer with some different pre-templated text.  E.g. for my general posts, have "[General] " in the title box and #General in the labels.  Alas.

[Technology] Let's Encrypt!

For a while, this blog's appearance was a bit broken, because of issues with a mix of unencrypted HTTP and encrypted HTTPS content.  Eek.

I finally went about using Let's Encrypt to get some signed certificates for my domains to help improve the privacy of your browsing session here.  Enjoy.

[General] Absolute Zero

My favourite local musician, Kitzl, released a new single recently, Absolute Zero, and I really like it.

[General] Miss Anthropocene

My favourite musician of the past several years, Grimes, has released a new album, "Miss Anthropocene".  Wahoo.  I particularly like the track "Delete Forever" on it.