[Microblog] Life and smiles

I was responsible and did something good for me, and am now smiling at grass again.

[Technology] Accessing files.web.uoguelph.ca from Linux when off campus

Until a few weeks ago, I was able to login remotely to my University's web hosting server.  Now, I need to connect via VPN first.

On Fedora Linux, I made sure I had vpnc installed and NetworkManager-vpnc.   Then I went to Network Manager's settings/preferences, added a new connection (+ in the bottom left), chose VNC, and chose the vpnc client.  Then I used the settings from this page:

https://www.uoguelph.ca/ccs/internet-phones/virtual-private-network-vpn/using-vpn

It beats using the creepy script from vpn.uoguelph.ca at least.

[General] Heading Homeward

It's the Flesherton Fling this weekend!  You should totally come out.  We used to have the Split Rail Festival.  Then we started making fences from metal and it seemed outmoded.  Now we just have flings - the moral decay of our century.

[Technology] Automating my life away

A decade ago, part of a job was to help automate creation of a report that normally involved a human scanning through dozens of spreadsheets each year, copying and pasting numbers and generating graphs and tables, manually.

The place where I worked predominantly used Perl.  (It was a science laboratory.)   My solution was to use OpenOffice's GUI to bulk-convert the Microsoft Excel spreadsheets (binary XLS files) to whichever file format OpenOffice used at the time that was actually a collection of XML in a zipped directory. (Was that .sdc?)

I then used Perl's libxml wrapper (probably wrapping libxml2) to parse the XML into an SQLite database.  (Or was it PostgeSQL like everything else there?)  I had tried to use some other Perl XML library, but it was Super Slow compared to the snappy libxml2 (written in C).  Like, a difference in many minutes to run versus one second.

From there, I wrote some Perl CGI scripts to automatically generate most of the tables and graphics into HTML form, in some fashion that looked nice when imported into OpenOffice. (Really?!)  The person who was responsible for the report, who normally took a few weeks to do, said that it would save him so much of the time, that it would take him now a couple days to look at the data and write conclusions and maybe do some cleaning.  He marveled at automation and how quickly he could be replaced.  The automation revolution in action.

Today I used LibreOffice to convert about a hundred spreadsheets into CSV files with this one command:

find . -name "*.xls" | while read line; do
  (cd "$(dirname "$line")" &&
   libreoffice --headless --convert-to csv "$(basename "$line")");
done

I am not sure that OpenOffice could do headless conversion back then.  I was probably overwhelmed by all the potential everywhere to discover the functionality if it exists.  This would have helped free a few hours of my life.

Automation, a revolution.

UPDATE:
Converting to CSV only does the 1st sheet tab.  You apparently can't specify a different sheet to be the active sheet.  I also had to convert some other old spreadsheets for easy parsing (mostly just grep'ing through them to find data).   In that case, I used --convert-to html which was beautiful, even if one of the resultant HTML files was 138MB large.

[Technology] Starting Firefox Faster

summary: if you use pinned tabs, and they're all loading at start, despite setting tabs to not load until selected, this post is for you.

I have two hard disks, a 24GB SSD (fast!) and a spinner.  I noticed that to fully load Firefox and wait for everything to settle, it was taking about a minute and a half, back when my root directory was on the HDD.  Recently (see earlier posts), I moved my root onto the SSD.  That improved things marginally.  Firefox was now settling in around 1m12s.  I tried moving my profile onto the SSD.  Now I was down around 1m00s.  (These are averaged from multiple warm runs.)  That is still abysmal.

One thing I notice is that a private web app I wrote (look up My Daily in past posts maybe) is guarded by an .htaccess file, and I have it as a pinned tab.  In that full minute, before things finish settling, I get prompted to authenticate myself.  I then recalled that I had enabled the Firefox option to load tabs when clicked (Preferences > General > "Don't load tabs until selected"), so why is it loading?

All the pinned tabs actually seem to be loading at startup.  Grr.  Perhaps that is tunable?

I went to about:config, and sure enough, typing in "pinned" filtered down to this setting:

browser.sessionstore.restore_pinned_tabs_on_demand

defaulting to "false".  I set it to "true", and now Firefox takes <3s to open and settle.  Sure, clicking on a pinned tab now takes a normal time to load the tab, but it's better than loading GMail, Google Calendar, Blogger, Google Drive, Google Keep and my web app ALL AT ONCE.

[Technology] Resizing my encrypted /home partition

I just wrote a post about why I reinstalled Fedora 22: because I wanted root to be on the SSD, and I needed to purge a lot of old installed software anyway. :P  And how that led to trying out the open source b43 wireless driver.

This is more interesting than my wireless problem.  Because I no longer needed a / partition on my HDD, I wanted to extend my home partition.  It isn't as dead-simple as back a decade ago.  Back then, I would use fdisk to delete and recreate partitions and resize2fs to resize the file system.

Now I have layers upon layers of useful abstraction.  I'm calling each something potentially inaccurate, and the tool I needed in brackets. (Those with wildcards match multiple tools with the same beginning.)

Top

• ext4 file system (resize2fs)
• Logical volume (lv*)
• Volume group (vg*)
• Physical volume (pv*)
• partitions (fdisk)

Bottom

At first I was almost a little annoyed at the apparent complexity, but it's actually straight forward.  My drive is still divided into multiple partitions in a partition table.  (/dev/sda1, /dev/sda2, /dev/sda3, etc; I have 8 for various reasons).

Physical volumes seem to correspond to those partitions, but mostly just the useful storage ones (e.g. I don't have them for EFI or swap partitions).  pvcreate lets me define them.

The physical volumes are collected into volume groups.  For example, /dev/sdb1 (my intended root) might be a part of "pantaloons-fedora-ssd" (if I named my machine pantaloons), and /dev/sda2 and /dev/sda4 might both be part of "pantaloons-fedora-hdd" (two large areas for my /home that I'd like to combine later, but which are currently separated by some other, annoying partition).

The volume group kind of looks like a single contiguous (!) entity (despite being a collection of disparate physical partitions).  Then, I can define logical volumes atop the volume groups.

I can have both / and swap as separate logical volumes atop "pantaloons-fedora-ssd" (talk about cramped - maybe I don't want to hibernate after all), and have a single logical volume for /home atop the "pantaloons-fedora-hdd" volume group.

This is where things get a little trippy though.  Where does the encryption happen?

For my root file system, it looks like it happens like:
partition /dev/sdb1 > LUKS > physical volume > EXT4 file system

For my home file system, it looks like this:
partitions /dev/sda2,/dev/sda4 > two physical volumes > 1 volume group > 1 logical volume > LUKS > EXT4 file system

FS	Ext4		Ext4
(LUKS)			/dev/mapper/luks-BAR
LVs	root	swap	/home
VGs	pantaloons-fedora-ssd		pantaloons-fedora-hdd
PVs	/dev/mapper/luks-FOO		/dev/sda2	/dev/sda4
(LUKS)	/dev/mapper/luks-FOO		/dev/sda2	/dev/sda4
partitions	/dev/sdb3
disks	/dev/sdb (SSD)		/dev/sda (HDD)

The home partition and its LUKS (using cryptsetup/dm-crypt) was originally configured through whichever tool Fedora provided years ago (preupgrade? fedup? anaconda?), while the configuration on my SSD was what was recommended in F22.

Basically, I had to (with steps that actually increase space emboldened):

• define a new partition in the partition table, 
• define a new Physical Volume (PV) over top it, 
• extend my Volume Group (VG) to include the new Physical Volume
• unmount the file system I was going to extend, 
• close its LUKS setup
• extend the file system's Logical Volume (LV) overtop the new space in the Volume Group
• reopen the LUKS setup
• resize the LUKS setup (are these two out of order? hope not!), and
• resize the Ext4 partition

A bit more detail, here are the steps I used on my system (from my bash_history, probably missing some steps; I think I also used lvremove or pvremove to remove LVs and PVs related to the old home partition, oh well!):

• # cfdisk, to delete the old root on the /dev/sda4 partition and recreate it
• # pvcreate /dev/sda4 (creates a physical volume)
• # vgextend pantaloons-fedora-hdd /dev/sda4 (add the PV into the VG increasing its overall size)
• make sure /home isn't mounted
• # fsck.ext4 -C 0 -f /dev/mapper/luks-BAR (let's make sure the inodes are fine on our file system before messing with it)
• # cryptsetup luksClose luks-BAR (close LUKS while we change the underlying LV)
• # lvextend -L +61G /dev/pantaloons-fedora-hdd/home (increase the size of our LV within the VG)
• # lvextend -L +242M /dev/pantaloons-fedora-hdd/home (catch a little more space)
• there is probably a way to just extend it to the full space available, maybe read the man page
• # cryptsetup luksOpen /dev/pantaloons-fedora-home luks-BAR
• # cryptsetup resize luks-BAR (resizing the LUKS setup)
• # fsck.ext4 -C 0 -f /dev/mapper/luks-BAR (for paranoia maybe)
• # resize2fs /dev/mapper/luks-BAR (finally, let's resize our Ext4 file system)

Sadly, I can't cite the sources I pieced some parts together from, because I did this form a live USB key which is supposed to keep history as persistent storage but the browser history was empty when I tried checking it later :S  However, man pages were also useful.

[Technology] How to Mount the Overlay Filesystem of a Live USB

So, I wanted to demonstrate the commands in the previous post used for resizing my file system, but of course I did that from a live USB (since I wanted my file systems unmounted). However, I did not want to reboot my computer just now to copy root's bash history, so I looked up how to mount the overlay using these instructions:

https://fedoraproject.org/wiki/LiveOS_image

The simplified version would be:

1. insert USB key into computer
2. identify it
   $ journalctl --system
   mine was created as /dev/sdc1
3. mount it, creating target directories as you go:
   # mount /dev/sdc1 /tmp/sdc1
4. mount the read-only squashed file system:
   # mount /tmp/sdc1/LiveOS/squashfs.img /tmp/squashfs
5. create a block device for its ext3fs image:
   # losetup /dev/loop1 /tmp/squashfs/LiveOS/ext3fs.img -r
6. create a block device for the overlay:
   # losetup /dev/loop2 /tmp/sdc1/LiveOS/overlay-LIVE-foo
7. calculate size of loop1
   # blockdev -q --getsz /dev/loop1mine was 12582912
   
8. define a device combining the two
   # dmsetup create myliveos --table "0 12582912 snapshot 7:1 7:2 P 8"
   myliveos: my own name for the device, call it what you will
   12582912: the # of blocks I acquired from step 7, put in yours
   7:1 references loop1
   7:2 references loop2
9. mount the combined device
   # mount /dev/mapper/myliveos /tmp/myliveos

Now I can cd into /tmp/myliveos, and get my .bash_history!

[Technology] Installing Fedora 22, again: wireless Internet and resizing file systems

I have a 24GB SSD and a 500GB HDD in my computer.

A few months ago, every time my computer hibernated or slept, my file system on my SSD got corrupted, and I had to fsck it to repair it.  I ended up just creating a new partition on my HDD and installing my OS there.

Recently, I've grown reckless, and decided I wanted to have my OS on the SSD after all.  I did some SMART tests and my SSD seems to be fine, so pretending to forget about the earlier drama, I reformatted the SSD and re-installed Fedora 22.

Wireless Internet

Of course, the Internet doesn't work out of the box.  That's because of Broadcom's proprietary drivers and firmware.  Previously, I used the wl driver, using akmod-wl and kmod-wl.  My device:

03:00.0 Network controller: Broadcom Corporation BCM43228 802.11a/b/g/n

I looked at the open source driver, b43, though, and it claimed to now support it(!).  I apparently needed the firmware from the Windows driver, though.  Sigh.  Indeed, journalctl --system (instead of dmesg) reported this:

[  162.191310] b43-phy0: Broadcom 43228 WLAN found (core revision 30)
[  162.191764] b43-phy0: Found PHY: Analog 9, Type 4 (N), Revision 16
[  162.191776] b43-phy0: Found Radio: Manuf 0x17F, ID 0x2057, Revision 9, Version 1
[  162.192544] Broadcom 43xx driver loaded [ Features: PMNLS ]
[  162.192570] b43 bcma0:1: Direct firmware load for b43/ucode30_mimo.fw failed with error -2
[  162.192596] b43 bcma0:1: Direct firmware load for b43/ucode30_mimo.fw failed with error -2
[  162.192995] b43 bcma0:1: Direct firmware load for b43-open/ucode30_mimo.fw failed with error -2
[  162.193015] b43 bcma0:1: Direct firmware load for b43-open/ucode30_mimo.fw failed with error -2
[  162.193019] b43-phy0 ERROR: Firmware file "b43/ucode30_mimo.fw" not found
[  162.193022] b43-phy0 ERROR: Firmware file "b43-open/ucode30_mimo.fw" not found
[  162.193024] b43-phy0 ERROR: You must go to http://wireless.kernel.org/en/users/Drivers/b43#devicefirmware and download the correct firmware for this driver version. Please carefully read all instructions on this website.

This lead me to http://linuxwireless.sipsolutions.net/en/users/Drivers/b43/#Fedora, which has a broken link to http://linuxwireless.sipsolutions.net/en/users/Drivers/b43#other_distros further down the page.


In the end, I had to do these things

$ dnf install b43-fwcutter wget     # dnf instead of yum
$ export FIRMWARE_INSTALL_DIR="/lib/firmware"
$ wget http://www.lwfinger.com/b43-firmware/broadcom-wl-5.100.138.tar.bz2
$ tar xjf broadcom-wl-5.100.138.tar.bz2
$ sudo b43-fwcutter -w "$FIRMWARE_INSTALL_DIR" broadcom-wl-5.100.138/linux/wl_apsta.o
$ modprobe b43

And it worked.

Mounting encrypted /home partition

I cheated and used gnome-disks to make my encrypted /home partition auto-mount.  Sorry.

[Microblog] Transient Thought

Bitcoon, a Canadian fork of bitcoin?

[Microblog] Transient Thought

I have lost my playfulness.  I am grown too serious.

[General] Simplicity and Innovation

It can be so hard to just find something simple/"normal".

I go into a store and I want to buy a binder.  They all have designs on them.  None of them are just "normal" - plain and simple.

I want to find a simple muffin recipe.  No, no walnuts, no blueberries, no bananas, absolutely no animal parts, no no no no.  Just a muffin.  Recipe websites have large categories with 90 different types of muffins.  I don't care.  I just want some fat, some sugar, and some wheat in the shape of a muffin.  I'll add something later.

Perhaps that will be where AI assistants/future websites have room to grow: helping simplify what should be simple.

Eventually I found this:
http://vegweb.com/recipes/simple-muffins

[Technology] GoogleCL, OAuth2, and bitrot

I have a fancy process for managing photos I take, curating them, and uploading them to PicasaWeb/Google+ Photos/Google Photos/whatever you want to call it.  It partially relies on googlecl (github, google code (old)).

However, it only authenticates using OAuth1, and that was deprecated recently.  Maybe when I'm done with school I'll try to take it over.  In the meantime, I'll be resurrecting an old libgdata tool I wrote 6 years ago on the other side of the planet.  Yay!

[Technology] Memory leaks, vala, and copying delegates

Some of the Gee data structures, like a TreeSet, can get a comparator, a GLib.CompareDataFunc comparator. 

If I define a compliant static function and hand it over to the TreeSet's constructor, I get a warning that copying delegates is not supported.  I can't really change how TreeSet handles it, and I couldn't find how I was supposed to pass it elsewhere, so I passed it over like this (not exact code):

class ApplePie : Pie {
  private GLib.CompareDataFunc get_cmp () {
    return (a,b) => { return a.foo - b.foo; };
  }
}
class CherryPie : Pie {
  private GLib.CompareDataFunc get_cmp () {
    return (a,b) => { return a.bar - b.bar; };
  }
}
class Pie {
  TreeSet ts;
  public Pie () {
    ts = new TreeSet (this.get_cmp ());
  }
}

You see, I needed different comparators for different subclasses, but I couldn't just hand over static functions.  However, somehow, passing a function instantiated in this manner to new TreeSet () resulted in an increase to the reference count for pie!   And, in fact, I had 10 different collections that made use of 3 different comparators like this.  And I went through 10,000s of pies!  I think you can start to see my problem.

The functions were causing reference cycles so none of my pies were being freed when I was finished with them, and I quickly ran out of memory. :D  Hooray!

Right now, I'm doing what I'm not supposed to, and passing static functions (copying delegates) to the new TreeSet constructor (they're passed from the subclass to the superclass through base (), so that might be a part of the problem) and things are working alright.

I'm grateful for Malloc.mallinfo ().uordblks, and for GLib.Object.ref_count for helping me see into problem today.

[Technology] Infinite Loops

I read some advice published by NASA Jet Propulsion Laboratory’s (JPL) Laboratory for Reliable Software that advocated always having a counter on loops that might otherwise depend on some state changing (e.g. like while (set_is_empty (set) == FALSE) { ... }) (their rule #2).

I used to prioritise smaller code.  I still do.  But I'm happier to also write more and more complete code now and do it more carefully.  I'm now willing to take more time to write it in the first place, replete with appropriate error checks, rather than trying to take a lot of shortcuts.  For example, using C versus a higher level language - if I'm hoping for the code to be around for a while, I don't mind the extra time it might take to write it in C.  I used to worry about whether my editor was slowing me down.  Now I think it almost never does - planning out what I'll do takes way more time.

So, back to NASA.  In my thesis code, I started doing that.  I've defined some limits for loops that wouldn't normally have a counter, and when the loop exits, I check to see whether the counter reached the limit, and if it did, I now report an error.

Today, that came in useful, and helped me quickly discover an error in a comparator elsewhere in my code.  (A tree wasn't reliably ordering nodes so a loop trying to clear nodes was repeating continuing infinitely.  Whoops.) 

Problem solved.

[Technology] emacs and vala-mode: set-difference

When using emacs 24.5 and vala-mode, I started getting this error:
"Symbol's function definition is void: set-difference"

After some googling, it turns out to be related to cc-mode using cl (CommonLisp).  The not-ideal workaround for now is to apparently include

(require 'cl)

In your ~/.emacs file.

[General] Fear and Loathing on Facebook

I am a shy person.

Actually, I just have very low self-esteem.

Since grade 4 and before.  I always assume people don't really want me around.  It's because sometimes someone has been around that I didn't want around.  Or because I'd be a witness to someone bearing around that others obviously didn't want them around but the person really wanted to be included.

I am a bit stupid.  I should have focused on being inclusive of the undesirables.  They are still people and have something beautiful to offer, probably.  Sometimes I wanted to be cool, though, so I wouldn't.  I wanted to be desirable.  Why?  Who mattered so much that I had to start watching what I did and what I said so much?

It used to be crippling.  Interacting with most people involve a very explicit façade.  It doesn't really now.  It's a more implicit and ingrained façade I guess.  Sometimes it's not a façade at all.

Of course, I've always treasured the friends I've felt natural with.  My best friend is my best friend because of an important decision I made in high school to be happy and to give up on a group of jerky jerks and to focus on that consistent friend who didn't seem to be concerned so much with what the others thought.

In daily life, I tend to be reluctant to initiate social activity.  I tend to wait to be included by others.  I assume most of my friends aren't super keen on hanging out with me, so it's better not to oblige them by asking, but to wait until they feel like it.  That's probably not entirely true.  I end up very happy when a friend seems consistently interested in my presence, but with the transience of all things, that tends to pass for various reasons (e.g. moving, life changes, etc).  By not proposing things, I remain uncontroversial.

This is also how I approach sharing on facebook.  I share things that are mostly neutral.  They're not about me; I share events that I am helping promote.  I sometimes post questions.  I don't share things about me, because inevitably a portion of people who read it will groan and think I'm stupid.  Some people will decide I'm annoying.

Today I was catching up on VlogBrothers episodes.  I treat them like real friends, in a way!  I get super excited to watch them.  I like their ideas and the way they talk about them.  I wanted to share that.  I wanted to connect with others on that point.  I wanted to share something important to me and connect with others through it.

However, I have friends who don't like them.  There are complaints.  Of course there are.  They're not perfect.  My friends who complain about them are also not perfect, though.

I get tired by how much energy gets devoted to hating things.  Especially among some of my social circles.  They get very political, and people seem to really enjoy hating those they disagree.  In theory, you should care about those you disagree with, because you'd want to help them by helping them resolve some misunderstanding that causes the disagreement.  If you want to make progress with the thing you care about, you'd want to be able to bring on board even those who currently act as obstacles, right?

Anyway, hatred makes me a bit tired.  So, I did share my love for the VlogBrothers, and I also "tipped my hat" to those who dislike them, sincerely offering them a cookie to help deal with the trauma, so hopefully they won't feel the need to get into a fury about it.

So, I found sharing something on facebook really scary.  But I'm going to try to keep sharing things on facebook.  Maybe a thing a day.  Oh my.

[General] Away from Computer, Be Left Back

I am going to try to take a break from everything except my thesis for next 9 weeks. See you in the summer.

[Microblog] Transient Torture

I just cut myself on pasta sauce.

[Microblog] Words

Before there were typos, there were writos.

[General] My everynight

A long walk late at night up a steep hill, down a strange residential street.  Looking back, down at the downtown, I see all the lights.  I see the cathedral and the crane and some of my favourite streets, all so tiny like little models.  The air is clear and crisp and hurts if I breathe too deeply.  The sky is so empty, but if I wait and stare for long enough, it’s really not that empty at all.  It’s noisy, with a stellar static.

I walk up some lonely grey steps, quite a steep hike, and their summit gives the best view of the night.  I can even see towards the east, towards where I lived last winter nearer the edge of town and the river.  I make a guess as to what the river is in the darkness.  I can see the train tracks near my current home.  I could have gotten where I was by riding those rails, but no trains roll by at this hour.  I hear there’s a strike, anyway.

Further ahead there’s a swing set.  I could swing there, and I remember that I already have.  Swinging, a cornerstone of my identity and my history.  Play.  Play sometimes feels heavy to me now, so tied to old memories, but then I can always force myself to do it, just to start.  I should have pushed myself more before, but now’s better than never.  I can’t swing right now, as friends are expecting me.  I shuffle on.

I sing to myself a bit, quietly.  An observer with his cigarette removed temporarily from between his lips comments on my cute fuzzy hat, I can’t make out what he said so I reply with something ambiguous.  He says something more that I can’t make out over the snow’s crunch so I laugh at what might have been a jest and shuffle on.  Meaningless and joyful interactions.

There are pretty houses with pretty snow gardens.  A few still have lights out, really pretty lights, and I praise these hold outs for resisting calendars. I wish every night could be like this.

[General] Microtriumphs

I finally have imposed a system on my Gmail labeling.  It took an hour today, but it has re-organised my "organic" approach that started a decade ago.  And as we know, organic materials rot.

Basically, it meant giving up on many of my more complex plans for labels, and just accepting something simpler.  I have some major groupings, like mailing lists, broken down by social, tech, work, etc, which are then broken down by their source.  I have a people grouping, broken down by family, friends, and other, then broken down by name.  I have alternative inboxes (where filters cause messages to skip my main one), for groups that I'm a part of, and those messages also get filtered into their own "mailboxes" so when I remove them from their custom inbox, they're still neatly grouped.

Simple and useful.

[Budo] Resolve

"You do it simply through the sheer effort of will it takes to be beaten but still show up for practice."
- Kim Taylor

[Photos] My wall is a whiteboard

German Grammar

Crayola Ultra-Clean Washable Markers

[General] Purified memories

You know how over time your memories lose some details and some things you end up idealising?  I think I like that effect.  I think it's generous and good to strip away unflattering details and recall people and events more fondly than they originally warranted.   Well, as long as you don't use the rosefied memory to inform important decisions. :D

[Microblog] The stupidity of Google Music (Play Listen)

I don't understand can look at my entire music collection and then fail so horribly to predict what music I'd like.  They're wrong 9 out of 10 times.

Also, I have no idea why it's so hard to get it to Just Play music.  They keep asking me to narrow things into restrictive boxes.  I just want a system that learns what I might like and introduces me to new stuff.

[School] 100 Days of Thesis: Day 91

I have done a horrible job of blogging about my Masters.  I originally had grand plans.  So, starting today, I will count down the last 100 days of my Masters.  I think I actually have slightly less than that.  Yes, today is day 91. 

[General] Lots of thoughts, no time to articulate them

There is a lot I would like to write about in here. 

How efficiency involves keeping people uncomfortable.  If people are becoming comfortable, there's probably 'waste' in the system that can be stripped away until you find the threshold beyond which enough people start failing that the system collapses.  "Equilibrium."  Sellers generally probably wish they could have greater profit margins and buyers wish they could pay less.

How imbalances, where one person profits at the expense of another, probably can't be sustained; something is getting depleted. In theory, if one supplier is charging more than they "should" (based on their own costs and the spending power of their buyers), people seeking opportunities can correct that by supplying the product at a more reasonable price.  In practise, suppliers collude, even implicitly, and people abuse the difficulty in entering certain markets to help prevent competition and prolong an exploitation.

Some types of imbalance apparently can be maintained for a prolonged period of time, and even increased, though.  Growing income gaps are a good example, where wealth gets consolidated and used to increase consolidation, squeezing more out of more people.  But perhaps that's a push for efficiency.  The situation has changed so that more people can technically survive with less.  People can exist with a lot of debt that they can never reasonably repay and be relatively unemployed and still eat and enjoy some form of shelter.

That's disturbing.  In theory, the more debt you hold, the less free you are.  You can't easily quit your job, because you owe tens of thousands of dollars to your creditors.

Every time a change comes along (e.g. a new technology) which should in theory make people more comfortable, free up their capacity to live life, that capacity gets quickly filled in.  Computers and printers greatly expedite tasks from the 1980s, but rather than enjoy more free time as a result, that freed time was just filled up by other things.  More work got done in the same period of time.  Is that bad or good? 

In theory, my computer should work incredibly smoothly with little resistance in my usage, since processors and memory have increased by an order of magnitude over my youth, but no, my computer is still quite slow, my phone even slower, despite being almost as powerful as my previous computer.  Why?  Why can't people let the increased capacity be and let others relax and enjoy existence a little?  Why do I have to feel frustrated at my phone when it processes 1,600,000,000 operations a second (twice, since it's dual core) but I can't efficiently send a text message - a task I was able to do fine on a phone that was 20 times slower 8 years ago?  Why is 'good, lasting performance' not a sector that people try to fill?  Why is there no market for it?  My phone is 3 years old.  My friend's iPhone is 3 years old.  We have the same complaint.  Our phones fundamentally do the same stuff from 3 years ago, but are we just being pushed via software updates to replace them?

Your allegedly average work week (spend 40 hours at a location working + 7.5 hours of transportation + 2.5 ancillary time lost per week = 50 hours per week) almost seems designed to consume all the time in an employee's week possible without destroying them.  Week days, after 8 hours of sleep, 10 hours of work, and 6 hours of being tired (consumed by chores and need-to-dos), there is almost nothing but to wait for a small, two day break where time is quickly lost to things you should have been doing all week anyway and not wanting to do anything because you're tired.

I feel like having the most productive and efficient system is counter-productive in terms of individual happiness.

[General] MEN ON MARS, the story of Curiosity's odd shadow

This has appeared on my Facebook news feed a few times and it's disappointing that people believe it.  Anyway, people claim the Curiosity rover is either on earth or there are people on Mars, since this photo taken by Curiosity itself, accidentally reveals the shadow of a person maintaining it:

Shadow of a man?

NASA Source

Popular news articles:

• http://www.dailymail.co.uk/news/article-2925341/Life-Mars-New-NASA-photo-shows-workman-fixing-space-Rover-Red-Planet-conspiracy-theorists-claim.html
• http://www.huffingtonpost.com/2015/01/27/mars-rover-shadow-photo_n_6552762.html

Wikipedia article on the Curiosity Rover:

• http://en.wikipedia.org/wiki/Curiosity_%28rover%29

Image of the whole rover (on earth):

Full size

Shadow of the rover's mast head and the arm:

Another:

NASA source

"Shadow of a man" again for comparison:

NASA Source

So, you can still see the "arm" and the "oxygen tank" and the "head" but it doesn't look so much like a person anymore, does it?  Why are people blindly reporting and believing this?

[Technology] Bitrotting brain

TL;DR

• have an encrypted /home
• hard shut down of computer
• wouldn't boot back up to the login screen
• couldn't mount /home from emergency mode
• had to boot a Live USB key and do fsck.ext4 on / (yes, /)
• problems with mounting /home were probably more to do with emergency mode and what gets enabled for different targets via systemd
• I need to update myself on my system's work-abouts :D

Premise

I know a lot about my computer and computers.  There are a lot of people who know a lot more than me.  Relative to the rest of the population, there was a time 3.5 years ago when my knowledge was probably ranked at a higher percentile.

I haven't had the same time and motivation these past few years, though.  One of the largest complications has been my Masters.  Sure, I know a lot more about machine learning, linguistics, natural language processing, etc., but that hasn't helped me really understand my computer much more.  I was fortunate enough to participate in the Google Summer of Code 3 summers in a row, but my time was always tight for that, and full-use was probably not made of those opportunities working on GNOME.

However, today was a harsh wake-up call about the consequences of idleness.  My knowledge of init and rc and Linux's old boot sequence is still fairly strong, having grown up in Slackware on the command-line.  However, while I've read about evolving configurations and utilities since, I haven't gotten much practise in them. 

And so, it was with horror when I realised that my system would not fully boot today and I could not immediately address the issue.

Today's Predicament

My /home directory is an ext4 filesystem on an encrypted LUKS partition.  Perhaps you can immediately see what is going to go wrong.  Earlier today, my computer froze up.  That's not uncommon.  As much as I love Linux, I am never surprised when it spontaneously freezes (especially on a laptop (what with having its Internet connection and video mode and power supply changing constantly)).  After trying to get it to respond for 15s, I just held down the power button as usual and then turned it back on.

Plymouth came up, I saw the fedora logo start to fill, it was interrupted by the password prompt for my /home partition, I entered the password, and then the fedora logo filled up and - nothing.  It just waited there.  It should switch to gdm's login screen, but it wouldn't.  I could go see the terminal in the background, but I couldn't access any shells.  Just the start up log.  There was nothing obviously wrong.

Sometimes I would see this message, "EXT4-fs error (device dm-0): ext4_mb_generate_buddy:757: group 83, block bitmap and bg descriptor inconsistent: 21682 vs 21673 free clusers", but dm-0 is my root directory.  dm-3 is my home directory.

I tried to boot into rescue mode to see if I could still access all my files.  No.  After supplying my password for decryption, I would see this error message "device-mapper: table: 253:3: crypt: unknown target type
Failed to activate: Invalid argument".

That sounds bad.  I entered the emergency mode shell, and checked the journal.  This is something I only do occasionally using journalctl, as I used to directly check logs or dmesg.  Uh oh.  Here it was failing to mount /home, and it was failing at the step of "/usr/lib/systemd/systemd-cryptsetup attach luks-1496...e527 none" [LUKS name contracted].  Uh oh.

At this point I started to suspect that my encrypted partition was corrupted from my hard shut-down.  There's not much else I can do with my computer when it freezes up, but I haven't suffered from it being left in a volatile, at least until now.

I tried to manually mount it, but then realised I have almost never manually mounted a file system encrypted using LUKS.  I've used encfs, but - uh, now what?  Also, lvm2 and systemd I'm only passingly familiar with.  How am I supposed to be confident in my computer's reliability if I don't even understand the fundamental tools that house my data any more?

I double-checked that my back-up was still working (I've had back-up hard drives die on me before!) using another old computer and thankfully it was OK.  However, I didn't want to have to back-up from it if I didn't have to, as I had done work in the past 24 hours that I didn't want to lose, damn it.  (I'm so glad that I've gotten the knack of backing up every 24 hours at least!).  I also wasn't completely convinced that my data was inaccessible.  On plain text file systems, it's easy to grep or scan to reclaim many/most files, even if the file system itself is corrupted or the partition boundaries have been lost.  You can't readily do that when the file system is encrypted, but perhaps there was a way to repair it?

Also, this made me wonder, why was the default multi-user graphical mode stalling out?  If it couldn't mount /home, it should still be able to display the login screen.  I don't need a /home for that.  I tried to launch gdm from emergency mode and couldn't because of failed dependencies, one of which was for binary formats.  That seemed weird.  I didn't remember so many things failing to start back when I was booting in the normal way.  So I tried the normal way again and ... it claimed that /home mounted cleanly and even gave a plausible count of files.  (!)

I then decided to boot in single-user mode (edit the grub command for the kernel I'm booting into and add 'single' on the kernel line).  This took me through the same process as usual, which seemed to mount /home correctly, but instead of going all the way to the point where it was stalling, it stops successfully before at a command-line for root.  Hooray!

Once there, I poked around.  I couldn't immediately tell why from emergency mode I couldn't successfully mount my file systems, but there's an egregious lack of familiarity in me regarding how systemd, lvm2, and luks interact, so that's no big surprise.  I did a passive run of fsck.ext4 on my / directory (remember how dm-0 had errors reported earlier?) and yes, there seemed to be quite a lot.  I grabbed my Fedora live USB key from my drawer, re-booted into it, ran fsck on my computer's root partition, and let it fix all the errors.  This always alarms me, as I never know what data has been lost on my file system to lead to the errors in question.  There's never a guarantee that cleaning up after the errors will resolve the problem in question.

Anyway, I rebooted, I went through a normal boot and - tada, the login screen.

Conclusion

So why did I write all that?  In part to motivate me to learn more about the current state of my system to regain the glory of self-sufficiency.  And in part so the few error messages I encountered that another in my case might encounter will be documented, and searchable. :D

[Microblog] Tradition

I've lived 9 years in this town and I've barely walked its streets at all.

[General] Writing

I thought of something beautiful to write today.

And now I've forgotten it.

I've had a lot of opportunities in the past few years that I might not have had.  That's an obvious sentence.  You can never fully appreciate what else could have been, since it wasn't.  It's like if something horrible happens, and you wish you could undo it, it's a bit unsatisfying to know that if you could undo it, nobody would appreciate the difference, because the worse outcome wouldn't have been experienced. 

I like the odd anime.

My Little Monster has a title that reminds me of a cartoon I watched as a kid, My Pet Monster.  It was quite different and quite the same.  It has some lessons that I've avoided, because I'm told they're bad.  Perseverance is one of them.

Are the best things in life simple and obvious?

I'm sitting in a splendid room.  Decorated with bed sheets, predominantly white in colour, all the better to glow under a black light.  There are glow-in-the-dark stars on the wall.  A bright orange salt lamp.  Fake clouds atop the bookshelf.  A comfy futon under me. 

I suffocate the life around me.

[General] Neu

Years and calendars are relatively arbitrary (at least they align with physical seasons more or less), but I suppose they are useful structure.

What I've accomplished in 2014

• escaped crippling depression
• escaped crippling poverty
• took a break from a crippling Masters
• worked well, diligently, productively and regained self-respect and self-confidence
• improved my German greatly (hooray Duolingo)
• worked on photography
• managed to finally upgrade to a Micro Four Thirds (from a camera phone), wrote some handy scripts for my workflow, managed to mostly share an average of 40 photos/week.  (Still haven't managed to integrate that stream into this site well :D)
• worked on budo
• learned a long-sword school, Kage Ryu
• improved two-sword Niten Ichi Ryu
• attained the rank of Nidan in ZNKR Iaido
• managed to make it to 1/3 of kendo practises (goal was 1/2, oh well)
• learned the jo side of Ran Ai
• grew my social video gaming :D (Wii U + Mario Kart 8 + Super Smash Bros + Hyrule Warriors), managed to play a fair amount with my best friend who is in another province
• caught up with anime (mostly through a circle of friends and regular night)
• found someone to teach me violin in exchange for cookies :D
• got my father out to Victoria to see his daughters
• went hiking, camping, and canoeing a lot
• improved my independence - no longer feel dependent on others!
• even have a basic understanding of what it is to "love" oneself
• started drawing way more often (work on the teddy bear web comic is underway!)
• made my own kitchen knife (!) in a FORGE and my own pants and top (for iaido) from 'scratch'
• expanded my baking repertoire (birthday cake, zucchini bread)
• started my own company
• negotiated a compressed work week contract :D
• managed to help my dad out a lot (zoom zoom)
• exercised!  Goal was daily, managed every-other-daily, still good results, especially in cycling up Gordon St. workdaily
• bike maintenance!
• journaled privately more often (that is, not even online :D, so totally frank and no filters), worked on improving own information organisation
• made some cool, new friends
• made a neat web app (My Daily) but didn't publish it yet :D
• went tobogganing
• organised a booth for Guelph VegFest 2014 and a large vegan potluck on campus :D
• explored a sewer and dumpster dove
• traveled (Victoria/Vancouver, Timmins, Calgary)

Where I've failed

• almost no relaxing downtime :|
• too busy a lot of the time, had to drop a contract even!
• managed to read just one book a "semester"
• didn't really do any of the Coursera courses I signed up for
• had two periods of notable stress (which I ended up catching)
• accomplished very little Open Source development
• failed to grow much as far as veganism goes (I need to get more familiar with more facts so I can make better arguments and retire bad ones)
• eating well - ended up spending more eating out just to make sure I had some complete meals, due to busyness
• got 80% of My Daily done, but haven't finished the remaining 20% :D

Goals for 2014

• work on integrating information
• I want photos to flow through this site
• I want my "microblog" posts to appear where they'll be read (Twitter) and where I live (here, kosmokaryote)
• want a private 'wiki' offline that I can store more facts about my own life
• practise violin weekly
• practise piano weekly
• start learning French
• further improve German
• reserve more time for me
• finish my Masters
• read more
• including about budo and veganism
• cook for myself, cook more healthily
• get out of Guelph
• improve on all the other stuff I've been doing :D