Inscrutable f-spot error messages

A new one today. I got this while trying to straighten an image:

Trying to straighten a jpegIt didn’t take me long to realise that any edit operation was producing this. Puzzling though, because I don’t use RAW. Just JPEGS. So that looked like a blushing herring.

A look at the debug output showed:

[1 Debug 13:38:11.221] open uri = file:///jpegs/2011/07/23/Durham%20Parkrun%20%23beta1%20-%2023%20Jul%202011%20-%201st%20Trial%20Run%20--%20Geoff%20Davis%20--%20Anna%20Seeley%20--%20Susan%20Davis%20--%20Angela%20Proctor%20--%20Sat%2023%20Jul%202011%2008-33-09%20BST.jpg
[1 Debug 13:38:12.469] FSpot.Editors.TiltEditor can be applied? True
[1 Debug 13:38:15.790] FSpot.Editors.TiltEditor can be applied? True
[1 Debug 13:38:15.802] open uri = file:///jpegs/2011/07/23/Durham%20Parkrun%20%23beta1%20-%2023%20Jul%202011%20-%201st%20Trial%20Run%20--%20Geoff%20Davis%20--%20Anna%20Seeley%20--%20Susan%20Davis%20--%20Angela%20Proctor%20--%20Sat%2023%20Jul%202011%2008-33-09%20BST.jpg
[1 Warn  13:38:16.733] Caught an exception - System.ApplicationException: URI could not be converted to local file location (in `Hyena')
  at Hyena.SafeUri.UriToFilename (System.String uri) [0x00000] in <filename unknown>:0 
  at Hyena.SafeUri.get_AbsolutePath () [0x00000] in <filename unknown>:0 
  at Hyena.SafeUri.get_LocalPath () [0x00000] in <filename unknown>:0 
  at PixbufUtils.SaveToSuitableFormat (Hyena.SafeUri destination, Gdk.Pixbuf pixbuf, UInt32 jpeg_quality) [0x00000] in <filename unknown>:0 
  at PixbufUtils.CreateDerivedVersion (Hyena.SafeUri source, Hyena.SafeUri destination, UInt32 jpeg_quality, Gdk.Pixbuf pixbuf) [0x00000] in <filename unknown>:0 
  at FSpot.Photo.SaveVersion (Gdk.Pixbuf buffer, Boolean create_version) [0x00000] in <filename unknown>:0 
[1 Warn  13:38:16.766] Caught an exception - System.ApplicationException: URI could not be converted to local file location (in `Hyena')
  at Hyena.SafeUri.UriToFilename (System.String uri) [0x00000] in <filename unknown>:0 
  at Hyena.SafeUri.get_AbsolutePath () [0x00000] in <filename unknown>:0 
  at Hyena.SafeUri.get_LocalPath () [0x00000] in <filename unknown>:0 
  at PixbufUtils.SaveToSuitableFormat (Hyena.SafeUri destination, Gdk.Pixbuf pixbuf, UInt32 jpeg_quality) [0x00000] in <filename unknown>:0 
  at PixbufUtils.CreateDerivedVersion (Hyena.SafeUri source, Hyena.SafeUri destination, UInt32 jpeg_quality, Gdk.Pixbuf pixbuf) [0x00000] in <filename unknown>:0 
  at FSpot.Photo.SaveVersion (Gdk.Pixbuf buffer, Boolean create_version) [0x00000] in <filename unknown>:0

 

Hmmm, that wasn’t helping much. A bit of pasting of random bits into Google didn’t help much either. Hyenas?

It kinda shouted filename problem. Permissions perhaps? Nope, fine. So weird filename perhaps? Yup.

Hash(#) in the filename. Rename the file and all is well.

Wireless not enabled automatically on Acer Aspire One

The first problem, or niggle really, is the wireless. When I boot the netbook wireless is not enabled automatically. It’s a simple matter to right-click the network manager applet and select Enable Wireless, then everything seems fine. Just annoying.

So of the the University of Google to find who else shares my pain. Before long I discover that this might be an Acer thing (none of my other PCs have this problem). Like many problems there’s often lots written and complex solutions. I don’t want complicated solutions, I just want the loose-wire solution. Press a button and it’s all lovely.

Sure enough, a post on the Linux Mint forums described exactly what I’m experiencing. There’s another forum post that describes one fix, but I ended up following the hints described in this thread.

Sometimes I like to understand what’s going on under the bonnet, sometimes I try and understand what’s going on, and sometimes I just copy and paste. Like so …

dougie@barra ~ $ rfkill list all
0: acer-wireless: Wireless LAN
    Soft blocked: yes
    Hard blocked: no
1: phy0: Wireless LAN
    Soft blocked: no
    Hard blocked: no
dougie@barra ~ $

and …

dougie@barra ~ $ sudo rmmod -f acer-wmi
dougie@barra ~ $ sudo rfkill unblock all
dougie@barra ~ $ rfkill list all
1: phy0: Wireless LAN
    Soft blocked: no
    Hard blocked: no
dougie@barra ~ $

As soon as I ran the rmmod line the wireless suddenly sprang into life.

The next trick is to get this to happen automatically on boot, which is pretty straightforwards. Just add the lines to the end of /etc/rc.local as explained in post 12 of this thread.

a Mint new Acer Aspire One 753 (see what I did there?)

It was a last minute decision to go for a netbook instead of a laptop. The netbook has always appealed, its portability, its battery life. Trouble is, they’ve always seemed a bit rubbish. It was Mr Amazon doing his (“I see you’ve been looking at this, are you sure you don’t want that?) routine that made me suddenly consider going for the Acer Aspire One 753.

It was over a nice pint (pl. pints.) of Landlord at the last Nelug meeting that Richard assured me I’d be able to dual-boot Windows 7 with Linux Mint on this netbook. The bar was closing and the landlord decided that the netbook was the right decision and the clicky clicky thing was done.

So how do you dual boot Linux Mint with Windows 7 on a netbook? I hadn’t really thought this through. I was apprehensive because the netbook arrived with negligible documentation. There’s no CD/DVD, because there’s no CD drive. So what happens if you need to re-install/recover Windows 7? Dunno. Presumably there’s a mystery recovery partition on the hard-drive that is available via an inscrutable key-press on boot. Let’s hope I don’t nuke it trying to set-up a dual-boot Linux install.

So how do you install Linux Mint alongside Windows 7 on an Acer Aspire One 753? Turns out it’s pretty easy. I checked out a lot of websites but in the end the answer was pretty simple. There’s probably a few ways of doing it but I followed a post on the Linux Mint forums that pointed to Unetbootin, an inscrutable acronym that has netboot, usb and, er, me, or you, in there somewhere. It was an absolute breeze. I ran the executable on a Windows7 PC, shoved in a USB drive, and it did stuff. And then I had a bootable Linux USB drive that I put into the Netbook. Booting Linux Mint from that offered me the option of shrinking my Windows7 installation to make space for a side-by-side Linux install.

That was the scary bit. Watching my Windows being shrunk. Or not watching, as clicking on Continue (or whatever the button was), resulted in a Zero Feedback Situation. What the hell was it doing? Anything? No spinning egg-timers? No progress reports? It was all good though, even the slightly worrying disk-check that Windows7 was kinda keen on doing on reboot as it sensed something was up, and its domain over the netbook had been weakened.

So right now I have Windows7 and Linux Mint 10 dual-booting on the Acer Aspire One 753, and dead easy it was too.

There are a few niggles, that will need sorted in the days ahead. There’s the usual repository shenanigans that seems to go hand in hand with Linux Mint that seems to come and go and I’ve stopped worrying about. This sort of stuff …

Something wicked happened resolving 'packages.linuxmint.com:http' (-5 - No address associated with hostname)
93% [Connecting to packages.linuxmint.com] [Connecting to archive.canonical.com]93% [Waiting for headers] [Connecting to archive.canonical.com] [Connecting to 

and this …

http://archive.canonical.com maverick Release.gpg
  Something wicked happened resolving 'archive.canonical.com:http' (-5 - No address associated with hostname)
99% [Connecting to archive.canonical.com] [Connecting to packages.medibuntu.org]                                                                                Ign 

This sort of stuff seems to come and go. I get it on my other machines and the less I try to fix it and the more I try to ignore it in the hope that it will go away, the more it usually does.

There’s still a long way to go to get things the way I want but so far so good.

Next task is to figure out why it always boots up with wireless disabled. Windows 7 has it enabled by default.

renaming jpegs based on f-spot tags

f-spot does not have an option for renaming files, so if you want to rename your images you need to export them from f-spot, rename them, then re-import. It sounds a hassle but actually it’s quite easy to streamline, especially since it’s possible just to drag and drop the images you’re interested in. The Drag’n’Drop ability also gets around the problem where really long filenames can’t be exported because of their filename length.

I rename my jpegs based on the the tags I’ve created in f-spot. I don’t know how f-spot chooses the order to write the tags to the EXIF fields, and I don’t much care. It’s sufficiently robust for me, and it means I can rapidly locate images by using the ‘locate’ command.

Here’s the script I use:

#!/bin/bash
#
#       Script: rename_fspot_jpegs
#       Notes:  Renames jpegs based on their embedded tags
#               
#               Modify the BASEDIR variable as required
#



scriptname="$(basename $0)"

# modify this section as required for your setup. 
# you should be able to just change the BASEDIR to your own home directory,
# and export your jpegs from f-spot into jpegs_in.
BASEDIR=/home/dougie
INDIR=${BASEDIR}/jpegs_in
OUTDIR=${BASEDIR}/jpegs_out

# Change this to whatever you want to separate the tags in your filenames
TAGSEP="--"

mkdir -p ${OUTDIR} || exit 2
echo "STARTING script: ${scriptname}"

# process jpegs and extract tags
echo "Process file list for renaming ..."
cd ${INDIR} || exit 2

ls | while read fname 
do
# END each filename with datestamp
        echo "Processing ${fname}"
        exif_created=`exiftool -overwrite_original -d %c -CreateDate "${fname}" | sed 's///-/g' | sed 's/^.*: //' | sed 's/:/-/g'`
        if [[ -z ${exif_created} ]] ; then
# Create date not found. Using original
                exif_created=`exiftool -overwrite_original -d %c -DateTimeOriginal "${fname}" | sed 's/^.*: //' | sed 's/:/-/g'`
        fi

#       Copy the contents of Description field into UserComment field.
#       UserComment field often ends up garbled and appears to be used
#       by some gallery sites - e.g. zenfolio.
#
        exif_description=`exiftool -overwrite_original -Description "${fname}" | sed 's/^.*: //'`
        exiftool -overwrite_original -UserComment="$exif_description" "${fname}"
# f-spot writes its tags to the Subject field.

        exif_tags=`exiftool -overwrite_original -Subject "${fname}" | sed 's/^.*: //' | sed "s/,/ ${TAGSEP}/g"`
        echo "Tags: ${exif_tags}"
        if [[ ! -z "${exif_tags}" ]] ; then
                newfname="${exif_tags} ${TAGSEP} ${exif_created}"
        else
                newfname="${exif_created}"
        fi

# we need the --backup=numbered option in case we have duplicate filenames
        cp "${fname}" "${OUTDIR}/${newfname}" --backup=numbered
done

# now we have a directory full of filenames with no .jpg suffix
cd ${OUTDIR} || exit 2
ls | while read fname
do
        mv "${fname}" "${fname}".jpg
done

# EOF: rename_fspot_jpegs

f-spot, exiftool, exiv2, and exif header weirdness

I decided to revisit an old bug that I logged about a problem I have with f-spot handling of exif headers. Now I’m more confused than ever. I can see what’s happening, but not how or why.

The problem I’m having is that when I upload some photos to gallery websites the handling of exif data doesn’t appear consistent. For example, have a look at this photo. If you look at the right hand side of the screen where the photo title is shown it also shows the caption. It looks like this:

From Zenfolio gallery

Now have a look at the same photo uploaded to picasaweb (the sizes are slightly different – I think picasaweb changed them on the fly when I uploaded via google plus). In this case there is no garbled text and picasaweb has used the exif description field for the photo caption. i.e:

from picasaweb

So what’s happening here? Well, here’s what I think is happening. Picasaweb is extracting the caption from the exif field Description, and Zenfolio is extracting the caption from the field UserComment. Here’s what I get if I run exiftool on the image file (you might need to click on the image to see it properly):

exiftool -UserComment -Description test1.jpg

As you can see the UserComment field contains garbled characters.

A lot of my photos have this and the inconsistencies can cause me headaches. I decided on a brute-force scan of my photo collection to address the issue. I thought, why not just duplicate the Description field into the UserComment field? That way, whatever package or gallery reads the image file will probably get the caption. So here’s the script I used:

#!/bin/bash

#
#       Process all jpegs and overwrite UserComment field with Description
#       field where they differ.
#
find /jpegs -type f -iname '*.jpg' | while read fname
do
        field_usercomment=$(exiftool -UserComment "${fname}" | sed 's/^.*: //')
        field_description=$(exiftool -Description "${fname}" | sed 's/^.*: //')

        if [[ "${field_usercomment}" != "${field_description}" ]] ; then

                echo "$fname"
                echo "User Comment: ${field_usercomment}"
                echo "Description: ${field_description}"
                echo

                echo "${fname}"
                exiftool -overwrite_original -UserComment="${field_description}" "${fname}"
                echo
        fi
done

After a satisfying hour or two my photos all looked a bit tidier. So now I have a cludge – something that will get round the problem. But I’d really like to treat the cause, and not the symptom. So what’s the cause?

My first guess has always been the camera. I have a knackered old Nikon Coolpix that gives me problems. But I see this problem with other cameras too, my Canon DSLR and compact camera have acquired the garbled field in some pictures too.

So I turn to f-spot, the photo-management software I use in Linux Mint. I checked the exiftool output from a test file, then imported it into f-spot. I then added a tag so that f-spot would write out the exif data (I have f-spot configured to Store Tags and Description inside image files where possible) then had a look at the image file. Here’s what I got:

exiftool output

The output from the first invocation of exiftool is on the jpeg before importing into f-spot. The second example shows the output of exiftool run on the file after it has been imported into f-spot. It looks like f-spot does something to the image that mangles the UserComment field.

At this point I assumed (erroneously I think) that f-spot must be storing the UserComment and Description fields in its sqlite database. Using sqlite3 on the command line I had a look at the structure of the photos table:

dougie@phoenix ~ $ echo '.schema photos' | sqlite3 .config/f-spot/photos.db
CREATE TABLE photos (
    id            INTEGER PRIMARY KEY AUTOINCREMENT NOT NULL,
    time            INTEGER NOT NULL,
    base_uri        STRING NOT NULL,
    filename        STRING NOT NULL,
    description        TEXT NOT NULL,
    roll_id            INTEGER NOT NULL,
    default_version_id    INTEGER NOT NULL,
    rating            INTEGER NULL
);
dougie@phoenix ~ $

There’s the description field, but no UserComment field. But it looks to me that f-spot is certainly writing to the UserComment field when it updates an image file. Perhaps it’s dependent on the type of contents of the Description field. It doesn’t, however, make any difference as far as I can tell what the contents of the Description field are. In this example the text ends with an exclamation mark, but I can’t detect any pattern in the text I put in Description fields that might provide a clue.

So if f-spot isn’t storing the UserComment field directly what, when, and why is it writing to the field in the image file? It seems that here we enter the murky world of XMP and I rapidly lose the tenuous grasp I had on the underlying technology up to this point.

Back in 2008 there was a discussion on the f-spot mailing list about how f-spot stores its data in the image file. Re-reading that thread it sounds as if f-spot writes its tags to the XMP metadata and not all utilities are able to read that. This was 2008 mind, and f-spot has jumped a few versions since then.

Out of the same thread came a hint on using exiv2. This is a tool I use a lot, especially for adjusting ‘date taken’ timestamps, but hadn’t found it any use for extracting the UserComment field. It turns out I hadn’t RTFM properly and it is possible if you tell exiv2 to look at XMP data. This leads to some interesting output. Here is the example file again, using both exiv2 and exiftool to examine the field contents:

dougie@phoenix ~ $
dougie@phoenix ~ $ exiftool -UserComment -Description /jpegs/2011/07/17/test1.jpg
User Comment                    : 桷⁹畲污湯⁧桴⁥敢捡⁨桷湥礠畯挠湡挠瑵琠敨挠牯敮ⅲ††††††††††††
Description                     : why run along the beach when you can cut the corner!
dougie@phoenix ~ $
dougie@phoenix ~ $ exiv2 -p a /jpegs/2011/07/17/test1.jpg
Exif.Image.ImageDescription                  Ascii      77  why run along the beach when you can cut the corner!                        
Exif.Image.Software                          Ascii      21  f-spot version 0.8.0
Exif.Image.ExifTag                           Long        1  162
Exif.Photo.DateTimeOriginal                  Ascii      20  2011:07:17 09:37:13
Exif.Photo.UserComment                       Undefined  84  why run along the beach when you can cut the corner!                        
Exif.Image.GPSTag                            Long        1  296
Exif.GPSInfo.GPSVersionID                    Byte        4  2.0.0.0
Exif.GPSInfo.GPSLatitudeRef                  Ascii       2  North
Exif.GPSInfo.GPSLatitude                     Rational    3  55deg 32' 25.410"
Exif.GPSInfo.GPSLongitudeRef                 Ascii       2  West
Exif.GPSInfo.GPSLongitude                    Rational    3  1deg 37' 58.960"
Exif.GPSInfo.GPSAltitudeRef                  Byte        1  Above sea level
Exif.GPSInfo.GPSAltitude                     Rational    1  9.5 m
Exif.GPSInfo.GPSTimeStamp                    SRational   3  09:37:13
Exif.GPSInfo.GPSMapDatum                     Ascii       7  WGS-84
Exif.GPSInfo.GPSDateStamp                    Ascii      11  2011:07:17
Xmp.xmp.CreateDate                           XmpText    19  17/07/2011 09:37:13
Xmp.xmp.Rating                               XmpText     1  0
Xmp.xmp.CreatorTool                          XmpText    20  f-spot version 0.8.0
Xmp.dc.description                           LangAlt     1  lang="x-default" why run along the beach when you can cut the corner!                        
Xmp.dc.subject                               XmpBag      2  Dougie Nisbet, Coastal Run - 2011
Xmp.exif.UserComment                         LangAlt     1  lang="x-default" why run along the beach when you can cut the corner!                        
dougie@phoenix ~ $

Interestingly, exiv2 is displaying the UserComment data in clear text, in two places, which I’m guessing must be something to do with the different types of metadata – XMP and EXIF.

So does this mean this has nothing to do with f-spot at all? If exiv2 is able to read the text, perhaps it’s something to do with the utility used to examine the image file? If I have a look at the file using geeqie and open up the EXIF window geeqie seems to understand the text too:

geeqie EXIF window

Perhaps this is an unrelated setting I need to have a look at. Language, Locale, who knows. The issue seems to crop up in different places so it may very well be unrelated to f-spot. For the time being I have a fairly straightforward work-around, I just need to remember to run a script to copy the description field into the UserComment field for any images I update in f-spot. It’s a bit of a nuisance but fairly easy to automate, and a lot less time-consuming than trying to get to the bottom of the mystery!

The Katsura Tree

As the title of this blog might suggest, I have a particular fondness for the Katsura tree (Cercidiphyllum japonicum). I have three in the garden, all youngsters. One is getting on for 2 or 3 metres tall, has been in for a few years (planted Nov 2007), and is looking good.

Cercidiphyllum japonicum

Healthy tree

Around the corner in the front garden there are a couple of tiny trees that I snapped up for a fiver each from Dawyck a couple of years ago. They are about 3 or 4 metres apart and have been absolutely fine since the day they were put in. I occasionally mulch with grass clippings or clear encroaching roses or beech from them, but all, in all, they’ve seemed happy.

But something has happened recently. The one on the right (click for a bigger picture) looks pretty healthy. Not a big tree but happy enough.

 

And then I walk about 3 metres along the flower bed to the next one. And it doesn’t look well at all.

Cercidiphyllum japonicum

Not a healthy tree

The mush you can see around the base is old grass clippings (no fertiliser, or weed and feed), and it is kept away from the base of the sapling. Same for the healthy one.

A closer look at the leaves shows them brown and mostly dead.

So, what is wrong with this picture? Why is one alive and apparently healthy, and one is dead, or nearly so? They are only a few metres apart and have no visible differences in light, soil or moisture.