Some Org mode workflow and graphs
@ Adrian Tritschler · Monday, Aug 14, 2017 · 4 minute read · Update at Aug 14, 2017 ·

Over the years far too many items catch my eye and have gone into the “one day/some day/maybe/perhaps”. Capturing them in Org mode was a major advance, at least then I had a record, but working through them is still a weak point. Some of them predate my use of Org mode, coming from bookmarks accumulated in, feedly, pinboard and manual links from prehistoric times.

Particularly problematic is an ever-growing (virtual) pile of PDF files, text files, EPUB books and bookmarked links, together with a fire-hose of RSS feeds. The latter is causing me some grief at the moment; the combination of org-feed pulling in feeds I’m interested, FOMO preventing me from being ruthless and culling them, and the website configuration of some sites causing org-feed to add a complete new copy of the last n sites every time it is run has conspired to cause my file to exceed 10MB – 105000 lines of text and 7690 items!

To try and get my head around what was piled up in the background I wrote a few snippets of shell, then decided to run them daily and graph the results. First up was the incoming bookmarks in, mostly captured from websites that have popped up during daily browsing. For some years I’d been using a capture template that pulled in items into the file as below:

* 2017
** 2017-08 August
*** 2017-08-02 Wednesday
**** [[][Books - A Showcase of Sketchnotes]]
    :DateCreated: <2017-08-02 Wed 18:10>
    :KEYWORDS: sketchnote
    :ID:       015b9f93-fadf-44e7-ae47-039715724cc6
     Source: [2017-08-02 Wed]

Then a grep "^\*"|wc -l give me a very quick and dirty count of items.

As soon as I had the current numbers I started kicking myself, wanting the historic values, so I reprocessed my git history:

for m in `seq 7`; do
  for d in `seq -w 28`; do
    (echo 2017-0${m}-${d};
      git show HEAD@{2017-0${m}-${d}} |\
      grep "^\*\+ ${n}" |wc -l)\
      |paste -d "," - - | tee -a stats.csv

Followed by a quick gnuplot template and I’ve got a graph of how high the pile of unread bookmarks is:


I put this into the daily cron job and let it run for a few days, then decided that I didn’t like having the year, month and day headings being counted so I’ve changed the capture template to add captured items as a “MAYBE” rather than a bare note, and a quick edit of the existing file to convert the existing bookmarks to MAYBE got me the sudden drop.

Some similar greps and and graphs and there’s a graph of the items waiting from my RSS feeds – clearly revealing that I’ve got a deal of housekeeping here.


As stated above, the RSS list has grown enormous and contains a mass of duplicated items. The drop from around 9000 to 7000 in mid-May was when I found that a previously behaving podcast feed had taken to republishing, daily, that the last 100 items were all “new” and fooled org-feed-update into adding each item again and again. More sorting and culling is required, when I get around to that there’ll be some similar drops in the item counts.

From there I decided to take a look at the breakdown of all tasks in all my org files; TODO, MAYBE, WAITING, STARTED… DONE. A slightly more complex bit of shell, grepping for "^\*\+ ${n}" where n is the keyword, piped out through wc(1) and munged into a CSV file. The numbers looked OK again I reprocessed my git history to find what it had looked like for the year to date.

To generate the graph took a few minutes of searching once I knew what I wanted and the correct terminology – stacked graph – to use. Courtesy of I now have a nice graph showing the stack of my TODO/MAYBE/STARTED..DONE tasks. et voila, we have:


The overall sawtooth shape is caused by me archiving off all completed work at the start of each month, the very large number of items in TODO and MAYBE is due to the presence of a number of extensive lists of items – eg a blog’s worth of articles turned into TODO items, or a list of someone’s “100 favourite books” – all present as low priority reading if I find sufficient time and interest.

In summary, I certainly found that visualising the changes in my captures, my RSS feeds and the state of my overall Org mode life through the year so far has given me some insight in how I’m using the tools.


  • [2017-08-17 Thu]: while removing vast numbers of duplicate entries in one of my RSS feeds I discovered that the overall graph hadn’t included the CANCELLED items so I added it to the statistics script and graph template, then reprocessed my git history to extract it as well. Two huge blocks of CANCELLED items show where I did some manual cleaning on the
  • [2018-05-03 Thu]: I’ve collecting the same stats and generating the graphs into 2018, today I decided to to show them off.

Adrian Tritschler's stuff
My website, an agglomerative mess, probably half-eaten by a gru

1960s 2016 2017 2018 2019 2020 2021 250cc 30daysofbiking 3pbs 3rrr 403 404 4wd 9-11 a1000 a4000 aaac aabill aac abbotsford act additives adsl adventure advertising afl agf agl airline airport ajft alley amazon ambulance amiga amiga1000 amiga4000 amys-ride angkor-wat anniversary antelope anzac apollo apple aps archicentre architecture armour aroundtuit art artwork ascension-records asus atbiad atomic attic au audax aus auspost australia australia-post autoconfiguration autumn avatar avebury baaw baboon baby backup ballarat balloon bandcamp bank barbecue bath bbq beach beer beijing benchmark benwerrin beryl bialetti bicycle bicycle-lane bigpond bigride bike-touring bikelane bikepath bikevic bingo bionicle bird birding birdingathome birds birthday bitcoin bite blackout blade blog blogmax blogx blondie blossom blosxom bluestone bluetooth bmw bnsw boardgame boat boobs book bookmark books bouncy-castle bp breakdown breakfast brewery bridge bridge-road briefcase bright broadband brolga bromeliad brothel bsd buddy bugger-grips builder bulky bullant bund bungendore bureaucracy bus bus-lane bushwalking butterfly buzzword bv cactus cafe cake calculator callistemon cambodia camera cameron camping cannondale canon cappuccino cappucino car carnegie casio cat caterpiller cbx cbx750 cbx750f cd cemetery censorship centro cereal chadstone challenge change chef childhood chilli china chn chocolate christening christmas church cicada cider cigarettebutts civet clayton cm cms cockatoo coco coffee coffeemug coffeeneuring coffeeneuring2020 coffeepot cold collision colnago comedy comet commonwealth-games commute commuting compiz computer concrete confusion connex coral corn couch court coworker crab crawl cremorne crime critical-mass crossstitch crumpler css ctc ctw2850s curry custard customer-service cycle-path cycle-touring cycling cycliq cyclist cyclops dalek dandenongs date death debian demo demo-scene demolition dentist derailment desk developer development dial-up dig digitisation digitise dinner disaster disgusting disk disqus distributed dizzy djerring-trail djerringtrail docbook dog dogs door dooring dorset dragon dragster dream driving drumming dtt duck dungeon durrell dvd dvico dyndns e-text e18 easyshare echidna edge edge305 edge705 egret electronics elephant elevenses emacs email endomondo engagement england english engrish environment eric-newby error erskine-river esp estuary evening excom exif exiftool f3jr f3jv facebook fafnir fail2ban failure falcon family fancy fancy-dress fault feet fence festival fiat fiat-500 fiat500 fidel filter fire firefox firmware fish fishandchips fitbit fitzroy fixie flex flickr float flood floriade flower fly12 foaf fog food footy for-sale fotothing fountain fra fragile france freebsd friday friendship frog frogmouth frost froty fruit fruittoast fungi gadabunud gadubanud gang-gang gansu garbage garden gardening garfish gariwerd garmin gas gbr geelong genres geocache geocaching geography geotagged getoffmylawn gibberish gig gippsland giraffe girlfriend glass glen-waverley gmail gnome gnuplot goanna goat goatbeer golang goodreads goofey google gor gorse government gplus gps gpx grammar gravelroad grddl greatsouthernrailtrail grenda gru grub gsrt gsx1000z gtd guinness guns gunzel gvbr gyroscope h1n1 habit hail hardware harvest hat hdtv heatwave helicopter heraldsun hfbv hide hiking hippo history holiday honda honeymoon hope hospital hot hotel hrb hsfff huffy hug hughesdale hugo hume-highway humour huntingdale hyundai ibis ibm icon identity idiot ikea imagemagick imap incentivise incompetence indieweb inferno injury ink inktob2020 inktober inktober2020 insect instagram intel interest internode intrepidtravel invitation iot iphone ipod iptc ipv6 iso8601 issue itch itsp itunes ixus ixus300 ixus700 jaywalker jekyll jersey jey journal jrb june justice justjoey kangaroo katana kawasaki kay-and-burton kde kellybrook kings-couriers kneejerk knitting kodak kookaburra kubernetes kudu ladder lake laneway laneways laptop lastfm lawyer leather lego lemon lemons leslie-charteris lifelogging light lighthouse lilo limar limes linkrot linux lion litter lizard location lockdown logrotate lorne lroty lufthansa lunch lunchbreak lvm lxra lysterfield macciato machine_tags machinery magazine maglev magnolia magpie map mapping marksense markup matches mathjax mathml mattress maze mbtc mc6809 mc6809e meerkat mekong melbourne melburn melburn-roobaix meme memolane memories metadata microformats microk8s microsoft millpond miniadventure minutae mist mlp mobile-phone modem monash monash-city-council monash-university monday monitoring monument moon mooramong mortality motionbased motorbike motorcycle motorcycling motorist motorola movie movies mozilla mrtg msi mtb mug mural murder murrumbeena muse muse-mode mushrooming music mycommute mysql names nanoadventure national-trust nbn neighbour nest netbook netbsd netware network newspaper nigeria nightworks nocleanfeed nofilter noise nomnomnom noneshallpass norco norky-bike north-road nostalgia notebook novell nsw nvidia nye nzl oakleigh obesity obituary obp obstruction oes2 oldcar ominous onenet open-tabs openbsd openid openindiana openphoto opensolaris opml oranges org-mode ostrich otter otways outage outpost owl oww oxford p100 palm-pilot panda paperwork parenting park parking parrot passbox pasta pbs pc pc3000 pc3121 pc3166 pc3232 pc3800 pentium pentium-iv peregrine perentie perl pesos pet petrol peugeot philosophy phone photo photograph photography photos php picasa picnic piggies pii piii piv pixelfed pizza plan9 plogging plu pobblebonk podargus podargus-strigoides poem police politics pollution portugal possum postcard postnuke pothole power ppp prawn probe problem proofreading prt pub pubs puck pumpkin pun punchcards puncture pyblosxom qantas qdos qfl qnx qotd r-class r707 radius raido rail-trail railtrail railway rain rainbow rainfall ramones random rant rc17 rdf rdup reading reading-books-rdf real-estate realestate recipe redevelopment renovation rent renting repair repairs reptile restaurant retrocomputing review rhino richmond ride2work ridetoworkday ring ringtail rip ritual river road-rage roadrage robot rocket rockpool rockshox rodent rose roundabout rowing rrd rrdtool rrr rss rta rtabigride rtfm rtwd rubbish ruby ruins russian-women samba samsonite sasl sbs scam scanning scent schmap science scribble script sculpture seabird seaslug security self-reference sensationalism sgml shakespeare shark shimano shoes shopping shopping-trolley sign signwriting silverbirch singing singlespeed sink site site-news site-stuff skateboard skink skink-link skip skyrail slashdot sleep sles sles10 slippery smarthome smidsy smog smoke smoker sneakernet snow snow-train social-media software soloslog solstice soup souvenir spain spam spelling spider spiegeltent spotify spotty-bike spring springer ssh stagecoach station-trail steam-engine storage storm storm-trooper storytlr strava stupidity submarine suburbia summer sunrise sunset supergirl supertuesday support surveillance survey suzuki svg switzerland t-shirt tag tagging taikoz tandem tax taxi tdf teac teamrc17 technicomps technology teeth telemarketing telephone television telstra tent terrorism the-saint the-slog theftbyfinding theme throw thunderstorm thursday ti ti99a tomato touring tourism towel toys tradesman traffic trailgator train-spotter tram trap travel trek trek-t50 tritschler trivia trovebox tunnel tv tvix twat-o-tron twitter typo ubuntu ugliness ui uk uluru unprecedented update upgrade uptime usb vandalism vcr vegemite vhs vic vicroads victoria video vietnam virus vista visualisation vitriol vline vlocity vmware vpn vpnc walking warranty wasp water water-meter waterfall wearegoingawol weather web webcam webfinger wedding wellbeing wemo wetlands wfh whatcouldpossiblygowrong wideopenroad wifi wig wildebeeste wildoz win2000 wind window windows windows7 windy wine winery winxp wired wisp witness wor wordplay workflow www wyvern xemacs xml xp xrd xsl xslt yak yellow yumcha zaf zebra zfs zoo zope zorse

© 1984 - 2021 Adrian Tritschler

Powered by Hugo with theme Dream.


ajft looking stylish and black

…The Owner

There’s not much more I can add to who I am.

…The Site

Vanity site? Technology experiment? Learning tool? Blog? Journal? Diary? Photo album? I could tell you, but then I’d have to kill you…

I experiment. I play. I write and I take pictures. Some of the site is organised around topics, other parts are organized by date, then there’s always the cross-references between them.

Its all been here a fairly long time. Like the papers on my desk, or the books on the bedside table, the pile just grew… and it all grew without much plan or structure. I try not to break URLs, so historical oddities abound.

Long ago it started as a learning experiment with a few static HTML pages, then I added a bit of server-side includes and some very ugly PHP. A hand-built journal/blog on top of that PHP, then a few experiments in moving to various static publishing systems. I’ve never wanted a database-based blogging engine, so over the years I’ve tried PHP, nanoblogger, emacs-muse, silkpage and docbook before settling on Emacs Org mode for writing and jekyll for publishing. But the itch remained… I never really liked jekyll and the ruby underneath always seemed so much black magic. So now the latest incarnation is Org mode and hugo.

…The ISP

  • Hosted by @cos

…The Grue