Writings on technology and society from Wellington, New Zealand

Friday, September 26, 2008

A little programming project, part 2

A week ago I posted about my attempts to automate an administrative task with Python. In essence, I’m trying to scrape links to sound files off the Radio New Zealand site and insert them into an entry on my blog. I can only test this on Thursdays, because the links are only present then.

Read on for my experiences yesterday and a revised program.

So, yesterday I ran it again, and I had to tweak it a bit but it did then work. I’ve also installed a loop to wait for the links. The biggest problem was that my assumption that the sound file names would contain New_Tech – this week they just contained the word “technology”.

So, the revised program – the one that worked – looked like this.

Where to from here?

  1. I’d like to improve the code that finds the sound file links – the link names change and it’s likely to prove unreliable. Instead, I could trigger on my name being in the link but that’s not a constant, either. Probably I’m going to have to re-write it to look for the text introducing the links, then find the links themselves. Just not now.
  2. I need to get this running without human assistance, probably using a cron job on a GNU/Linux server at home. Hopefully by next week.
posted by colin at 7:26 am  


  1. The page where you posted the current program seems to have suffered from WordPress’s over-eagerness to interpret your linktext, rather than just showing it to us as-is.

    Comment by Tim McKenzie — 26 September 2008 @ 5:41 pm

  2. Drat it, you’re right. It’s not the first time I’ve manually escaped all the angle brackets in that line of code it probably won’t be the last. Hopes that’s fixed it!

    Comment by colin — 26 September 2008 @ 9:50 pm

  3. I’ve just figured out how to suppress the sidebar, which was colliding with the text in the program listing. I did this by finding where the sidebar was called. From a command line, I went to the directory where the theme for the blog lives and issued:

    grep get_sidebar *

    This showed me that the sidebar was called from a file called footer.php. The call looked like this:

    <?php get_sidebar(); ?>

    I changed it to read:

    <?php if(is_page()) : else : get_sidebar() ; endif ; ?>

    The effect of this is to turn the sidebar off for all pages (as opposed to posts). I could have been more selective and just switched it off for that page by putting a page name argument between the brackets of the is_page() call.

    I got the information on how to do this from searching the WordPress forums.

    As a separate exercise, I’ve taken the Python program listing page out of the sidebar by changing its status to “Pending review”. You can still get to it from the link in this post.

    Comment by colin — 27 September 2008 @ 10:22 am

  4. Um, I now get a “Sorry, no posts matched your criteria.” message when clicking on the link.

    Comment by Peter Lynch — 28 September 2008 @ 3:13 pm

  5. Peter – sorry you get that – I don’t. Send me email if the problem persists.

    This week I set up a cron job on my Linux server, monad, to run the program automatically. I told the job to fire at 11:30 every Thursday. It didn’t work this week because I stupidly got a password wrong. But the program worked perfectly a hour later when I logged on to the Linux box using ssh and typed its name manually. I’m reasonably confident that the automation will work next week.

    Comment by colin — 3 October 2008 @ 8:56 pm

  6. I get the same problem as Peter.

    Comment by Tim McKenzie — 7 October 2008 @ 4:18 pm

  7. […] ago I blogged about writing a little program to make my life easier. (The entries are here and here.) In summary this program automates the messy but easy administrative task of editing links to the […]

    Pingback by » A little programming project - part 3 — 20 October 2008 @ 12:24 am

RSS feed for comments on this post.

Sorry, the comment form is closed at this time.

Powered by WordPress