iSpot & EML project blog

This is the first full week in what seems like a while, what with holidays and courses, and it’s been spent doing some maintenance work on the site.

With the amount of images being uploaded to the site it’s started to emerge that the directory structure needs to be changed to make it quicker and easier to access these images and the longer I leave it to do the harder it gets because of the amount of files that are involved, so that’s been the priority this week. The majority of the work has been looking at the database to make sure nothing is lost in the process of changing things. I’ve now got to the point of feeling confident about going ahead with the changes on the live site but I’ve now just discovered that one of our backup solutions is failing and I need to deal with this before making any live changes. I’ve decided now to put off making any changes until I return from leave, which is all on next week.

I’m hopeful that I’ll be able to make the changes live early in the week I return and then I’ll be able to get back to the location development which was coming along nicely before this interruption.

I’m making good progress on the new user input interface for the location component of iSpot. I’ve used the Google geocode API to create a predictive input field for users to search on location names, much like the the input field that Google Maps itself uses. I’ve also added a second search field that allows users to search on a valid OS grid reference.

My next task is to add pins to the map for all images that have been uploaded for the new observation that contain location info. This will then allow the user to use one of these as the location for the observation.

The location work is going well and I’ve now started work on the changes to the interface that users use to input location information with an observation. This will allow users to search for a location using the Google Maps Geocoding API, it will also allow users to go to a OS grid reference as well as showing any images with location data included as pins on the map.

I’ve successfully moved to using Google Maps version 3 of the API and am using this for displaying a location associated with individual observations along with the latitude/longitude and OS grid ref info in a popup bubble when the location is clicked.

Once I’ve finished the input side of things and have made sure the various links to location data are working I’ll add the progress so far to the approval server for peoples comments.

I’ve at last managed to start work on the location side of iSpot. So far I’ve created a new table structure for storing locations in the iSpot database, although I’m still working through a couple of ideas as to how best deal to with people’s nicknames of places and whether to also deal with actual place names. I’m now focusing on how to display individual locations of each observation before tackling the changes to the interface for users inputting locations. Once all this is done I’ll be moving on to look at the various outputs that we will be able to use the location data for, which will be to do with searching/filtering observations by location.

After having Monday off work sick, the other four days have annoyingly been taken up with various maintenance tasks relating to backend services.

I discovered that the hourly backup that we run of the live site has been failing and it’s taken rather a long time to try and sort this out. I’m hoping that I’m pretty much there with that now although I’ve yet to see the backup site working properly.

The next problem came after an OU network update stopped my ability to update the live site via my normal route. I’ve now sorted this out so I can again implement changes to the live site. On top of this there’s been a usual amount of normal tasks relating to the live site such as updating the S159 student list to reflect the latest changes.

The good news though is that I have now started on the location side of the site although at present I’m just getting up to speed with where I’d got to when I last worked on it. Next week should see me starting to do the code changes.

After the site’s poor performance when it was mentioned on Springwatch I spent a large amount of last week trying to track down the reasons behind it and to rectify them. The problems seemed to stem from connecting to the database server. This in turn was the result of two things combined, the configuration of the database server and a few specific tables in the database being slow to return results.

The actions I’ve taken so far to resolve these issues on the database are:

• using a different configuration of the database server which is tailored to more powerful servers.
• setting up a memory limit for the database query cache to allow this cache to start to be used.
• examining the slow tables in the database and adding indexes where needed to improve their response times.
• upping the maximum connections that the database allows to it by a factor of 10.

On the website side of things I’ve done the following:

• implemented caching where possible using Drupal’s caching mechanisms.
• changed the way Drupal connects to the database server to make use of persistent connections in PHP.
• making sure PHP’s maximum connections to the db reflect those allowed.

With these measures in place the site seems to be responding quicker and I am very confident that the site is much more able to handle increased load, the question is how much?

The next steps are for me to set up profiling on the site so we can start to get an idea of any issues that have yet to be addressed and then in the coming months to arrange load testing to see just how much the site can handle. This will need to be scheduled in to make sure it’s carried out with the lowest impact on users and course students for all courses reliant on the web server.

This week has been about resolving various issues and small feature requests before starting work on the location side of the site again. This has included:

• the upload of the latest batch of S159 students
• yet more work on the next/previous functionality which also now works in a few more contexts such as search results
• the addition of images to the favourites page in the my spot area
• the addition of a bioblitz area which gives a list of all tags that start with ‘bioblitz-’, each list item being a link through to all observations tagged with said tag.
• a fix to a bug that involves the Drupal core and its handling of file names with special characters. This has meant that some users have experienced problems with images that have had special characters in their name.

A few more potential small bugs have been reported that I’m going to check today but then I’ll be starting work on the location side of the site. This will mean that I’ll only really be able to be distracted by major bugs that need fixing although I will continue to make a note of all new feature requests.

I’ve added a new feature to the front page of iSpot – ‘observations from this time last year’. This gives a carousel view of observations made in the same 2 weeks from the previous year. Creating this new carousel was relatively straight forward, the thing that took the time was adapting the ‘next & previous’ functionality to allow it to accommodate a new feature such as this and future features that will also benefit from this. After a weekend of the new feature, along with the re-written next/previous functionality it’s clear there is a bug that causes the next/previous links to disappear in certain circumstances. I’ve got a pretty good idea of the cause of this and expect to fix it today.

There are some additional features that would be nice to implement in relation to the new features on the front page:

• make the second carousel on the front page tabbed to allow users to switch between the current feature.
• make an accompanying page for each feature giving a gallery view of all the observations

Unfortunately I think these may have to take a back seat to the re-build of the location side of the site as this is a priority.

Once I’ve fixed the current bug I’m planning on taking some time to check outstanding jobs and prioritise these. It may be worth checking if there are any small quick wins before getting stuck into the location features of the site.

The re-designed site went live at the beginning of the week. It took half a day as predicted and so was offline for the duration of Monday morning. Since going live the response so far seems to have been favorable.

Although the re-design had been up for a month prior to going live there have still been a hand full of tweaks needed, so that’s been the top priority this week. I’ve also developed a new feature for the front page, namely ‘this time last year’ which as the name implies shows observations from a similar date one year ago. The hold up on this going live has been some work needed to add the next and back functionality to this and future similar features. There turned out to be a problem with Drupal’s handling of session variables when there are concurrent ajax requests. After spending some time looking into this I have ended up having to change the way certain variables are held but it shouldn’t now take much longer to get the next and back feature ready for use with the various carousel features.

There’s not much to report in the way of major features and changes at the moment. There’s one week left until the re-designed site is made live and so all our time has been taken up with addressing minor issues and tweaks to it prior to this. As well as this there have been the normal stream of maintenance tasks relating to the present live site such as adding of new badges for various schemes and societies and the odd bug fix. Several of the bug fixes recently have involved third party modules and one actually turned out to be in the Drupal core itself.

The re-designed site is looking stable and will hopefully continue this way for the final week before the big unveiling!

Aside from the development work another thing to mention is that Richard Lovelock has come to the end of his time with us on this project although he’ll be remaining in IET to work on a new project.

We’re hoping that he may be back with us again in the future to help with further development if funding comes through, but in the mean time I’d just like to thank him for all his hard work. I’ve enjoyed working with him and I’ve been impressed with everything he’s produced. We most definitely wouldn’t have had many of the cool new features ready for the re-design launch without him!!