iSpot & EML project blog

The replacement of the current location functionality is pretty much ready for release. I’ve now finished the redevelopment of what was the my spot locations tab, now called the map tab, this is now just one large map with pins for each of a users observations. The pin icons themselves correspond to the group that the observation is in expect for locations with multiple observations, for these there is a different colour pin with the iSpot logo on.

In preparation for the live release of this I’ve updated the beta site – ispot-beta.open.ac.uk so this is now open to anyone how would like to try out/test the new features before hand.

The other bugs that there had been with the observation input form have now been resolved although there may still be others waiting to be uncovered

Since launching the overhauled location features on the approval site I’ve been working on extending the capabilities of the meta data capturing functions on the site, these can now also deal with the Adobe XMP format that is becoming more popular. There does still seem to be issues with some types of embeded exif data that Mike has alerted me to and I’m presently trying to work these out.

Other than this I’m also working on the redesign to the observation input form and this is taking shape nicely, with the use of help icons next to input fields instead of the help text running down the right hand side.

At last!!! The long awaited changes to the locations on iSpot are ready for approval. These include:

• the observation location being shown in a popup similar to the popup used for showing people who’ve agreed with a determination.
• pins used in place of vague green circles.
• a 1km grid square used instead of a pin where the user wants to hide the exact location.
• a re-worked input including; search by place name or by OS grid coordinate.
• use of meta data from images that are uploaded, allowing users to fill in details of the observation form from these.

I’ve made the new version of the site available on the approval site and I’ll email the link to people. After people have had a chance to test out the new features I’ll collect feed back and make any appropriate changes before testing again and then making live.

These changes will now pave the way for adding many new features based around locations, such as searching for observations in a given area.

This is the first full week in what seems like a while, what with holidays and courses, and it’s been spent doing some maintenance work on the site.

With the amount of images being uploaded to the site it’s started to emerge that the directory structure needs to be changed to make it quicker and easier to access these images and the longer I leave it to do the harder it gets because of the amount of files that are involved, so that’s been the priority this week. The majority of the work has been looking at the database to make sure nothing is lost in the process of changing things. I’ve now got to the point of feeling confident about going ahead with the changes on the live site but I’ve now just discovered that one of our backup solutions is failing and I need to deal with this before making any live changes. I’ve decided now to put off making any changes until I return from leave, which is all on next week.

I’m hopeful that I’ll be able to make the changes live early in the week I return and then I’ll be able to get back to the location development which was coming along nicely before this interruption.

I’m making good progress on the new user input interface for the location component of iSpot. I’ve used the Google geocode API to create a predictive input field for users to search on location names, much like the the input field that Google Maps itself uses. I’ve also added a second search field that allows users to search on a valid OS grid reference.

My next task is to add pins to the map for all images that have been uploaded for the new observation that contain location info. This will then allow the user to use one of these as the location for the observation.

The location work is going well and I’ve now started work on the changes to the interface that users use to input location information with an observation. This will allow users to search for a location using the Google Maps Geocoding API, it will also allow users to go to a OS grid reference as well as showing any images with location data included as pins on the map.

I’ve successfully moved to using Google Maps version 3 of the API and am using this for displaying a location associated with individual observations along with the latitude/longitude and OS grid ref info in a popup bubble when the location is clicked.

Once I’ve finished the input side of things and have made sure the various links to location data are working I’ll add the progress so far to the approval server for peoples comments.

I’ve taken the decision to move to using Google Maps version 3 api. I thought that it was probably something that we’d need to be doing at some point so it’s probably best to do it now whilst re-developing the location functionality. Using version 3 means that I’ll no longer be using the GMaps module for Drupal which I had been up until now. This is a very good module, but I’d always thought it was a bit more heavyweight than we needed and because it has not yet been upgraded to Google version 3 it is my opportunity to make a custom Google maps interface that will be much more lightweight, this should help in some way with page loading times.

I’ve also managed to get the grid references working on the maps so we can now make use of grid squares on the maps also I’ve now got the map for individual observations being shown in a javascript popup box akin to the box used for showing people who’ve agreed with IDs, so I’m moving along nicely with the location display side of things.

Other things I’ve been working on this week are:

• getting the research and backup versions of the site working properly again. This just involved getting the site code to reflect the current live site. The actual database data and images have been being copied across successfully.
• had to address a small bug with the S159 data export which involved the representation of the presentation code.
• been looking into performance monitoring/testing applications and have set up some test apps on the development server.

I’ve at last managed to start work on the location side of iSpot. So far I’ve created a new table structure for storing locations in the iSpot database, although I’m still working through a couple of ideas as to how best deal to with people’s nicknames of places and whether to also deal with actual place names. I’m now focusing on how to display individual locations of each observation before tackling the changes to the interface for users inputting locations. Once all this is done I’ll be moving on to look at the various outputs that we will be able to use the location data for, which will be to do with searching/filtering observations by location.

After having Monday off work sick, the other four days have annoyingly been taken up with various maintenance tasks relating to backend services.

I discovered that the hourly backup that we run of the live site has been failing and it’s taken rather a long time to try and sort this out. I’m hoping that I’m pretty much there with that now although I’ve yet to see the backup site working properly.

The next problem came after an OU network update stopped my ability to update the live site via my normal route. I’ve now sorted this out so I can again implement changes to the live site. On top of this there’s been a usual amount of normal tasks relating to the live site such as updating the S159 student list to reflect the latest changes.

The good news though is that I have now started on the location side of the site although at present I’m just getting up to speed with where I’d got to when I last worked on it. Next week should see me starting to do the code changes.

After the site’s poor performance when it was mentioned on Springwatch I spent a large amount of last week trying to track down the reasons behind it and to rectify them. The problems seemed to stem from connecting to the database server. This in turn was the result of two things combined, the configuration of the database server and a few specific tables in the database being slow to return results.

The actions I’ve taken so far to resolve these issues on the database are:

• using a different configuration of the database server which is tailored to more powerful servers.
• setting up a memory limit for the database query cache to allow this cache to start to be used.
• examining the slow tables in the database and adding indexes where needed to improve their response times.
• upping the maximum connections that the database allows to it by a factor of 10.

On the website side of things I’ve done the following:

• implemented caching where possible using Drupal’s caching mechanisms.
• changed the way Drupal connects to the database server to make use of persistent connections in PHP.
• making sure PHP’s maximum connections to the db reflect those allowed.

With these measures in place the site seems to be responding quicker and I am very confident that the site is much more able to handle increased load, the question is how much?

The next steps are for me to set up profiling on the site so we can start to get an idea of any issues that have yet to be addressed and then in the coming months to arrange load testing to see just how much the site can handle. This will need to be scheduled in to make sure it’s carried out with the lowest impact on users and course students for all courses reliant on the web server.