iSpot & EML project blog

After having Monday off work sick, the other four days have annoyingly been taken up with various maintenance tasks relating to backend services.

I discovered that the hourly backup that we run of the live site has been failing and it’s taken rather a long time to try and sort this out. I’m hoping that I’m pretty much there with that now although I’ve yet to see the backup site working properly.

The next problem came after an OU network update stopped my ability to update the live site via my normal route. I’ve now sorted this out so I can again implement changes to the live site. On top of this there’s been a usual amount of normal tasks relating to the live site such as updating the S159 student list to reflect the latest changes.

The good news though is that I have now started on the location side of the site although at present I’m just getting up to speed with where I’d got to when I last worked on it. Next week should see me starting to do the code changes.

After the site’s poor performance when it was mentioned on Springwatch I spent a large amount of last week trying to track down the reasons behind it and to rectify them. The problems seemed to stem from connecting to the database server. This in turn was the result of two things combined, the configuration of the database server and a few specific tables in the database being slow to return results.

The actions I’ve taken so far to resolve these issues on the database are:

• using a different configuration of the database server which is tailored to more powerful servers.
• setting up a memory limit for the database query cache to allow this cache to start to be used.
• examining the slow tables in the database and adding indexes where needed to improve their response times.
• upping the maximum connections that the database allows to it by a factor of 10.

On the website side of things I’ve done the following:

• implemented caching where possible using Drupal’s caching mechanisms.
• changed the way Drupal connects to the database server to make use of persistent connections in PHP.
• making sure PHP’s maximum connections to the db reflect those allowed.

With these measures in place the site seems to be responding quicker and I am very confident that the site is much more able to handle increased load, the question is how much?

The next steps are for me to set up profiling on the site so we can start to get an idea of any issues that have yet to be addressed and then in the coming months to arrange load testing to see just how much the site can handle. This will need to be scheduled in to make sure it’s carried out with the lowest impact on users and course students for all courses reliant on the web server.

This week has been about resolving various issues and small feature requests before starting work on the location side of the site again. This has included:

• the upload of the latest batch of S159 students
• yet more work on the next/previous functionality which also now works in a few more contexts such as search results
• the addition of images to the favourites page in the my spot area
• the addition of a bioblitz area which gives a list of all tags that start with ‘bioblitz-’, each list item being a link through to all observations tagged with said tag.
• a fix to a bug that involves the Drupal core and its handling of file names with special characters. This has meant that some users have experienced problems with images that have had special characters in their name.

A few more potential small bugs have been reported that I’m going to check today but then I’ll be starting work on the location side of the site. This will mean that I’ll only really be able to be distracted by major bugs that need fixing although I will continue to make a note of all new feature requests.

I’ve added a new feature to the front page of iSpot – ‘observations from this time last year’. This gives a carousel view of observations made in the same 2 weeks from the previous year. Creating this new carousel was relatively straight forward, the thing that took the time was adapting the ‘next & previous’ functionality to allow it to accommodate a new feature such as this and future features that will also benefit from this. After a weekend of the new feature, along with the re-written next/previous functionality it’s clear there is a bug that causes the next/previous links to disappear in certain circumstances. I’ve got a pretty good idea of the cause of this and expect to fix it today.

There are some additional features that would be nice to implement in relation to the new features on the front page:

• make the second carousel on the front page tabbed to allow users to switch between the current feature.
• make an accompanying page for each feature giving a gallery view of all the observations

Unfortunately I think these may have to take a back seat to the re-build of the location side of the site as this is a priority.

Once I’ve fixed the current bug I’m planning on taking some time to check outstanding jobs and prioritise these. It may be worth checking if there are any small quick wins before getting stuck into the location features of the site.

The re-designed site went live at the beginning of the week. It took half a day as predicted and so was offline for the duration of Monday morning. Since going live the response so far seems to have been favorable.

Although the re-design had been up for a month prior to going live there have still been a hand full of tweaks needed, so that’s been the top priority this week. I’ve also developed a new feature for the front page, namely ‘this time last year’ which as the name implies shows observations from a similar date one year ago. The hold up on this going live has been some work needed to add the next and back functionality to this and future similar features. There turned out to be a problem with Drupal’s handling of session variables when there are concurrent ajax requests. After spending some time looking into this I have ended up having to change the way certain variables are held but it shouldn’t now take much longer to get the next and back feature ready for use with the various carousel features.

There’s not much to report in the way of major features and changes at the moment. There’s one week left until the re-designed site is made live and so all our time has been taken up with addressing minor issues and tweaks to it prior to this. As well as this there have been the normal stream of maintenance tasks relating to the present live site such as adding of new badges for various schemes and societies and the odd bug fix. Several of the bug fixes recently have involved third party modules and one actually turned out to be in the Drupal core itself.

The re-designed site is looking stable and will hopefully continue this way for the final week before the big unveiling!

Aside from the development work another thing to mention is that Richard Lovelock has come to the end of his time with us on this project although he’ll be remaining in IET to work on a new project.

We’re hoping that he may be back with us again in the future to help with further development if funding comes through, but in the mean time I’d just like to thank him for all his hard work. I’ve enjoyed working with him and I’ve been impressed with everything he’s produced. We most definitely wouldn’t have had many of the cool new features ready for the re-design launch without him!!

The end of last week saw the design ready to be uploaded to the approval server. After the showing the team this at Mondays meeting, various suggestions were made for slight changes along with requests for a couple of new features to be added for the launch of the re-designed site.

This week has been taken up with implementing all the requests. The design now works on IE6 although it’s not quite to the standard of the newer browsers. The various small general issues have been fixed with the exception of one or two things yet to complete. The two big new features that have been added are:

• images to the search results, I’ve also changed the displaying of identification search results they’re now showing the observation they’re made on and added images to these too
• a related observation carousel on every observation page. This looks the same as the latest observations carousel on the front page but obviously is populated with observations that share the same likely ID

Another feature that we’ve yet to add is an extra section on the front page under the latest observations that will look similar but will be populated with old observations that are yet to have a likely ID, we were thinking of calling this block something like ‘can you help’.

The final thing that we’ve done this week is created a new domain for the beta testing of the re-design prior to launch. This is accessible externally and is a separate site to the internal approval site. The URL is……. ispot-beta.open.ac.uk

The site re-design is going very well, although we haven’t quite managed to get it to the point of uploading it to the approval site. I expect to now give people access to it on the approval site sometime in the coming week but I’d prefer it to be as near to completion as possible so any comments will be more valid. It present I’m putting the finishing touches to the ‘Add an observation’ form page and Rich is working on getting the latest observations carousel on the front page working.

This week we’ve been concentrating on creating a new theme for the site using the approved re-design. The first two and a half days were taken up with creating a design for the various elements of the site that are used elsewhere other than the front page. The rest of the week has been taken up with starting to implement this new design.

I’m hoping that by the end of next week we should have a site ready for viewing on the development server. I’ll send an email around when it’s in place for people to look at.

Richard G. and I are going to meet soon to consolidate our list of  ‘to-do’ tasks and discuss and prioritise these tasks in relation to the site redesign.

We are going to make use of the new issue tracker system in IET to log these items. The issue tracker will most likely be available to the project team to be able to update.

The next major phase of work is the site redesign which is due to begin very soon. We have discussed batching up a couple of the functionality changes (e.g. next/previous functionality, image upload functionality) to be released with the site redesign so that it feels as though it is more of a new, improved iSpot that has been released rather than just a change to the look and feel of the site.

Development on the new location features is also on going.