iSpot & EML project blog

There’s been the usual few maintenance points this week including, a new webservice for S159 to allow validity checking of the exported data, the clear my unread link getting fixed and some changes to the expert badges. As well as these I’ve extended the idea of the likely ID idea to now give an ‘initial ID’ on an observation that doesn’t yet have a likely ID, this means that most observations will now have some kind of ID for the summary view.

Aside from the above, the two big features that we’ve started work on this week are:

• A next & back feature that will allow users to navigate between observations without having to keep going to the parent page.
• An overhaul of the location side of the site.

Both of these features have some complexities to them that need considering in developing them but should provide powerful additions to the site.

This week has seen us able to again focus work on developing the site further, here’s what we’ve been doing this week:

• the addition of a knowledgeable user role for each of the top level groups. Allows users to have their reputation seeded without having to be classed as an expert.
• likely ID feature added. This rates each identification in an observation based on the sum total of the reputation of each user that has agreed with the reputation of the ID author, if the highest scoring ID is above a certain threshold the ID gets a ‘likely ID’ badge and is shown on the summary view of the observation.
• A bug with the oucu login has been fixed. This bug was preventing some users from seeing certain content.
• S159 students will now get a different badge upon completion of the course. This had previously been talked about but not implemented until now.
• A new login feature that allows users to login using an existing account from one of the following providers: google, yahoo, wordpress, facebook, twitter and windows live. At present this will only allow users to create a new profile on iSpot by logging in using this method but the module is being actively developed so linking to an existing profile may soon be possible.

Due to a warning light showing on our main database and file server we’ve moved the hosting of both iSpot and EML on to our backup server. This gave us a chance to test this process and well as allowing time to look at the database/file server.

After spending a significant amount of time documenting the process to transfer the sites to the backup system as well as the process of setting up the actual backup architecture, the move to the backup server was reasonably straight forward, the main issues were as follows:

1. Swapping the domain names to the backup server can take up to 24 hours to propagate through all domain name servers. This means, potentially, that people can not access the site for that amount of time.
2. The process of rebuilding all of the images used in the EML caused the backup web server to become unresponsive and so both sites became inaccessible until the server was restarted.

The latter issue should now have been solved with the upgrading of the backup web server and the sites now seem stable. The former issue relating to the swapping of domains between servers will pose the same problems when we come to move back to the main server. There’s not much that can be done about this this time and in practice people should be able to see the changed location much sooner than 24 hours but we are going to look at revising the process of moving to our backup server in case of future need, to minimise the site’s potential down time.

Now all the backup architecture has been successfully set up and tested work can begin again on the development of site features. Top of the list is allowing users to be seeded with a knowledgeable status to recognise users that aren’t experts but do have knowledge, this has is now under way.

With the backup architecture pretty much finished, just waiting for some network ports to be opened, we’ve got back to work on further site improvements. Rich has now added the ability to link observations into other social networking sites such as Twitter and Facebook. He’s also added a news feature about S159 as well as having carried out several site maintenance tasks such as spam user removal and fixing the ‘more’ link on the groups pages. I’ve managed to finish the work I’d started prior to the backup work on the My Spot area. This now comprises the my unread tab and the favourites tab as well as as the other previous tabs. I’ve also changed the image tab to now be unlinked images with the ability to delete any of these directly from the page.

We have now at last got the automated backup for the live site working on an hourly basis. There are still a couple of things that we’re waiting for to do with the networks to allow us to have a complete mirror site working for both the research and backup site, but the actual data is all now on an hourly backup cycle. The remaining network things should hopefully be sorted out today which will mean that we should have the research copy of the site up in the next few days.

On the maintenance front, I spent some time rewriting the role checking code for S159 students which should now mean no more problems with students not being assigned the correct privileges. It does mean that the code runs a bit more frequently than I’d wanted but this doesn’t impact on the user experience.

This week has been mostly occupied with configuring the back-up and recovery server solution. Although this has consumed a fairly sizeable chunk of time things have been going smoothly and there should only be a day or so of time left to spend configuring this. The result is that we should have a complete back up and disaster recovery solution for iSpot that we should be able to switch to fairly easily and quickly. Backups are going to be taken of the live site (files and database) at hourly intervals, so in the event of needing to implement the disaster recovery solution, the maximum amount of user data that we would potentially lose is one hour’s worth.

Richard G has also spent some time implementing some changes on the ‘Content of interest that has changed page’. I have been preparing a code for a change next week (Tuesday 1st December) after which time the XML files produced by iSpot as a result of S159 student submissions will now be picked up via a web service call from Moodle. This will remove the email and manual import procedure.

I have also spent a small amout of time looking at spam users. As well as potentially using the http://www.stopforumspam.com/api service to check for spam users on our exisiting database I noticed that many of the spam users have been registering with the same first and last name. After running a SQL query to find users that meet this criteria the query returned 249 results (i.e. different users). After manually checking through this data it appears that these were all spam users. A new database table has been created to hold spam users and these 249 records have been removed from the live users table on to the spam users table. I came up against a problem when testing the functionality to call the stopforumspam.com api in that there is a maximum daily call limit (from a given IP address I presume) to the API. I hit this limit during development today which resulted in me not being able to continue testing as I would have hoped. I hope to have the module to check the stopforumspam.com database when users register on iSpot put live on Monday or Tuesday of next week.

This week has seen the overhaul of the changes page in the My spot area of the site. It now has the ability to allow users to filter the list of changes by their type of link to each observation, so now they can choose to see any combination of the following:

• Observations authored by them
• Observation that they’ve commented on
• Observations on their favourites list
• Observations they’ve added an identification to
• Observations where they’ve agreed with an identification

The results are then displayed in sections depending on the nature of the change that’s taken place. The possible sections are:

• Core observation details edited by others
• Comments added by others
• Identifications added by others
• Identifications that have been agreed with by others

I’m now just working on adding a bit more help text to the page to try and improve the instructions to the users on what everything does.

The MegaLab officially stopped gathering data as of 31st October. This does not mean that the site is no longer available, in fact it’s business as usual for the Evolution MegaLab but it does mean that analysis has now begun on the data that has been collected to date. So to help with this I’ve added two new fields to the full data download that’s available to privileged users, these are: an id field that gives a unique reference to every line of the download, and the date the record was published – this is different from the date the record was made.

It’ll be interesting to hear the outcomes of the data analysis!

This week has seen the addition of links to both EOL and the NBN in iSpot. These links have been added to all identifications that have a valid taxon name in the NBN species dictionary, the scientific name is given precedence but if the scientific name is invalid and there is a valid English name then that is used instead. At present these links are only created once the identification has been published but I feel these links may be a good aid prior to publishing an identification and so it may be a good idea to look into how best to let users use these resources separately from publishing identifications.

As well as these links being added some of the other things that we’ve also done are as follows:

• Rich has added more custom reports for use by the iSpot team including User statistics which has some interesting information
• The active S159 students now have a badge so they can spot each other on the site
• it’s now possible to view who’s agreed with an identification, this still needs a bit of work to get the popup using a fixed size box and then the use of a scroll bar but it’s usable for the time being
• many small improvements to both the look & feel and functionality spread throughout the site!!

We’ve been doing a lot of work on addressing issues concerning various pages, both their look and usability. We’ve got the current theme working correctly on pages that previously had problems and have created a better way of viewing teasers of observations. The add some images page has been tidied up as has the second step page to it, which now also makes it clearer to users that they need to complete this step to use the images in the site.

The lightbox functionality has been tweaked for displaying images and this is now being used as the more feature on the latest observations block on the front page. The latest observations page that used to be linked to from this more link has been changed to the browse observations page a link to which can be found in the right hand ‘browse’ menu of the site. This page has lots of added functionality and allows users to apply filters to all the observations and browse the result.

The search functionality of the identification fields has also been improved and it now deals with hyphens as well as being able to search on multiple partial words. Hopefully this will be helpful to Mike and the problem with fungi