We’ve managed to get the sys-admin work carried out on the new servers and they should now be ready for me to transfer the hosting of iSpot to. I’ve also had a phone call from AACS to say that they’re looking to moving the hosting of the domain resitrations for the MegaLab domains, so hopefully that will be sorted out soon too.
I have been investigating the use of the geometry field that MySQL makes available as a way of storing the latitude and longitude values in the Evolution MegaLab. I had hoped to move to using this field because of the additional functionality that MySQL makes available with this, the problem is that Symfony uses an ORM layer to make it independant of a specific database, and this ORM layer does not have the ability to map to the geometry field in MySQL.
I’ve looked into using a different ORM layer, Doctrine instead of Propel, but they both seem to have the same problem. I’ve also considered extending Propel to somehow allow the use of the geometry field, but given the limited time I think that would take to long. My final thought was to hack it and initially tell Propel that the field is a VARCHAR field, and then manipulate the code once Propel had set up the db, but I think this would be an unwise solution, and also would possibly take too long.
So I’ve decided to go with the “if it ain’t broke, don’t fix it” approach and think about maybe implementing the geometry field sometime in the future if the additional fuctionality is needed. Hopefully if I come back to this in the future the ORM layer will then be able to cope with spatial extensions.
After taking a day and a half at the beginning of the week to get the latest version of iSpot to a more presentable state, I took the rest of the week to make updates to the Evolution MegaLab site. Most of the changes to the content of the site are done, although there are a couple of things that need to be discussed before completing the work. I’ve yet to make any of the changes to the database structure and other requested things, but hopefully the content changes will keep the translators happy for the time being.
I’ve linked my custom observation module into the drupal views module now to allow the creation of various views of the observation data. This is to enabled the likes of the ‘just seen’ block on the front page, and the observation section in the MySpot area.I’m now working on the location side of things and have created a new content type that will store a location associated with a user. I’m using the spatial extension in MySQL which Andy (the developer of the reputation module) told me about. This should allow us to use the stored data in some very cool ways. It may well be an idea to think about using this also for the MegaLab location infomation as it will allow the data to be manipulated in many more ways.I’ve also just got the default location for a new pin on a map working from a latitude longitude coordinate for the city which the machine ip address is.
Another short week this week due to the bank holiday, and the Evolution MegaLab workshop on Wednesday. It was really good to see so many people from different countries in Europe and also hear about the potential use in schools. Also after listening to the expert speakers talk about how to identify the snails, and the potential use of the data, I’m suddenly finding cepaea an even more interesting topic, and have stopped a couple of times already to examine snails I’ve come across 
As for iSpot, I’m still in the midst of developing the observation input for users, which would probably have been done by now if it wasn’t for the problems I’ve come across with the modules that I’m needing to use in its creation.
I’ll be meeting with Doug shortly with the aim to move the graphic design proccess along and get the design competition started, and also revisit the technical specification and how and when we’ll have further user input into this.
This week has seen a few more issues come to light that have needed addressing.
The first was associated with the adding of a further two translation languages. One of these languages was going to be assigned to a current translator, but this brought to light the fact that each translator was only able to be associated with one language. This is now resolved with multiple languages being able to be assigned to each translator.
The second issue involved the use of commas within fields in the .csv files used for uploading additional historical data. Excel allows this by signifying that commas are to be ignored by enclosing the particular field in double quotes. Although I did create an algorithm to deal with this, I was unable to construct a regular expression to do it which was annoying, I’d be interest to know if it’s possible to do this with a regular expression, as I sure it must be. In the end I decided that it would be best to ask for the commas that are not used as delimiters to be removed before uploading the file, as there usually is a large amount of records to be added and the additional processing of each line would have slowed the already slow upload time further.
The final issue came to light because many of the additional historical records did not have a habitat type defined. this stopped the site from generating the popup in the google map. I’ve fixed this by assigning ‘unknown’ as the habitat type for the popup, but it does mean that all of these records are unable to be used for comparison to newly added records because the matching of the habitat type is one of the criteria that must be met to create a comparison.
I’ve added a couple of things to the MegaLab site this week that are needed whilst it’s in beta. The first is an opt in/out request for users to participant in giving site feedback, along with this is the ability for administrators to download the list of users that have opted into this. The second thing is the addition of a country category to the historical records, this allows for the distinction between the various European countries that may wish to up load these records.
Last week a couple of issues were pointed out regarding the site.
The first turned out to be a problem with how Microsoft Excel opens .csv files. This is that any .csv is opened in ASCII format even when it’s actually in utf-8 format, this means that the extended character set of utf-8 is not displayed correctly. The fix for this is to first open Excel and then import the .csv file which allows you to set the format type to utf-8.
This second issue is related to the order in which a user fills in the record information for recording samples. If the user fills in the latitude or longitude field last, the ‘create this record’ button may not be activated. This is because the ‘onchange’ event which triggers the button activation is not fired until the curser leaves each field. This is standard behaviour, but has caused problems already for one user. I’ll need to give the solution to this some thought, but for the time being, I think all we can do is try and monitor how many people have this problem.
After fixing some more bugs at the beginning of the week and setting up the facility to have a different email support address for each language, I been looking at optimizing the live site. I had a meeting with Adrian Wells in AACS to discuss load testing and optimization and he gave me a few tips. As a result I’ve now installed a PHP accelerator onto the production server. I’ve used APC as this is the one that seems to be favored by the PHP core development team and will be included as standard in PHP 6.
Another thing suggested by Adrian in to use connection pooling when connecting to the database. This is something I need to look into a bit more because the Symfony documentation says there’s pros and cons and isn’t best for all circumstances.
I haven’t conducted any load testing as yet, although I have downloaded some free software for doing this called JMeter, but there looks to be quite a lot to setting up the test actions which will require me spend some time learning about this.
I wonder if it would be best to use this beta phase to monitor performance and consider load testing at the end of this year if it’s felt necessary.
The next round of accessibility testing is scheduled for the last week in January and is being carried out by Chez Colwell, who has said she’d be prepared to write an accessibility information page for the site, which I think would be good.
The other thing left to do is schedule some user testing.
It’s been a good week for sorting out bugs on the live site, and it’s been very helpful having translators actively using not only the translation area but also the site in general.
None of the bugs so far have been major, and have been easily fixed.
I have just noticed one that, although not necessarily major, may take slightly longer to fix. This is that at the moment the bar chart that is produced for a location is only produced in the selected language for that user. This means that if another user or the same user selects a different language there is no bar chart shown.