The weekly iSpot blog will now be on our new team blog http://www.open.ac.uk/blogs/LTT_IET/ and will continue to appear there with other blogs updating you on the projects and the work of the Learning and Teaching Technologies team in IET.
Weekly Update from the Learning and Teaching Technologies (LTT) team on technical developments on iSpotJanuary 23rd, 2015 by Ola Fadoju
As from this Friday onwards the Learning and Teaching Technologies team will be providing a weekly update on what the team is working on regarding the iSpot website back-end services. We are aiming to post these updates on Fridays.
What was scheduled for this week
The team had a technical meeting on Tuesday 20th January to prioritise bug-fixes, performance and functionality issues that have been identified on the iSpot website. After discussing the list and reviewing the logs and statistics we have been collecting for the last few weeks, it was clear that some of the bugs may actually be the result of an underlying problem with the architecture, which has only become evident after importing the large amount of data from the South African site. Specifically we can see evidence of a regularly high contention ratio on the database, which is causing very poor performance for our users, and appears to be contributing to duplicate observations being posted among many other things. We believe that a number of the high priority bugs and problems users are having can be solved by updating back-end technologies, which would include a change of database engine to MySQL 5.7. Therefore the immediate focus is upgrading the software architecture.
We would like to make clear that we need to make these changes and then review whether any of the issues have been solved or need further attention. We also have a list of additional bugs which will start to drop in to our schedule on a weekly basis – these will be identified here, as we go. These include things like ‘duplicates appearing in carousels’, ‘cannot clear unread messages’, fixing ‘add interaction’ functionality.
What has been completed this week
The development site has been upgraded from PHP5.3 to PHP5.5 with performance gains (as time to generate various pages) of about 15%. Testing is continuing, and the update will be applied to the live iSpot website either next Tuesday or Thursday, if no problems arise.
The development site has had some updates to core Drupal and PHP fixes. This will be finalised this week and the live site will be updated either next Tuesday or Thursday if no problems are encountered during testing.
What will happen next week
Firstly we will be completing testing and updating the servers with the changes identified above. We will then monitor to see if the changes have resolved some of the major bug issues. In addition, the developers will continue to work on fixing a number of bugs; three have been classified as high priority and are as follows:
I. Duplicate observations being posted
II. Editing comments and replies on an observation changes the date to the edit date
III. Editing comments and replies in a forum posting changes the date to the edit date
Secondly, we will be planning our database migration to MySQL 5.7, which is a significant piece of work. If all goes well, we should be trying the migration on a test server by the end of the week.
There is no guarantee that bugs will be fixed before next week Friday, however the team will continue to work on key performance and functionality issues at the back-end.
Things are going well with the sites merger. I’ve now successfully completed a full merger of the two sites on a test install and am confident that this side of things is ready for use on the live sites. The full process took something like 3 days to complete although on the live DB this may well be significantly quicker.
One thing that’s worth flagging up is how the user merger will happen. The algorithm I’ve written uses the following steps to import a user:
- Check for an existing user with a matching email address on the global site – if one exists merge these two users using the global user’s username.
- Check for a matching user name on the global site – if the user name exists rename the southern African user to a unique name and create a new global user with this name.
- if the southern African user’s email and username are unique – create a new global user using the southern Africa user details.
With this in mind it’s worth noting that if any southern African users have signed up to both sites but used different email addresses, I know of at least a few that have, then this will mean they’ll end up with two user accounts. It will save a lot of time later if any user wishing to have their accounts merged into one could check that both accounts use the same email address and if they don’t then change them so they do.
The groups pages will now be found hierarchically under the communities so meaning the southern Africa community will keep their version of these pages. This goes for the forums also with areas now created for each community. This means the SA forums will all be imported into the SA community forum area. There may need to be some editing of topics after this to bring anything into line that may need to sit in a different ‘site’ related area.
Badges are assigned on a per user basis and so have no concept of community. This will mean any badging information page per community will need to be hand written and maintained at present.
To bring the two sites into line, the social points system will change to using the points scoring that the SA site uses.
The sensitive location area can now be specified per community which allows curator’s to ensure their rules in this area are adhered to. This applies also to the habitat list so this can vary between communities.
After the merger has taken place each node, whether that’s a story, page, forum topic or observation will be assigned a redirect to their new URL on the global site. The southern Africa site itself will be placed in offline mode to ensure no further content can be added. This will mean anyone attempting to access any content on the SA site will be redirected to the appropriate URL on the global site.
I think that probably covers most of what I’ve done so far. At present I’m working on the UI changes that are needed to accommodate the above changes such as the habitat list. But I’m hopeful that I’m nearing the end.
So for the past two weeks I’ve been focused on the up coming merger of the southern Africa site with iSpot Global. I had previously been making changes during the version 2.6 development with the merger in mind but the time has now come to tackle the outstanding issues and also develop the framework for carrying out the actual entities migration.
The first major change has been the removal of the Organic Groups module, a third party module, which is a hangover from the very beginning of iSpot. In the beginning it was invisaged to be one of the underlaying major components of the site but for various reasons this did not become the case. The module has long since been deprecated and, I think, has possibly had security implications (although not in the way we’re using it). It’s been rather involved to remove, mainly because many of our custom modules make reference to it. I’ve now completed this which will enable the separation of the iSpot groups within each community but I think the alternate method I’ve now used should also allow for more flexibility and extensibility in the future… Farewell Organic Groups you’ve caused a lot of headaches over the history of iSpot!
The next major programming task is to create the migration framework, which I’ve now started. I’ve got a fair idea of the task at hand but I think many of the intricacies will only become apparent as I develop it. I’ve got local versions on both sites and the only way to be sure is extensive testing of each step of the process.
Well version 2.6 has been live since the 20th August. The actual upgrade went very smoothly, the site was offline for around 2 hours (as anticipated) and didn’t suffer any catastrophic bugs once it went live. The biggest initial issue was that of the use of HTTPS by the oembed module that had been created for the Kew project. This really came down to the fact that the SSL on the server was only finally set up on the day of the launch and then we found that the module in question hadn’t been tested to work over SSL. Might have been better to postpone the Kew launch for a few days but anyway, I fixed the issues and all seems well with the oembed stuff now.
Since the launch there have been various minor issues that have come to light (as they always do) and I’m sure there are still more to be found. Here’s a quick run down of the issues that have been dealt with so far:
- Previews not working when adding an observations – this seems that it may have been an issue prior to v 2.6 but it’s fixed now!
- The ‘Home’ link in the Explore community drop down menu linking to incorrect pages in some instances.
- The filter used to create the initial ‘Your view’ didn’t have a map bounds associated with it, so the filter wasn’t working.
- Multiple issues on IE 8.
- A filter option on the communities map page but not on list or gallery page – I’v also now added a block giving a current filter summary on these pages too.
- The pager (next, previous etc…) wasn’t working correctly in some instances.
- some lists on the your iSpot home page were showing duplicates.
- Removed the tick box from the user profile for their consent to email them with updates (incorrect wording at present, so best not to be visible).
- The add identification and add species interaction sections were unintuitive – hopefully slightly better now.
- The user favourites list now defaults to date added descending order.
- Auto scrolling happening on the image carousel on individual observation pages.
- Incorrect polygon showing on projects with no geographic bounds.
Something we’re hoping to implement on iSpot, sometime soon, is the ability to ‘friend’ other users. This is obviously quite a basic thing in the world of online social networking. So I’d been thinking about it recently and a slight twist on this suddenly occurred to me. Why don’t we allow users on the site with a certain reputation to ‘mentor’ other users. My initial idea is as follows:
- users on the site would be able to request a high reputation user to mentor them, maybe by way of a ‘mentor me’ button on their profile. The user being asked would obviously be able to accept of decline maybe giving feedback as to why e.g. already ‘mentoring’ enough users etc…
- once the link is established the mentor would get notifications about the protégé, such as a new observation added or a question asked.
- We could have separate areas, communities or projects maybe, for each mentor and a protégé could decide whether to upload just to that area. It could still be a public area but may be less daunting to post to.
- There could even be the possibility of developing a mobile app to complement it. Maybe allowing real time ‘mentoring’ in the field.
I’ve had various other ideas about the intricacies of how the system might work but I think this about covers the basics.
My reasoning behind this was that it might be a way of both dealing with the issues around novice users whilst at the same time possibly appealing to the expect users.
Just an idea at present but you never know something that could be built into a proposal for funding maybe?
The past week has seen user testing of the new look site. This has comprised both, the use of the IET user labs for a structured testing process aimed at novice users, and the use of existing iSpot users accessing our approval site which was more open-ended.
The results have highlighted some known areas which need improvement such as the map pages that show observations and the users’ expectations of these, but also many other areas that were less obvious (until you’re watching a user on the site!). There were things like the users completely missing the main navigation bar when it was situated at the top of the page, users never clicking through from the teaser view of an observation to the main view of it and novice users feeling that adding an identification on the ‘add observation’ page was compulsory and so being rather put off.
We’ve also identified some core Information Architecture issues that combined with many other, seemingly small, usability issues have conspired to prevent the novice users from engaging with the site in the ways we would hope.
We’re developing a plan to carry out the highest priority areas of improvement prior to launch and, I think, we’re all confident that with the suggested changes in place iSpot will become a very much more usable website!… fingers crossed
So, other than the various annoying tweaks and bug fixes that start needing to be addressed at this point in the development cycle, MySpot has been the area receiving a fair amount of my time in an attempt to start bringing a more personalised experience to the user.
I’ve now given a separate tab to the filter area for users in an attempt to allow users to find and leverage this functionality more. This is then separated from the user’s observations tab that we felt was important enough to always have its own area. I’ve also given a view of this as a tab in the user profile are, so allowing other users to easily view a users contributions at a glance. The hub page has now under gone somewhat of a change with
- a carousel for their current filter
- a carousel for their home community latest observations
- a feed showing changes to any of their observation
- a news feed again from their home community
- a recommendations area that shows observations that might be of interest to the user.
At present the recommendations area is a bit rudimentary with just observations we think the user may be able to help with being shown but we’re hoping to extend this in time. We’re also hoping to add a statistics area giving the user some interesting statistics about their usage of the site.
Things are continuing in the right direction. This week I’ve, at last, managed to finish the context bar so this should now give the user a reasonable reflection of the context of their current page. This has been a real nightmare, and one of those times that Drupal as really felt like it’s getting in the way! It seems that pretty much each different page type has involved me first tracking down where the breadcrumb was being set and then working out the best of overriding this!
As well as the progress bar, I’ve also overhauled the interactions tab in the communities area to reinstate the ecological web page as well as the intro page.
Now this is out of the way I can press on with the growing list of ‘final’ improvements as well as addressing some maintenance issues on the live site that have been hanging around for a couple of weeks now.
There still seems to be a way to go now before the site is ready for release but I’m really hopeful that the changes have been worth it and who knows may help revitalise the iSpot community.
This week I’ve added the following to the site:
- a location block on the observation page in place of the old location link.
- fixed the problem with the user defined carousel not loading
- an opt out checkbox in the user profile for email updates – defaulting to the user opting in
- a custom written module to allow a chosen node (page or story) to become a one-off pop up to display to the user when they visit the site after updates
- fixed some bugs that had come to light
- automated permission creation linked to the creation of a curator role for new communities
- fix the permission checking to facilitate all roles that show see the edit tab on the community home page to so!
Things are coming together now, although I’ve still got to make sure the correct context bar is shown in every situation. This is my next to do.
Rich L has been busy with many additional changes to the theme including:
- moving the ‘add observation’ button up onto the context bar for greater prominence
- moving the logo up into the top bar navigation for the smaller screen device views for better use of space
- various other tweaks that Rich himself can elaborate on if needed